怎么训出穿衣服脱衣服的lora

0 / 1529

最近知乎有人介绍了这个lora: Clothing+- ,收藏量 已经上万,跑去看模型作者的页面,真心厉害。

(文末附有 链接)

这个lora真的效果惊人:

watermark

watermark
6ac5041583b8cff64940f883ea0f0c9e.png

lora:ClothingAdjuster3:-1, 1man palying basketball, masterpiece,best quality,ultra high res,realistic skin, (chicken), ((Braces trousers)), white background, simple background

知乎里面展示的效果

要牢记住正负数,正数是减衣服,负数是加衣服,所以我们用负数。

直接用-1看一下效果。

为了维持角色外观基本不变,我使用了ControlNet插件的canny

(masterpiece:1.4, best quality), (intricate details), unity 8k wallpaper, ultra detailed, beautiful and aesthetic, perfect lighting, (1girl), (blue hair, blue eyes, medium breasts),, dynamic pose, dynamic angle, lipstick, slim, slim body, medium breasts, , detailed background, realistic, solo, perfect detailed face, detailed eyes, highly detailed, blush, hair ornament, rolling_eyes, cross-eyed,<lora:Clothing+-:-1>

基本上该遮住的地方,都遮住了,符合我们想象中的效果(真的这么想吗?)

把权重调整到-0.8:

继续把权重调整到-0.5

训练方法

如何制作与“Clothing +/- Adjuster”类似的LoRA:

本模型受青龙大佬在此视频中所介绍的第一种方法“复印学习法”启发。我进行了一些改进以实现多张图情况下的批量训练。具体的步骤如下:

步骤一:挑选N组不同人物的状态A与状态B对比图片,并形成状态A图像训练集与状态B图像训练集。保证相同人物的状态A与状态B两张图片的文件名相同。

步骤二:对状态A训练集添加txt标签,每个图片只打一个可以区分不同人物的特殊词汇标签。比如有10个人物,那就给每个人物从jinitaimei1至jinitaimei10分配各自的标签。然后将状态A训练集的所有标签复制粘贴进状态B图像训练集中。

步骤三:选择与训练集画风相近的底模C,使用状态A图像训练集进行Lora训练直至模型过拟合,输入人物N的对应标签后,只能生成人物N的状态A照片。

步骤四:将训练得到的过拟合LoRA模型以1.0的比例融合进底模C中(更新:经进一步测试,勾选上same to strength效果会更好),然后用状态B图像训练集基于新底模进行Lora训练。该训练过程不一定要训练至严重过拟合,可以选择LoRA过程文件进行AI绘图测试,只要能通过调节权重,实现状态A至状态B的过渡即可。(更新:经进一步测试,甜蜜点大概在每张图片400在800步范围内)

步骤五:如果训练的LoRA所涉及场景较复杂,在高权重下会出现过拟合的情况。有两种改良的建议:一是进行LoRA分层调节,降低LoRA中与A/B状态切换无关的层数的权重;二是压缩LoRA的维度,比如从64压缩至4。

How to make a LoRA similar to "Clothing +/- Adjuster":

This model is inspired by the first method "Copy Learning" introduced by Qinglong in this video. I made some improvements to implement batch training in the case of multiple images. The specific steps are as follows:

Step 1: Select N groups of comparison images of different characters in State A and State B. Put them into the State A image training set and the State B image training set, respectively. Please ensure that the file names of the two images of State A and State B for the same character are the same.

Step 2: Add txt labels to the State A training set, with each image receiving only one unique word label that distinguishes different characters. For example, if there are 10 characters, assign each character a label from jinitaimei1 to jinitaimei10. Then copy and paste all labels from the State A training set into the State B image training set.

Step 3: Choose a base model C with a similar style to the training set. Use the State A image training set for Lora training until the model overfits.

Step 4: Merge the overfitted LoRA model obtained from training into the base model C at a ratio of 1.0 (Update: After further testing, it will be better to choose same to strength). Then, use the State B image training set for Lora training based on the new base model. This training process does not necessarily need to be trained until severe overfitting occurs; you can choose the LoRA process file for AI drawing tests, as long as the transition from State A to State B can be achieved by adjusting the weight. (Update: 400~800 steps for single pic)

Step 5: If the LoRA training involves complex scenes, overfitting may occur at high weights. There are two suggested improvements: one is to perform layer-by-layer adjustment of LoRA, reducing the weights of layers unrelated to the A/B state switch; the other is to compress the dimensions of LoRA, such as from 64 to 4.

除了这个LoRA之外,我的作品还包括:

  • FilmGirl/胶片风** Lora模型:如果你想增加你所绘制人物的真实感,这个Lora就是目前最佳的选择。**
  • MoonFilm** CheckPoint模型:可能是整个civitai上皮肤真实感Top5的写实模型。**
  • Pallas's cat/兔狲** LoRA模型:兔狲是这个世界最有趣的猫科动物,请将你的GPU算力献给可爱的猫猫。**
  • 拍立得/Polaroid** Lora模型:一个实现拍立得照片质感的LoRA,目前还在进一步改进中。**
  • EVA『新世纪エヴァンゲリオン』LoRA**模型:**可以生成类EVA风格泛用人型决战兵器的LoRA,适配EVA 00、EVA 01、EVA 02、EVA 08四款机型。

In addition to this LoRA, my works also include:

  • FilmGirl Film Style Lora Model: If you want to increase the realism of the characters you draw, this Lora is currently the best choice.
  • MoonFilm CheckPoint Model: Possibly one of the top 5 most realistic skin models on Civitai.
  • Pallas's Cat LoRA Model: Pallas's cat is the most interesting feline in the world. Please dedicate your GPU power to these adorable cats.
  • Polaroid Lora Model: A LoRA that achieves the texture of Polaroid photos, currently undergoing further improvements.
  • EVA "Neon Genesis Evangelion" LoRA Model: A LoRA that generates EVA-style general-purpose humanoid combat weapons, compatible with EVA 00, EVA 01, EVA 02, and EVA 08 models.

https://civitai.com/models/88132?modelVersionId=117151

知乎介绍