stable diffusion ControlNet使用介绍与进阶技巧

ControlNet是什么？固定构图、定义姿势、描绘轮廓、单凭线稿就能生成一张丰满精致的插画……它几乎无所不能。

有人把它称为AI绘画界的“革命性”突破，但在我看来，它不过是StableDiffusion迈向“工业化”的第一步。

ControlNet扩展与模型下载地址

扩展地址：

https://github.com/Mikubill/sd-webui-controlnet

1.1版本模型地址：

（下载地址）https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main

（模型介绍）https://github.com/lllyasviel/ControlNet-v1-1-nightly

1.14版本后更新的新模型（包括社区模型）与XL模型的地址：https://huggingface.co/lllyasviel/sd_control_collection/tree/main

*模型下载后，一般放置于stable-diffusion-webui（根目录）\extensions\sd-webui-controlnet\models or stable-diffusion-webui（根目录）\models\ControlNet

预处理器下载地址（缺东西在里面找）：

https://huggingface.co/lllyasviel/Annotators/tree/main

*预处理器下载后，一般放置于stable-diffusion-webui（根目录）\extensions\sd-webui-controlnet\annotator\downloads

生成信息

注：生成信息也可以读取到ControlNet，但无法直接一键应用。

你可以从最后这些字段中，查看这张图片用了几个ControlNet，权重、引导步数分别是多少。

转载来源：https://nenly.notion.site/10-ControlNet-a802ab59c48d45f8ae4ad1cfea0a0d4d

补充

AI二维码+艺术字+光影光效+创意Logo生成方法

艺术字实现介绍

在ps中制作一个白底黑字的图片

拉入到ControlNet

生成参数参考

stone, rock, coal,sand,from above,
high quality, highres, masterpiece, solid background
Negative prompt: (worst quality:2),(low quality:2),(normal quality:2),lowres,watermark,monochrome
Steps: 20, Sampler: Euler, Schedule type: Automatic, CFG scale: 7, Seed: 2109960192, Size: 960x540, Model hash: 038ba203d8, Model: 二次元：AbyssOrangeMix2_sfw, ControlNet 0: "Module: depth_midas, Model: control_v11f1p_sd15_depth [cfd03158], Weight: 1, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", Version: 1.9.4

使用不同的模型,logo，ControlNet预处理和提示词（不同材质）可以生成不同风格的图片

介绍图片融字实现

两个新模型：brightness（亮度模型） & illumination（明度模型）

模型下载链接（来自nenly网盘）：https://pan.baidu.com/share/init?surl=0rzgzIjzad7AKmj-w8zO_w&pwd=nely

修改ControlNet参数如下（这里用invert是交换黑白部分）

生成效果

参数

gorgeous lips, cinematic, (masterpiece), (best quality), (ultra-detailed), very aesthetic, illustration, perfect composition, intricate details, absurdres, detailed face, (anime, masterpiece, intricate:1.3), (best quality, hires textures, high detail:1.2), (4k),(incredibly detailed:1.4)
Masterpiece, highest quality, high resolution, cinematic lighting, anime illustration, (rabbit ears), rabbit ears, ear fluff, girl looking up at the viewer, girl looking back in the forest, angle looking down on girl, bust shot, solo, angle looking down from above, hairs between eyes, long sideburns, red eyes, long hair, silver hair, hair swaying in the wind, petite, ((short)), young, red ribbon on chest,
BREAK
White maid outfit, maid headband, puff sleeves, short sleeves, shadow on chest, long skirt, frills,
BREAK
Birch forest, basket, sunlight filtering through the trees, flowers, white flowers, background of square in forest, detailed background, lens flare, chromatic aberration, depth of field,
Negative prompt: loli, child, longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, worst quality, low quality, normal quality, watermark, artist name, signature
Steps: 30, Sampler: Euler, Schedule type: Automatic, CFG scale: 8, Seed: 4076082077, Size: 960x540, Model hash: 038ba203d8, Model: 二次元：AbyssOrangeMix2_sfw, VAE hash: df3c506e51, VAE: kl-f8-anime2.ckpt, ControlNet 0: "Module: invert (from white bg & black line), Model: control_v1p_sd15_brightness [5f6aa6ed], Weight: 0.5, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0.1, Guidance End: 0.8, Pixel Perfect: True, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", Version: 1.9.4

将文字添加在人物衣服上

例如这张图

Photo of a stunning country 22 year old woman, freckles, beauty spots, slightly curvy, flannel shirt open, winter hat, hard shadows, soft light, summer,full body,
Negative prompt: (deformed iris, deformed pupils, semi-realistic, cgi, 3d, render, sketch, cartoon, drawing, anime, mutated hands and fingers:1.4), (deformed, distorted, disfigured:1.3), poorly drawn, bad anatomy, wrong anatomy, extra limb, missing limb, floating limbs, disconnected limbs, mutation, mutated, ugly, disgusting, amputation
Steps: 8, Sampler: DPM++ 2M SDE, Schedule type: Exponential, CFG scale: 1.5, Seed: 1811433106, Size: 675x1013, Model hash: f47e942ad4, Model: realisticVisionV60B1_v51HyperVAE, VAE hash: df3c506e51, VAE: kl-f8-anime2.ckpt, Version: 1.9.4

拉入photostop，将文字放到合适的位置

好了之后隐藏人物图层，导出文字，再拉入ControlNet

先再unit0放入人物图片，开启openpose

再到unit1放入文字图片，开启invert

生成效果（这里我关掉了unit0，不知道为啥带上图片会发黄）

二维码

首先准备一张二维码这里我以本文网址为例

unit0

unit1

生成效果