当前位置:首页> AI教程> 用Stable Diffusion和ControlNet模型生成惊艳图片

用Stable Diffusion和ControlNet模型生成惊艳图片

释放双眼,带上耳机,听听看~!
了解如何使用Stable Diffusion和ControlNet模型以及美图秀秀生成惊艳图片。本教程将介绍环境部署、预备工作和效果展示。

原文地址:zaozaoliao.feishu.cn/wiki/L4Axwl…

「 效果展示 」

提示词 目标字样 原图 生成效果
This stunning image captures the beauty and innocence of youth, with vibrant colors and a serene seaside setting. High-definition photo of a breathtaking moment, perfect for any project that aims to evoke a sense of joy and tranquility. 用Stable Diffusion和ControlNet模型生成惊艳图片 用Stable Diffusion和ControlNet模型生成惊艳图片 用Stable Diffusion和ControlNet模型生成惊艳图片

「 预备工作 」

  1. 开发环境 – 基于 Linux

  • Ubuntu 20.04

  • CUDA 11.4

  • python 3.8

  • torch 1.11.0+cu113

  • Nvidia Tesla V100

用Stable Diffusion和ControlNet模型生成惊艳图片

  1. 图片素材

准备一张自定义尺寸字体图片,工具任选,譬如美图秀秀、PS、Word、PPTX 均可,注意字体需要黑底白字,预先排好字体位置,字体最好免费可商用。

这里用的在线美图秀秀,字体 GreatVikes,分辨率 512*512。

用Stable Diffusion和ControlNet模型生成惊艳图片

  1. 代码模型

StableDiffusion ControlNet
代码 AUTOMATIC1111/stable-diffusion-webui Mikubill/sd-webui-controlnet
模型 beautifulRealistic_v60.safetensors control_v1p_sd15_brightness.safetensors

SD 模型这里用的 Beautiful Realistic Asians,可根据个人需求选择,下载模型请移步大名鼎鼎的 C 站 civitai.com

ControlNetcontrol_v1p_sd15_brightness,若更换模型,需下载对应代码的模型,否则加载模型会报错(参考本文最后的错误集锦)。

用Stable Diffusion和ControlNet模型生成惊艳图片

「 环境部署 」

  1. StableDiffusion

代码: AUTOMATIC1111/stable-diffusion-webui

模型: beautifulRealistic_v60.safetensors

将下载好的 SD 模型放入 stable-diffusion-webui/models/Stable-diffusion 目录下即可

安装部署

# 代码下载
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git

# 一键安装
./webui.sh

打开 WebUI

网页输入 127.0.0.1:xxx,即可打开 webUI 界面

用Stable Diffusion和ControlNet模型生成惊艳图片

  1. ControlNet

代码: Mikubill/sd-webui-controlnet

模型: control_v1p_sd15_brightness.safetensors

将下载好的 ControlNet 模型放入/stable-diffusion-webui/models/ControlNet 即可。

安装部署

打开 WebUI 后,进入 Extensions -> Install from URL,输入 ControlNet 上面的代码链接,点击 Install 即可,安装好后重启 WebUI。

用Stable Diffusion和ControlNet模型生成惊艳图片

安装好后,即可看到 /stable-diffusion-webui/extensions 中多了 sd-webui-controlnet

用Stable Diffusion和ControlNet模型生成惊艳图片

「 图片生成 」

  1. 示例 1

用Stable Diffusion和ControlNet模型生成惊艳图片

Prompt

Highest quality, ultra-high definition, masterpieces, 8k quality, 1girl, very delicate features, very detailed eyes and mouth, long hair, curly hair, delicate skin, big eyes, white sweater, necklace, street, Sunshine, standing on the beach with a backdrop of blue skies and ocean waves. This stunning image captures the beauty and innocence of youth, with vibrant colors and a serene seaside setting. High-definition photo of a breathtaking moment, perfect for any project that aims to evoke a sense of joy and tranquility.
Negative prompt: blurry, low quality, pixelated, (unattractive), (grimy), (old woman), (dark), (out of focus), (unclear), (dirty), ((extra arm)), ((extra leg)), ((extra nose)), ((extra mouth)), (out of frame), (bad composition), (too bright), (too dark), ((extra flowers)), ((extra hand)), (poor lighting), (bad color grading), (red-eyed), (morphed face), (unnatural posture), (awkward pose), (frozen animation), (poorly photoshopped), (low-res), (bad framing), (insipid)

Parameters

StableDiffusion

  • checkpoint: beautifulRealistic_v60.safetensors

  • Sampling method: DPM++ SDE Karras

  • Sampling Steps: 30

ControlNet

  • Enable

  • Control Type: All

  • Model: control_v1p_sd15_brightness.safetensors

  • Control Weight: 0.4

  • Ending Control Step: 0.6

用Stable Diffusion和ControlNet模型生成惊艳图片用Stable Diffusion和ControlNet模型生成惊艳图片

用Stable Diffusion和ControlNet模型生成惊艳图片用Stable Diffusion和ControlNet模型生成惊艳图片

  1. 示例 2

用Stable Diffusion和ControlNet模型生成惊艳图片

Prompt

Highest quality, ultra-high definition, masterpieces, 8k quality, A man wearing a black jacket and jeans, very delicate features, very detailed eyes and mouth, with a black hat and sunglasses, holding a cigarette in his right hand and his left hand in his pocket, standing on the beach with blue skies and sea waves in the background. High-quality photo of a rugged man enjoying a smoke break by the sea, capturing the sense of freedom and relaxation. Shot in high definition, this image is perfect for any project that requires a cool and laid-back vibe.
Negative prompt: blurred, low quality, pixelated, (unattractive), (grimy), (old man), (dark), (out of focus), (unclear), (dirty), ((extra arm)), ((extra leg)), ((extra nose)), ((extra mouth)), (out of frame), (bad composition), (too bright), (too dark), ((extra cigarette)), ((extra hand)), (poor lighting), (bad color grading), (red-eyed), (morphed face), (unnatural posture), (awkward pose), (frozen animation), (poorly photoshopped), (low-res), (bad framing), (insipid)

Parameters

StableDiffusion

  • checkpoint: beautifulRealistic_v60.safetensors

  • Sampling method: DPM++ SDE Karras

  • Sampling Steps: 30

ControlNet

  • Enable

  • Control Type: All

  • Model: control_v1p_sd15_brightness.safetensors

  • Control Weight: 0.4

  • Ending Control Step: 0.6

用Stable Diffusion和ControlNet模型生成惊艳图片用Stable Diffusion和ControlNet模型生成惊艳图片

  1. 一些小 Tips

  • 正反提示词:复制 prompt 到第一格空格后,直接点击右侧蓝色小箭头,即可自动拆分正反提示词;

用Stable Diffusion和ControlNet模型生成惊艳图片

用Stable Diffusion和ControlNet模型生成惊艳图片

  • Control Weight:使用 ControlNet 权重,建议 0.4-0.7,参数越大,自定义的图片越明显;
Control Weight 0.4 0.6 0.8
Result 用Stable Diffusion和ControlNet模型生成惊艳图片 用Stable Diffusion和ControlNet模型生成惊艳图片 用Stable Diffusion和ControlNet模型生成惊艳图片
  • Ending Control Step:ControlNet 终止介入的时机,建议 0.55-0.75。
Ending Control Step 0.4 0.6 0.9
Result 用Stable Diffusion和ControlNet模型生成惊艳图片 用Stable Diffusion和ControlNet模型生成惊艳图片 用Stable Diffusion和ControlNet模型生成惊艳图片

「 错误集锦 」

  1. 执行 webui.sh 会安装虚拟环境,若报 No module named ‘importlib.metadata’ ,是因为系统不支持 python3.7,将 webui.sh 47 行改成 python3.8 或者 3.10 即可
# webui.sh 47行
# python3 executable
if [[ -z "${python_cmd}" ]]
then
    python_cmd="python3.8"
fi
  1. 若前面修改为 python3.8,会导致 network.py 不支持 Network | None 用法,需载入 Union,且修改 103 行
# network.py的103行
from typing import Union
class ModuleType:
    def create_module(self, net: Network, weights: NetworkWeights) -> Union[Network, None]:#Network | None:
        return None
  1. 虚拟环境默认安装 torch2.0,修改 launch_utils.py 的 273 行,以安装与 cuda 配套的指定版本,系统默认安装 torch 2.0.0,我本地为 cuda11.4,所以这里安装 torch==1.11.0+cu113
# launch_utils.py的273行
torch_command = os.environ.get('TORCH_COMMAND', f"pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html")
  1. OSError: libtorch_cuda_cpp.so: cannot open shared object file: No such file or directory,注意 torch 版本

  2. 若报下面这一坨问题,大概率是 ControlNet 模型不匹配问题

huggingface.co/ioclab/ioc-…

ERROR: ControlNet cannot find model config [/XXX/models/ControlNet/diffusion_pytorch_model.yaml] 
ERROR: ControlNet will use a WRONG config to load your model. 
ERROR: The WRONG config may not match your model. The generated results can be bad. 
ERROR: You are using a ControlNet model [diffusion_pytorch_model] without correct YAML config file. 
ERROR: The performance of this model may be worse than your expectation. 
ERROR: If this model cannot get good results, the reason is that you do not have a YAML file for the model. 

Hint: You can take a look at [/XXX/extensions/sd-webui-controlnet/models] to find many existing YAML files.
*** Error running process: /XXX/extensions/sd-webui-controlnet/scripts/controlnet.py
    Traceback (most recent call last):
        raise RuntimeError('Error(s) in loading state_dict for {}:nt{}'.format(
    RuntimeError: Error(s) in loading state_dict for ControlNet:
        Missing key(s) in state_dict: "time_embed.0.weight", "time_embed.0.bias", "time_embed.2.weight", 
本网站的内容主要来自互联网上的各种资源,仅供参考和信息分享之用,不代表本网站拥有相关版权或知识产权。如您认为内容侵犯您的权益,请联系我们,我们将尽快采取行动,包括删除或更正。
AI教程

AI的基础知识:了解强弱人工智能和机器学习与深度学习的区别

2023-11-20 9:53:14

AI教程

如何使用Postman和OpenAI的API进行文本生成

2023-11-20 10:05:14

个人中心
购物车
优惠劵
今日签到
有新私信 私信列表
搜索