this post was submitted on 11 Aug 2023
7 points (100.0% liked)

Stable Diffusion

4338 readers
8 users here now

Discuss matters related to our favourite AI Art generation technology

Also see

Other communities

founded 2 years ago
MODERATORS
 

Basically the title. I'm looking for a way to start generating an image, use ControlNet halfway though the generation to detect the pose and stabilize it for the remaining steps. Is there any way to do this? Any help would be appreciated!

top 6 comments
sorted by: hot top controversial new old
[–] jasparagus 2 points 1 year ago* (last edited 1 year ago) (1 children)

There's a way to do this in Auto1111 (sort of):

  1. generate an image with part of your steps
  2. Enable openpose
  3. Add the partially generated pixel image
  4. Set controlnet to start at the halfway point, etc.
  5. re-generate the image with the same settings

This feels pretty janky, though. I think you could do it better (and in one shot) in comfyUI by processing the partially generated latent, feeding that result to a controlnet preprocessor node, then adding the resulting controlnet conditioning plus the original half-finished latent to a new ksampler node. You'd then finish generation (continuing from the original latent) at whatever step you split off.

[–] [email protected] 1 points 1 year ago

Thanks for the suggestion! I will give ComfyUI a try.

[–] [email protected] 1 points 1 year ago (1 children)

Isn't there a starting step/ending step parameter you can just use on controlnet with Autmatic1111? It uses percentage of the generation so just set the start to .5.

[–] [email protected] 1 points 1 year ago (1 children)

Yeah, that's what I tried first, but it won't work without an image or pose already loaded into the ControlNet tab.

[–] [email protected] 2 points 1 year ago (1 children)

oooh now I see the conundrum. You are basically trying to find a way to sort of bake openpose into the image generation process.... yeah I'm not sure how to do that other than what was suggested below or coding an extension for it.

[–] [email protected] 1 points 1 year ago

Yeah, that's exactly it.