adding lora support to z-image controlnet pipelines

[christopher5106]

If you don't see any problem having Lora support for Controlnet pipelines for Z-Image, here is the PR.

@sayakpaul

[sayakpaul]

Could you provide an example?

[christopher5106]

Example:

import torch
from diffusers import ZImageControlNetPipeline
from diffusers import ZImageControlNetModel
from huggingface_hub import hf_hub_download

controlnet = ZImageControlNetModel.from_single_file(
    hf_hub_download(
        "alibaba-pai/Z-Image-Turbo-Fun-Controlnet-Union",
        filename="Z-Image-Turbo-Fun-Controlnet-Union.safetensors",
    ),
    torch_dtype=torch.bfloat16,
)

pipe = ZImageControlNetPipeline.from_pretrained("Tongyi-MAI/Z-Image-Turbo", torch_dtype=torch.bfloat16, controlnet=controlnet)

pipe.load_lora_weights(
    "multimodalart/tarot-z-image-lora", weight_name="tarot-z-image.safetensors",
)

[asomoza]

thanks!