+hE SSKJrJrJr SSKrSSKrSSKJr SSK J r J r J r SSK Jr SSKJrJrJr SSKJr S S KJr S S KJr S S KJr S S KJrJrJr \"5(a SSKJ s J!r" Sr#OSr#\RH"\%5r&Sr'"SS\\5r(g))ListOptionalUnionN) CLIPTokenizer) AutoencoderKLControlNetModelUNet2DConditionModel) PNDMScheduler)is_torch_xla_availableloggingreplace_example_docstring) randn_tensor)BlipImageProcessor)Blip2QFormerModel)ContextCLIPTextModel)DeprecatedPipelineMixinDiffusionPipelineImagePipelineOutputTFa Examples: ```py >>> from diffusers.pipelines import BlipDiffusionControlNetPipeline >>> from diffusers.utils import load_image >>> from controlnet_aux import CannyDetector >>> import torch >>> blip_diffusion_pipe = BlipDiffusionControlNetPipeline.from_pretrained( ... "Salesforce/blipdiffusion-controlnet", torch_dtype=torch.float16 ... ).to("cuda") >>> style_subject = "flower" >>> tgt_subject = "teapot" >>> text_prompt = "on a marble table" >>> cldm_cond_image = load_image( ... "https://huggingface.co/datasets/ayushtues/blipdiffusion_images/resolve/main/kettle.jpg" ... ).resize((512, 512)) >>> canny = CannyDetector() >>> cldm_cond_image = canny(cldm_cond_image, 30, 70, output_type="pil") >>> style_image = load_image( ... "https://huggingface.co/datasets/ayushtues/blipdiffusion_images/resolve/main/flower.jpg" ... ) >>> guidance_scale = 7.5 >>> num_inference_steps = 50 >>> negative_prompt = "over-exposure, under-exposure, saturated, duplicate, out of frame, lowres, cropped, worst quality, low quality, jpeg artifacts, morbid, mutilated, out of frame, ugly, bad anatomy, bad proportions, deformed, blurry, duplicate" >>> output = blip_diffusion_pipe( ... text_prompt, ... style_image, ... cldm_cond_image, ... style_subject, ... tgt_subject, ... guidance_scale=guidance_scale, ... num_inference_steps=num_inference_steps, ... neg_prompt=negative_prompt, ... height=512, ... width=512, ... ).images >>> output[0].save("image.png") ``` c#^\rSrSrSrSrSrS(S\S\S\ S\ S \ S \ S \ S \S \S\\S\\4U4SjjjrSrS)SjrS*SjrS*SjrS+Sjr\R2"5\"\5S,S\\S\R<R<S\R<R<S\\S\\S\\R@S\S\S\S\S \\!\RD\\RD4S!\\S"\S#\S$\\S%\#4 S&jj55r$S'r%U=r&$)-BlipDiffusionControlNetPipeline[a Pipeline for Canny Edge based Controlled subject-driven generation using Blip Diffusion. This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods the library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.) Args: tokenizer ([`CLIPTokenizer`]): Tokenizer for the text encoder text_encoder ([`ContextCLIPTextModel`]): Text encoder to encode the text prompt vae ([`AutoencoderKL`]): VAE model to map the latents to the image unet ([`UNet2DConditionModel`]): Conditional U-Net architecture to denoise the image embedding. scheduler ([`PNDMScheduler`]): A scheduler to be used in combination with `unet` to generate image latents. qformer ([`Blip2QFormerModel`]): QFormer model to get multi-modal embeddings from the text and image. controlnet ([`ControlNetModel`]): ControlNet model to get the conditioning image embedding. image_processor ([`BlipImageProcessor`]): Image Processor to preprocess and postprocess the image. ctx_begin_pos (int, `optional`, defaults to 2): Position of the context token in the text encoder. z0.33.1z qformer->text_encoder->unet->vae tokenizer text_encodervaeunet schedulerqformer controlnetimage_processor ctx_begin_posmeanstdc n>[T U]5 URUUUUUUUUS9 URXU S9 g)N)rrrrrrr r!)r"r#r$)super__init__register_modulesregister_to_config) selfrrrrrrr r!r"r#r$ __class__s {/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/pipelines/controlnet/pipeline_controlnet_blip_diffusion.pyr'(BlipDiffusionControlNetPipeline.__init__zsP  %!+   mCPc"URXSS9$)NF) image_input text_input return_dict)r)r* input_image src_subjects r,get_query_embeddings4BlipDiffusionControlNetPipeline.get_query_embeddingss|| Y^|__r.c /n[X5HJupgSUSUR53nURSRU/[ X4-5-55 ML U$)Nza  z, )zipstripappendjoinint)r*prompts tgt_subjectsprompt_strength prompt_repsrvprompt tgt_subjects r, _build_prompt-BlipDiffusionControlNetPipeline._build_prompts] #&w#= F+a '78F IIdii3/L+M MN O$>  r.c XX44n [U[5(a*[U5U:wa[S[U5SUS35eUc [ XXeS9nOUR XeS9nXR R-nU$)Nz/You have passed a list of generators of length z+, but requested an effective batch size of z@. Make sure the batch size matches the length of the generators.) generatordevicedtyperIrJ) isinstancelistlen ValueErrorrtorinit_noise_sigma) r* batch_size num_channelsheightwidthrJrIrHlatentsshapes r,prepare_latents/BlipDiffusionControlNetPipeline.prepare_latentss69 i & &3y>Z+GA#i.AQR&<'gi  ?"5fZGjjj>   * "V* "''* 223j@ ++&00'',    r.c URRUX#S.SSSSS9SRU5nURSn U S:XaUn OUn UR U SS 9nURXgS 9nU(a[ R "U/S -5nU$) N)rUrTTFr\)size do_rescaledo_center_crop do_normalizer_ pixel_valuesr)dimrKr)r! preprocessrPrWrepeat_interleavetorchcat) r*imagerUrTrRnum_images_per_promptrIrJdo_classifier_free_guidanceimage_batch_size repeat_bys r,prepare_control_image5BlipDiffusionControlNetPipeline.prepare_control_images$$//  3  0  "V* !;;q> q "I.I'' q'94 &IIugk*E r.rCreference_imagecondtioning_imagesource_subject_categorytarget_subject_categoryrVguidance_scalerTrUnum_inference_stepsrH neg_promptr@rA output_typer2c bURnURRX RRURR SS9SnUR U5n[U[5(aU/n[U[5(aU/n[U[5(aU/n[U5nURUUU US9nURX$5nURUUU5nUS:nU(aURRRRnUR!U /U-SUSS9nURUR"R U5SS 9S n[$R&"UU/5nS [UR(RR*5S - -nUR-UUR(RR.UU-U U-U UUR(R0US 9n0nUR2R4"U 40UD6 UR7UU UUS UUR8R0US9nR=UR2R>55HunnUS:nU(a[$R&"U/S -5OUnUR9UUUUSS9unn UR)UUUUU S9Sn!U(aU!RAS 5un"n#U"UU#U"- --n!UR2RCU!UU5Sn[D(dM[FRH"5 M URJRMX`RJRRN- SS9S n$URRQU$US9n$URS5 U(dU$4$[UU$S9$)a Function invoked when calling the pipeline for generation. Args: prompt (`List[str]`): The prompt or prompts to guide the image generation. reference_image (`PIL.Image.Image`): The reference image to condition the generation on. condtioning_image (`PIL.Image.Image`): The conditioning canny edge image to condition the generation on. source_subject_category (`List[str]`): The source subject category. target_subject_category (`List[str]`): The target subject category. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will ge generated by random sampling. guidance_scale (`float`, *optional*, defaults to 7.5): Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://huggingface.co/papers/2207.12598). `guidance_scale` is defined as `w` of equation 2. of [Imagen Paper](https://huggingface.co/papers/2205.11487). Guidance scale is enabled by setting `guidance_scale > 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. height (`int`, *optional*, defaults to 512): The height of the generated image. width (`int`, *optional*, defaults to 512): The width of the generated image. seed (`int`, *optional*, defaults to 42): The seed to use for random generation. num_inference_steps (`int`, *optional*, defaults to 50): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. neg_prompt (`str`, *optional*, defaults to ""): The prompt or prompts not to guide the image generation. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). prompt_strength (`float`, *optional*, defaults to 1.0): The strength of the prompt. Specifies the number of times the prompt is repeated along with prompt_reps to amplify the prompt. prompt_reps (`int`, *optional*, defaults to 20): The number of times the prompt is repeated along with prompt_strength to amplify the prompt. Examples: Returns: [`~pipelines.ImagePipelineOutput`] or `tuple` r\) image_mean image_stdr_rr)r>r?r@rA?r[)r]r[r_N)r`rarrrs)rRrSrTrUrHrVrJrI)ryrUrTrRrzrIrJr{F)encoder_hidden_statescontrolnet_condr2)timesteprdown_block_additional_residualsmid_block_additional_residualsample prev_sample)r2)r)images)+rbr!rurdr#r$rPrLstrrNrEr5rkrrcrerr`rwrxrblock_out_channelsrX in_channelsrJr set_timestepsr~r enumerate progress_bar timestepschunkstep XLA_AVAILABLExm mark_steprdecodescaling_factor postprocessmaybe_free_model_hooksr)%r*rCrrrrrVrrTrUrrHrr@rArr2rIrRrgrjr{r[ uncond_inputuncond_embeddingsscale_down_factorextra_set_kwargs cond_imageitlatent_model_inputdown_block_res_samplesmid_block_res_sample noise_prednoise_pred_uncondnoise_pred_textrys% r,__call__(BlipDiffusionControlNetPipeline.__call__sL''..99  (8(8DKKOOdh: *,,V4 fc " "XF -s 3 3'>&? # -s 3 3'>&? #[ ##0+# $  00Z ,,\66J&4s&:# &**55<<TTJ>> z)$%# *L !% 1 1&0033F;#!2!! $ii):O(LMO#dii&6&6&I&I"JQ"NO&&!))55..,,))//'   $$%8M '=XG9q=!9^e ;???"&5 *!  Q Q+Q Q # Q ! Q#Q$Q,QQ5kQ%[QQ8`"H%*"H ]]_12+/ ##%MQ$&!$%* #u1S u1u199?? u1 "&c u1 "&c u1%,,'u1u1u1u1!u1E%//43H"HIJu1SMu1u1u1 c]!u1"#u13u1r.r))typingrrr PIL.Imagerrw transformersrmodelsrr r schedulersr utilsr r rutils.torch_utilsr$blip_diffusion.blip_image_processingrblip_diffusion.modeling_blip2r blip_diffusion.modeling_ctx_cliprpipeline_utilsrrrtorch_xla.core.xla_modelcore xla_modelrr get_loggerrloggerrrrr.r,rs)( &JJ' .E=C\\))MM   H %+\P1&=?PP1r.