+htSSKrSSKJrJrJrJrJrJrJr SSK r SSK J r J r J r SSKJr SSKJr SSKJrJr SSKJrJrJr SS KJr SS KJr SS KJr SS KJ r J!r!J"r"J#r#J$r$J%r%J&r& SS K'J(r( SSK)J*r*J+r+J,r, SSK-J.r. \""5(a SSK/J0s J1r2 Sr3OSr3\#Rh"\55r6Sr7"SS\*\,\\5r8g)N)AnyCallableDictListOptionalTupleUnion) CLIPTextModelCLIPTextModelWithProjection CLIPTokenizer)CLIPTextModelOutput)VaeImageProcessor)StableDiffusionLoraLoaderMixinTextualInversionLoaderMixin) AutoencoderKLPriorTransformerUNet2DConditionModel)get_timestep_embedding)adjust_lora_scale_text_encoder)KarrasDiffusionSchedulers)USE_PEFT_BACKEND deprecateis_torch_xla_availableloggingreplace_example_docstringscale_lora_layersunscale_lora_layers) randn_tensor)DiffusionPipelineImagePipelineOutputStableDiffusionMixin)StableUnCLIPImageNormalizerTFa Examples: ```py >>> import torch >>> from diffusers import StableUnCLIPPipeline >>> pipe = StableUnCLIPPipeline.from_pretrained( ... "fusing/stable-unclip-2-1-l", torch_dtype=torch.float16 ... ) # TODO update model path >>> pipe = pipe.to("cuda") >>> prompt = "a photo of an astronaut riding a horse on mars" >>> images = pipe(prompt).images >>> images[0].save("astronaut_horse.png") ``` c0H^\rSrSr%SrSS/rSr\\S'\ \S'\ \S'\ \S'\ \S'\ \S '\\S '\ \S '\\S '\ \S '\\S'S\S\ S\ S\ S\ S \ S \S \ S \S \ S\4U4SjjrS7S\\\\4S\\R,4SjjrS8S\\R,S\\R,S\\4SjjrS9S\\R,S\\R,S\\S\\4SjjrSrSrSrS:SjrSr S7S\R,S \S!\\R,S"\\RB4S#jjr"\RF"5\$"\%5S;S$\\\&\'\&4S%\\S&\\S'\S(\S)\\\&\'\&4S*\\S+\S"\\RBS,\\R,S\\R,S\\R,S-\\&S.\(S/\\)\\\R,/S4S0\S1\\*\&\+4S \S2\S3\S4\\R,S\\4,S5jj55r,S6r-U=r.$)<StableUnCLIPPipelineFa Pipeline for text-to-image generation using stable unCLIP. This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods implemented for all pipelines (downloading, saving, running on a particular device, etc.). The pipeline also inherits the following loading methods: - [`~loaders.TextualInversionLoaderMixin.load_textual_inversion`] for loading textual inversion embeddings - [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_weights`] for loading LoRA weights - [`~loaders.StableDiffusionLoraLoaderMixin.save_lora_weights`] for saving LoRA weights Args: prior_tokenizer ([`CLIPTokenizer`]): A [`CLIPTokenizer`]. prior_text_encoder ([`CLIPTextModelWithProjection`]): Frozen [`CLIPTextModelWithProjection`] text-encoder. prior ([`PriorTransformer`]): The canonical unCLIP prior to approximate the image embedding from the text embedding. prior_scheduler ([`KarrasDiffusionSchedulers`]): Scheduler used in the prior denoising process. image_normalizer ([`StableUnCLIPImageNormalizer`]): Used to normalize the predicted image embeddings before the noise is applied and un-normalize the image embeddings after the noise has been applied. image_noising_scheduler ([`KarrasDiffusionSchedulers`]): Noise schedule for adding noise to the predicted image embeddings. The amount of noise to add is determined by the `noise_level`. tokenizer ([`CLIPTokenizer`]): A [`CLIPTokenizer`]. text_encoder ([`CLIPTextModel`]): Frozen [`CLIPTextModel`] text-encoder. unet ([`UNet2DConditionModel`]): A [`UNet2DConditionModel`] to denoise the encoded image latents. scheduler ([`KarrasDiffusionSchedulers`]): A scheduler to be used in combination with `unet` to denoise the encoded image latents. vae ([`AutoencoderKL`]): Variational Auto-Encoder (VAE) Model to encode and decode images to and from latent representations. priorimage_normalizerz+text_encoder->prior_text_encoder->unet->vaeprior_tokenizerprior_text_encoderprior_schedulerimage_noising_scheduler tokenizer text_encoderunet schedulervaec >[T U]5 URUUUUUUUUU U U S9 [USS5(a/S[ UR R R5S- -OSUl[URS9Ul g)N) r+r,r)r-r*r.r/r0r1r2r3r3r r$)vae_scale_factor) super__init__register_modulesgetattrlenr3configblock_out_channelsr6rimage_processor) selfr+r,r)r-r*r.r/r0r1r2r3 __class__s u/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_unclip.pyr8StableUnCLIPPipeline.__init__s$  +1+-$;%  W^^bdikoVpVpc$((//*L*L&MPQ&Q Rvw0$BWBWXNtext_model_outputtext_attention_maskcUGc[U[5(a [U5OSnURUSURRSSS9nUR n UR R5RU5n URUSSS9R n U RSU RS:a[R"X5(dURRU SS2URRS- S245n [RS URRS U 35 U SS2SURR24n URU RU55n U R nU R"nOUS RS nUS USpUn UR%US S 9nUR%US S 9nU R%US S 9n U(Ga\S /U-nURUSURRSSS9nUR R5RU5nURUR RU55nUR nUR"nURSnUR'SU5nUR)Xs-U5nURSnUR'SUS5nUR)Xs-US5nUR%US S 9n[R*"UU/5n[R*"UU/5n[R*"UU /5n XU 4$)Nr$ max_lengthTptpaddingrG truncationreturn_tensorslongestrJrL\The following part of your input was truncated because CLIP can only handle sequences up to tokens: r)dim) isinstancelistr;r+model_max_length input_idsattention_maskbooltoshapetorchequal batch_decodeloggerwarningr, text_embedslast_hidden_staterepeat_interleaverepeatviewcat)r?promptdevicenum_images_per_promptdo_classifier_free_guidancerDrE batch_size text_inputstext_input_ids text_maskuntruncated_ids removed_textprior_text_encoder_output prompt_embedstext_enc_hid_states uncond_tokens uncond_inputuncond_text_mask0negative_prompt_embeds_prior_text_encoder_outputnegative_prompt_embedsuncond_text_enc_hid_statesseq_lens rA_encode_prior_prompt)StableUnCLIPPipeline._encode_prior_prompts  $(264(@(@VaJ..$//@@# /K)22N#22779<i ~W"03ZT5I5I5Z5Z3Z0Z![(,(?(?@Q@QRX@Y(Z %5AAM";"M"M +1-33A6J1B11EGXYZG[.+I%778MST7U 1CCDY_`Ca//0E1/M &D:-M//$//@@# 0L ,::??ADDVL ?C?V?V&&))&1@ <&V%a%a ")Y)k)k &-2215G%;%B%B1F[%\ "%;%@%@Acel%m "066q9G)C)J)J1Ncef)g &)C)H)H2GR* & 0AABW]^A_ "II'=}&MNM"'))-GI\,]"^  #3Y"?@I9<Xl[(d[ UR U5 O[ UR U5 Ub[U[5(aSn O3Ub![U[5(a [U5n OURSn UGc[U[5(aURXR5nURUSURRSSS9n U Rn URUSSS 9Rn U RS U RS :a[ R""X5(djURR%U SS2URRS- S 245n[&R)S URRS U35 [+UR R,S 5(aAUR R,R.(aU R0R3U5nOSnU c%UR U R3U5US9nUSnOQUR U R3U5USS9nUS U S-*nUR R4R7U5nUR bUR R8nO0UR:bUR:R8nO UR8nUR3UUS9nURunnnUR=SUS5nUR?UU-US 5nU(GaUGc|UcS/U -nOUb;[AU5[AU5La$[CS[AU5S[AU5S35e[U[5(aU/nO2U [U5:wa![ESUS[U5SUSU S3 5eUn[U[5(aURUUR5nURSnURUSUSSS9n[+UR R,S 5(aAUR R,R.(aUR0R3U5nOSnUR URR3U5US9nUSnU(aGURSnUR3UUS9nUR=SUS5nUR?X-US 5nUR b6[U[5(a![(a[GUR U5 Xg4$)a, Encodes the prompt into text encoder hidden states. Args: prompt (`str` or `List[str]`, *optional*): prompt to be encoded device: (`torch.device`): torch device num_images_per_prompt (`int`): number of images that should be generated per prompt do_classifier_free_guidance (`bool`): whether to use classifier free guidance or not negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. If not defined, one has to pass `negative_prompt_embeds` instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, text embeddings will be generated from `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input argument. lora_scale (`float`, *optional*): A LoRA scale that will be applied to all LoRA layers of the text encoder if LoRA layers are loaded. clip_skip (`int`, *optional*): Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. Nr$rrGTrHrIrMrNrOrPrQuse_attention_mask)rX)rXoutput_hidden_states)dtyperhrS?`negative_prompt` should be the same type to `prompt`, but got  != .z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.)$rTr _lora_scalerrr0rstrrUr;r[rmaybe_convert_promptr/rVrWr\r]r^r_r`hasattrr<rrXrZ text_modelfinal_layer_normrr1rdretype TypeError ValueErrorr)r?rgrhrirjrrrrxr}rrkrlrmrorprXprompt_embeds_dtypebs_embedrz_rtrGrus rAr"StableUnCLIPPipeline.encode_prompt)sQV  !j7U&V&V) $#.t/@/@*M!$"3"3Z@  *VS"9"9J  Jvt$<$<VJ&,,Q/J  $ ;<<226>>J..$>>::# )K)22N"nnVYW[n\ffO$$R(N,@,@,DDU[[NN $~~::#At~~'F'F'JR'O$OP  778 ,Q t((//1EFF4K\K\KcKcKvKv!,!;!;!>!>v!F!% $ 1 1.2C2CF2K\j 1 k -a 0 $ 1 1"%%f-ncg!2! !.b 1IM2B C !% 1 1 < < M Mm \    ("&"3"3"9"9  YY ""&))// "/"5"5 %((/B6(R ,22'1%,,Q0EqI %**86K+KWVXY  '+A+I&!#z 1 #V D>$%# *Lt((//1EFF4K\K\KcKcKvKv!-! " &,2215G%;%>%>EXag%>%h "%;%B%B1F[]^%_ "%;%@%@Acelnp%q "    ($ >??DTDT#D$5$5zB44rCcTSn[SSUSS9 SURRR- U-nURR USS9SnUS - S -R SS5nUR 5RSS S S5R5R5nU$) Nz{The decode_latents method is deprecated and will be removed in 1.0.0. Please use VaeImageProcessor.postprocess(...) insteaddecode_latentsrFrr$ return_dictrr g?r) rr3r<scaling_factordecodeclampcpupermutefloatnumpy)r?latentsrimages rAr#StableUnCLIPPipeline.decode_latentss\"G-@PUVdhhoo444w>U;A>S''1- ##Aq!Q/557==? rCcnS[[R"URR5R R 55;n0nU(aX$S'S[[R"URR5R R 55;nU(aXS'U$Neta generator)setinspect signaturer-step parameterskeysr?rr accepts_etaextra_step_kwargsaccepts_generators rAprepare_prior_extra_step_kwargs4StableUnCLIPPipeline.prepare_prior_extra_step_kwargss s7#4#4T5I5I5N5N#O#Z#Z#_#_#abb  '*e $(3w/@/@AUAUAZAZ/[/f/f/k/k/m+nn -6k *  rCcnS[[R"URR5R R 55;n0nU(aX$S'S[[R"URR5R R 55;nU(aXS'U$r)rrrr2rrrrs rAprepare_extra_step_kwargs.StableUnCLIPPipeline.prepare_extra_step_kwargss s7#4#4T^^5H5H#I#T#T#Y#Y#[\\  '*e $(3w/@/@ATAT/U/`/`/e/e/g+hh -6k *  rCc rUS-S:wd US-S:wa[SUSUS35eUbUb6[U[5(aUS::a[SUS[U5S35eUbUb [S5eUcUc [S 5eUbA[U[5(d,[U[ 5(d[S [U535eUbUb [S 5eUb>Ub;[U5[U5La$[ S [U5S [U5S35eUbCUb@URUR:wa&[SURSURS35eUS:d#XPRRR:a0[SURRRS- S35eg)Nr5rz7`height` and `width` have to be divisible by 8 but are z and rz5`callback_steps` has to be a positive integer but is z of type z[Provide either `prompt` or `prompt_embeds`. Please make sure to define only one of the two.zeProvide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined.z2`prompt` has to be of type `str` or `list` but is zProvide either `negative_prompt` or `negative_prompt_embeds`. Cannot leave both `negative_prompt` and `negative_prompt_embeds` undefined.rrzu`prompt_embeds` and `negative_prompt_embeds` must have the same shape when passed directly, but got: `prompt_embeds` z != `negative_prompt_embeds` z$`noise_level` must be between 0 and r$z , inclusive.) rrTintrrrUrr[r.r<num_train_timesteps) r?rgheightwidthcallback_steps noise_levelrrrrxs rA check_inputs!StableUnCLIPPipeline.check_inputss# A:?eai1nVW]V^^cdicjjklm m  "  & >30O0OSaefSfGGWX(),   -";m  >m3w   z&#'>'>zRXZ^G_G_QRVW]R^Q_`a a  &+A+M\   /"=F|4#88UVZ[jVkUlmV ~Q(  $)?)K""&<&B&BB --:-@-@,AB.445Q8 ?k-I-I-P-P-d-dd6t7S7S7Z7Z7n7nqr7r6ssA erCcUc [XX2S9nOUc)[URXARURS9n[R "U/URS-URS9nUR RUR5 UR RU5nURRXUS9nUR RU5n[X!RSSSS9nURUR5n[R"X4S5nU$) a Add noise to the image embeddings. The amount of noise is controlled by a `noise_level` input. A higher `noise_level` increases the variance in the final un-noised images. The noise is applied in two ways: 1. A noise schedule is applied directly to the embeddings. 2. A vector of sinusoidal time embeddings are appended to the output. In both cases, the amount of noise is controlled by the same `noise_level`. The embeddings are normalized before the noise is applied and un-normalized after the noise is applied. rrrh) timestepsrrOT)r embedding_dimflip_sin_to_cosdownscale_freq_shiftr$)rr[rhrr\tensorr*rZscaler. add_noiseunscalerrf)r?rrrrs rAnoise_image_embeddings+StableUnCLIPPipeline.noise_image_embeddingsWs& = ""i@S@S[g[m[mEllK=<3E3Ea3H#HQ]QdQde    !4!45,,22<@ 33==lin=o ,,44\B ,!1C1CB1GY]tu "nn\%7%78 yy,!URRRUXRRRRV- S S9Sn1OU n1URXR[U1U S9n1UR]5 U(dU14$[_U1S9$)u The call function to the pipeline for generation. Args: prompt (`str` or `List[str]`, *optional*): The prompt or prompts to guide image generation. If not defined, you need to pass `prompt_embeds`. height (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`): The height in pixels of the generated image. width (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`): The width in pixels of the generated image. num_inference_steps (`int`, *optional*, defaults to 20): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. guidance_scale (`float`, *optional*, defaults to 10.0): A higher guidance scale value encourages the model to generate images closely linked to the text `prompt` at the expense of lower image quality. Guidance scale is enabled when `guidance_scale > 1`. negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts to guide what to not include in image generation. If not defined, you need to pass `negative_prompt_embeds` instead. Ignored when not using guidance (`guidance_scale < 1`). num_images_per_prompt (`int`, *optional*, defaults to 1): The number of images to generate per prompt. eta (`float`, *optional*, defaults to 0.0): Corresponds to parameter eta (η) from the [DDIM](https://huggingface.co/papers/2010.02502) paper. Only applies to the [`~schedulers.DDIMScheduler`], and is ignored in other schedulers. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor is generated by sampling using the supplied random `generator`. prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, text embeddings are generated from the `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, `negative_prompt_embeds` are generated from the `negative_prompt` input argument. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generated image. Choose between `PIL.Image` or `np.array`. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.ImagePipelineOutput`] instead of a plain tuple. callback (`Callable`, *optional*): A function that calls every `callback_steps` steps during inference. The function is called with the following arguments: `callback(step: int, timestep: int, latents: torch.Tensor)`. callback_steps (`int`, *optional*, defaults to 1): The frequency at which the `callback` function is called. If not specified, the callback is called at every step. cross_attention_kwargs (`dict`, *optional*): A kwargs dictionary that if specified is passed along to the [`AttentionProcessor`] as defined in [`self.processor`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py). noise_level (`int`, *optional*, defaults to `0`): The amount of noise to add to the image embeddings. A higher `noise_level` increases the variance in the final un-noised images. See [`StableUnCLIPPipeline.noise_image_embeddings`] for more details. prior_num_inference_steps (`int`, *optional*, defaults to 25): The number of denoising steps in the prior denoising process. More denoising steps usually lead to a higher quality image at the expense of slower inference. prior_guidance_scale (`float`, *optional*, defaults to 4.0): A higher guidance scale value encourages the model to generate images closely linked to the text `prompt` at the expense of lower image quality. Guidance scale is enabled when `guidance_scale > 1`. prior_latents (`torch.Tensor`, *optional*): Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for image embedding generation in the prior denoising process. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor is generated by sampling using the supplied random `generator`. clip_skip (`int`, *optional*): Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. Examples: Returns: [`~pipelines.ImagePipelineOutput`] or `tuple`: [`~ pipeline_utils.ImagePipelineOutput`] if `return_dict` is True, otherwise a `tuple`. When returning a tuple, the first element is a list with the generated images. )rgrrrrrrrrxNr$rg?)rgrhrirjrr )timestepproj_embeddingencoder_hidden_statesrX)rsamplerFr) rgrhrirjrrrrxr}r)rrr)r[rrhrrr2)r class_labelsrrorderlatentr)r)images)0r1r< sample_sizer6rrTrrUr;r[_execution_devicer{r- set_timestepsrr)rrrr enumerate progress_barr\rfscale_model_inputpredicted_image_embeddingchunkrpost_process_latentsgetrr zeros_liker2 in_channelsrrr: XLA_AVAILABLExm mark_stepr3rrr> postprocessmaybe_free_model_hooksr")2r?rgrrrrrrirrrrrrxrrrrrrrrrrrkrh!prior_do_classifier_free_guidanceprior_prompt_embeds prior_text_encoder_hidden_statesprior_text_maskprior_timesteps_tensorrprior_extra_step_kwargsitlatent_model_inputr predicted_image_embedding_uncondpredicted_image_embedding_textrrjtext_encoder_lora_scalernum_channels_latentsr[r noise_prednoise_pred_uncondnoise_pred_textstep_idxrs2 rA__call__StableUnCLIPPipeline.__call__sPO499++77$:O:OOM))558M8MM )#+'#9    *VS"9"9J  Jvt$<$<VJ&,,Q/J"77 '' -A3,F)RVQjQj"7(I RkR N= **+DV*T!%!5!5!?!? ))77 ,,  '  % %      #'"F"Fy"Vd//0FGHDAqCdM?Q+>!?jw !%!5!5!G!GHZ\]!^ (, "2&F. )3) (' &1SlSrSrstSuP02P,LOc25UUP-)!0055)$*  "   M#N(:a(?A}-9I< 77 F $ '5s&:#:P9[ " & &w 5ae 150B0B"7(C+'#9.1C 1 -  '!II'=&MNM22%#3 '%*%5%5l%C " !99&r6)NN)NNNN)NNNNN)NNN)NNNg$@Nr$gNNNNpilTNr$Nrg@NN)/__name__ __module__ __qualname____firstlineno____doc___exclude_from_cpu_offloadmodel_cpu_offload_seqr __annotations__r rrr%r rrr8rr r rr\Tensorr{rrrrrrrrr Generatorrno_gradrEXAMPLE_DOC_STRINGrrrYrrrr__static_attributes__ __classcell__)r@s@rAr'r'Fs&$L")*< =I#"33 ..2166 (( #Y'#Y8 #Y  #Y 3 #Y6#Y";#Y!#Y$#Y##Y-#Y !#YXJN6:[= $E*=u*D$EF [=&ell3[=H049=&*  -!) 6UON049=&*#'t5  -t5!) 6t5UOt5C=t5n !$!0#:z )-/3 ,ll,, % , EOO, ,\ ]]_1237 $##% $;?/0/3*.049=%* GK;?)+&)04#'3q1sDI~./q1 q1 } q1 ! q1q1"%T#Y"78q1 (}q1q1EOO,q1%,,'q1  -q1!) 6q1c]q1 !q1"8S#u||$rloadersrrmodelsrrrmodels.embeddingsr models.lorar schedulersrutilsrrrrrrrutils.torch_utilsrpipeline_utilsr!r"r#stable_unclip_image_normalizerr%torch_xla.core.xla_modelcore xla_modelrr get_loggerrr_rr'rrCrAr2sDDD RRF0RKK793.YYG))MM   H %$r 1+-HJhr 1rC