+hASSKJrJrJr SSKrSSKJrJrJrJ r J r SSK J r SSK Jr SSKJrJr SSKJrJr SS KJrJrJr SS KJr SS KJrJrJrJr S S K J!r!J"r" S SK#J$r$J%r%J&r&J'r' SSK J(r( \RR"\*5r+SS\RXS\\RZS\.4Sjjr/"SS\!5r0"SS\!5r1"SS\!5r2"SS\!5r3g))ListOptionalTupleN)CLIPImageProcessor CLIPTextModelCLIPTextModelWithProjection CLIPTokenizerCLIPVisionModelWithProjection) FrozenDict)ClassifierFreeGuidance)PipelineImageInputVaeImageProcessor) StableDiffusionXLLoraLoaderMixinTextualInversionLoaderMixin) AutoencoderKLImageProjectionUNet2DConditionModel)adjust_lora_scale_text_encoder)USE_PEFT_BACKENDloggingscale_lora_layersunscale_lora_layers)ModularPipelineBlocks PipelineState) ComponentSpec ConfigSpec InputParam OutputParam) StableDiffusionXLModularPipelineencoder_output generator sample_modec[US5(a!US:XaURRU5$[US5(a US:XaURR5$[US5(a UR$[ S5e)N latent_distsampleargmaxlatentsz3Could not access latents of provided encoder_output)hasattrr'r(moder*AttributeError)r#r$r%s r/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/modular_pipelines/stable_diffusion_xl/encoders.pyretrieve_latentsr//s}~}--+2I))00;;  / /K84K))..00  + +%%%RSSc\rSrSrSr\S\4Sj5r\S\\ 4Sj5r \S\\ 4Sj5r \S\\ 4Sj5r\SS j5rS r\R&"5S \S \S\4S j5rSrg)StableDiffusionXLIPAdapterStep<stable-diffusion-xlreturncg)NaIP Adapter step that prepares ip adapter image embeddings. Note that this step only prepares the embeddings - in order for it to work correctly, you need to load ip adapter weights into unet via ModularPipeline.load_ip_adapter() and pipeline.set_ip_adapter_scale(). See [ModularIPAdapterMixin](https://huggingface.co/docs/diffusers/api/loaders/ip_adapter#diffusers.loaders.ModularIPAdapterMixin) for more detailsselfs r. description*StableDiffusionXLIPAdapterStep.description?s   r0c [S[5[S[[SSS.5SS9[S[5[S[ [S S 05SS9/$) N image_encoderfeature_extractor)size crop_size from_configconfigdefault_creation_methodunetguiderguidance_scale@)rr rr rr r8s r.expected_components2StableDiffusionXLIPAdapterStep.expected_componentsIsc /+H I #"!3S"AB(5   &"6 7 &!#3S"9:(5    r0c$[S[SSS9/$)Nip_adapter_imageTz%The image(s) to be used as ip adapter)requiredr:)rrr8s r.inputs%StableDiffusionXLIPAdapterStep.inputs\s" ""C    r0cf[S[RSS9[S[RSS9/$)Nip_adapter_embedszIP adapter image embeddings type_hintr:negative_ip_adapter_embedsz$Negative IP adapter image embeddingsr torchTensorr8s r.intermediate_outputs3StableDiffusionXLIPAdapterStep.intermediate_outputsgs5 +u||Qn o ,,,B   r0Ncd[URR55Rn[ U[ R 5(dURUSS9RnURX%S9nU(aqURUSS9RSnURUSS9nUR[ R"U5SS9RSnURUSS9nXg4$URU5RnURUSS9n[ R"U5n X4$) Npt)return_tensorsdevicedtypeToutput_hidden_statesrdim)nextr= parametersr` isinstancerWrXr> pixel_valuesto hidden_statesrepeat_interleave zeros_like image_embeds) componentsimager_num_images_per_promptrbr`image_enc_hidden_statesuncond_image_enc_hidden_statesrnuncond_image_embedss r. encode_image+StableDiffusionXLIPAdapterStep.encode_imagersAZ--88:;AA%..00t0LYYE4 &0&>&>u[_&>&`&n&noq&r #&=&O&OPekl&O&m #-7-E-E  'd.F.mB. *.L-]-]%1.^. *+J J%33E:GGL'99:OUV9WL"'"2"2<"@ 4 4r0c /nU(a/nUGc'[U[5(dU/n[U5[URRR 5:waB[ S[U5S[URRR 5S35e[X!RRR 5Hiup[U [5(+n URXUSU 5upURU SSS245 U(dMQWRU SSS245 Mk OEUH?n U(a$U RS5upWRU 5 URU 5 MA /n[U5Hwup[R"U /U-SS9n U(a2[R"WU/U-SS9n [R"X/SS9n U RUS9n URU 5 My U$) NzK`ip_adapter_image` must have same length as the number of IP Adapters. Got z images and z IP Adapters.r!rrrd)r_)rhlistlenrFencoder_hid_projimage_projection_layers ValueErrorziprruappendchunk enumeraterWcatrj)r9rorMip_adapter_image_embedsr_rqprepare_unconditional_embedsrnnegative_image_embedssingle_ip_adapter_imageimage_proj_layeroutput_hidden_statesingle_image_embedssingle_negative_image_embedsis r.prepare_ip_adapter_image_embeds>StableDiffusionXLIPAdapterStep.prepare_ip_adapter_image_embedss; '$& ! " *.55$4#5 #$JOO,L,L,d,d(ee abefvbwaxyEFIJTJYJYJjJjJBJBFCEDDQR>A //"B"B"Z"Z>9'+55E*W&W#DHDUDUDWEA###$7a$@A//)001MdTUg1VW>(?#/H[HaHabcHdE0)001MN##$78 (? #%&/ &= "A"'))-@,ADY,Y_`"a +/4yy:OPQ:R9SVk9kqr/s,&+ii1M0cij&k#"5"8"8"8"G  # * *+> ?'>'&r0rostatec URU5nURRS:UlURUlUR UURSUR SURS9UlUR(aa/Ul [UR5HAupEURS5upeURRU5 XSRU'MC URX#5 X4$)Nr!)rMrr_rqrr)get_block_staterGnum_conditionsr_execution_devicer_rrMrRrUrrr~set_block_state)r9ror block_staterrnrs r.__call__'StableDiffusionXLIPAdapterStep.__call__s**51 3=3D3D3S3SVW3W 0'99 (,(L(L (99$(%%"#)4)Q)Q )M) %  3 357K 2#,[-J-J#K6B6H6H6K3%66==>ST3?--a0$L U0  r0r7)N)__name__ __module__ __qualname____firstlineno__ model_namepropertystrr:rrrJrrOr rY staticmethodrurrWno_gradr"rr__static_attributes__r7r0r.r2r2<s&J  S   T-%8  $ Z(   d;&7  5521'f ]]_!#C!M!^k!!r0r2c\rSrSrSr\S\4Sj5r\S\\ 4Sj5r \S\\ 4Sj5r \S\\ 4Sj5r\S\\4Sj5r\S 5r\SS \S \\S \\R*S\S\S\\S\\S\\R0S\\R0S\\R0S\\R0S\\S\\4Sjj5r\R6"5S\S\S\4Sj5rSrg ) StableDiffusionXLTextEncoderStepr4r5cg)NzMText Encoder step that generate text_embeddings to guide the image generationr7r8s r.r:,StableDiffusionXLTextEncoderStep.descriptions^r0c [S[5[S[5[S[5[S[5[S[[ SS05SS 9/$) N text_encodertext_encoder_2 tokenizer tokenizer_2rGrHrIrBrC)rrrr r r r8s r.rJ4StableDiffusionXLTextEncoderStep.expected_componentssW .- 8 *,G H +} 5 - 7 &!#3S"9:(5    r0c[SS5/$)Nforce_zeros_for_empty_promptT)rr8s r.expected_configs1StableDiffusionXLTextEncoderStep.expected_configss94@AAr0c~[S5[S5[S5[S5[S5[S5/$)Npromptprompt_2negative_promptnegative_prompt_2cross_attention_kwargs clip_skip)rr8s r.rO'StableDiffusionXLTextEncoderStep.inputssA x z " ( ) * + / 0 { #   r0c [S[RSSS9[S[RSSS9[S[RSSS9[S [RSS S9/$) N prompt_embedsguider_input_fieldsz2text embeddings used to guide the image generation)rT kwargs_typer:negative_prompt_embedsz;negative text embeddings used to guide the image generationpooled_prompt_embedsz9pooled text embeddings used to guide the image generationnegative_pooled_prompt_embedszBnegative pooled text embeddings used to guide the image generationrVr8s r.rY5StableDiffusionXLTextEncoderStep.intermediate_outputsst ,,1P   (,,1Y   &,,1W   /,,1`  '  r0cURb_[UR[5(d@[UR[5(d![ S[ UR535eUR ba[UR [5(dA[UR [5(d![ S[ UR 535eggg)Nz2`prompt` has to be of type `str` or `list` but is z4`prompt_2` has to be of type `str` or `list` but is )rrhrrxr|typer)rs r. check_inputs-StableDiffusionXLTextEncoderStep.check_inputss    );--s33J{GYGY[_<`<`QRVWbWiWiRjQklm m  ! ! -;//55jI]I]_c>d>dSTXYdYmYmTnSopq q?e5.r0Nrrr_rqrrrrrrr lora_scalerc  U=(d URnU b[U[5(aXlURb8[ (d[ URU 5 O[URU 5 URb8[ (d[ URU 5 O[URU 5 [U[5(aU/OUnUb [U5nOURSnURbURUR/O UR/nURbURUR/O UR/nUGcU=(d Un[U[5(aU/OUn/nX/n[UUU5GH;unnn[U[5(aUR!UU5nU"USUR"SSS9nUR$nU"USSS9R$nURS URS :ah[&R("UU5(dLUR+USS2UR"S - S 245n[,R/S UR"S U35 U"UR1U5SS 9nUSn U cUR2SnOUR2U S-*nUR5U5 GM> [&R6"US S9nUSL=(a UR8R:nU(a8U c5U(a.[&R<"U5n [&R<"U 5n GOU(GawU GcsU=(d SnU=(d Un[U[5(aX/-OUn[U[5(aX/-OUnUb;[?U5[?U5La$[AS[?U5S[?U5S35eU[U5:wa![CSUS[U5SUSUS3 5eXg/n/n[UUU5Hunnn[U[5(aUR!UU5nURS nU"USUSSS9nU"UR$R1U5SS 9n U Sn U R2Sn UR5U 5 M [&R6"US S9n URb%UR1URRDUS9nO$UR1URFRDUS9nURunnn URIS US 5nURKUU-US 5nU(aU RS nURb%U R1URRDUS9n O$U R1URFRDUS9n U RIS US 5n U RKX-US 5n U RIS U5RKUU-S 5n U(a%U RIS U5RKUU-S 5n URb6[U[5(a![ (a[MURU 5 URb6[U[5(a![ (a[MURU 5 XX4$)a% Encodes the prompt into text encoder hidden states. Args: prompt (`str` or `List[str]`, *optional*): prompt to be encoded prompt_2 (`str` or `List[str]`, *optional*): The prompt or prompts to be sent to the `tokenizer_2` and `text_encoder_2`. If not defined, `prompt` is used in both text-encoders device: (`torch.device`): torch device num_images_per_prompt (`int`): number of images that should be generated per prompt prepare_unconditional_embeds (`bool`): whether to use prepare unconditional embeddings or not negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. If not defined, one has to pass `negative_prompt_embeds` instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). negative_prompt_2 (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation to be sent to `tokenizer_2` and `text_encoder_2`. If not defined, `negative_prompt` is used in both text-encoders prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, text embeddings will be generated from `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input argument. pooled_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated pooled text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, pooled text embeddings will be generated from `prompt` input argument. negative_pooled_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative pooled text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, pooled negative_prompt_embeds will be generated from `negative_prompt` input argument. lora_scale (`float`, *optional*): A lora scale that will be applied to all LoRA layers of the text encoder if LoRA layers are loaded. clip_skip (`int`, *optional*): Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. Nr max_lengthTr\)paddingr truncationr]longest)rr]r!z\The following part of your input was truncated because CLIP can only handle sequences up to z tokens: rarcrrdz?`negative_prompt` should be the same type to `prompt`, but got z != .z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.)r`r_)'rrhr _lora_scalerrrrrrryshaperrr}rmaybe_convert_promptmodel_max_length input_idsrWequal batch_decodeloggerwarningrjrkr~concatrDrrmr TypeErrorr|r`rFrepeatviewr)!rorrr_rqrrrrrrrrr batch_size tokenizers text_encodersprompt_embeds_listpromptsrr text_inputstext_input_idsuntruncated_ids removed_textzero_out_negative_prompt uncond_tokensnegative_prompt_embeds_listr uncond_inputbs_embedseq_len_s! r. encode_prompt.StableDiffusionXLTextEncoderStep.encode_prompt'sv7:77  !j=]&^&^%/ "&&2''2:3J3JJW%j&=&=zJ((4''2:3L3LjY%j&?&?L'44&&  VJ&,,Q/J ##/ ! !:#9#9 :(() &&2 $ $j&?&? @++,   )6H%/#%>%>zHH"$ (G36w M3Z/ <j*EFF'<->v-F]a b (5Q'7$$$1$?$?$CM%2$?$?)a-@P$QM"))-8E4[H"LL);DM$3d#:#mz?P?P?m?m ',B,JOg%*%5%5m%D ",1,<,<=Q,R ) ).D.L-3O 1 D_ AK?\_@`@`j+<?PRU4V4V 00\m  !d6l$:O&OUVZ[jVkUlmV ~Q(s?33 )/)::J3K_J`ax/ |<33 "1 D *, 'STYY , ,b  (,I,P,PQRTi,j,o,o00"- )  " " .*&FGGL\L\#J$;$;ZH  $ $ 0*&FGGL\L\#J$=$=zJ6Jiir0rorc6URU5nURU5 URRS:UlUR UlURbURRSS5OSUl URUURURUR SURURURSSSSURURS9uUlUlUlUlUR)X#5 X4$)Nr!scale)rrrrrr)rrrGrrrr_rgettext_encoder_lora_scalerrrrrrrrrrr)r9rorrs r.r)StableDiffusionXLTextEncoderStep.__call__s **51  +&3=3D3D3S3SVW3W 0'99  11=  . . 2 27D A +             4 4  ' '  ) )#'!%*."::!++    %  .  ,  5$ U0  r0r7) NNr!TNNNNNNNN) rrrrrrrr:rrrJrrrrOr rYrrrrWr_intboolrXfloatrrr"rrrr7r0r.rrs&J _S__  T-%8    B$z"2BB Z(   d;&7  8rr#')-%&-1)-+/049=7;@D&*#'pjpj3-pj& pj # pj '+ pj"#pj$C=pj  -pj!) 6pj'u||4pj(0 '=pjUOpjC=pjpjd ]]_%!#C%!M%!^k%!%!r0rc\rSrSrSr\S\4Sj5r\S\\ 4Sj5r \S\\ 4Sj5r \S\\ 4Sj5rS\R S \R"4S jr\R&"5S \S \S\4S j5rSrg)StableDiffusionXLVaeEncoderStepiCr4r5cg)NzIVae Encoder step that encode the input image into a latent representationr7r8s r.r:+StableDiffusionXLVaeEncoderStep.descriptionFsZr0c X[S[5[S[[SS05SS9/$)Nvaeimage_processorvae_scale_factorrBrCrrrr r8s r.rJ3StableDiffusionXLVaeEncoderStep.expected_componentsJs7 % / !!!#5q"9:(5    r0c [SSS9[S5[S5[S5[S[RSS 9[S [[S S 9/$) NrpTrNheightwidthr$r`z Data type of model tensor inputsrSpreprocess_kwargszA kwargs dictionary that if specified is passed along to the `ImageProcessor` as defined under `self.image_processor` in [diffusers.image_processor.VaeImageProcessor])rrWr`rdictr8s r.rO&StableDiffusionXLVaeEncoderStep.inputsVsY w . x w  { # w%++Ce f #"4.E   r0c6[S[RSS9/$)N image_latentszUThe latents representing the reference image for image-to-image/inpainting generationrSrVr8s r.rY4StableDiffusionXLVaeEncoderStep.intermediate_outputses# ,,s   r0rpr$c S=pE[URRS5(agURRRbF[R "URRR5R SSSS5n[URRS5(agURRRbF[R "URRR5R SSSS5nURnURRR(a7UR5nURR[RS9 [U[5(af[UR S5Vs/sH-n[#URR%X'US-5X7S9PM/ nn[R&"USS9nO#[#URR%U5US9nURRR(aURRU5 URU5nUbaUb^URUR(US 9nURUR(US 9nX- URRR*-U- nU$URRR*U-nU$s snf N latents_meanr! latents_stdr`rr$rdr^r+rrDr rWtensorrr r` force_upcastrrjfloat32rhrxrangerr/encoderr_scaling_factor r9rorpr$r r r`rrs r._encode_vae_image1StableDiffusionXLVaeEncoderStep._encode_vae_imageqsV%)) :>>((. 9 9jnn>S>S>`>`>l << (=(=(J(JKPPQRTUWXZ[\L :>>((- 8 8Z^^=R=R=^=^=j,,z~~'<'<'H'HINNqRSUVXYZK  >> - -KKME NN  EMM  2 i & &u{{1~..A!!6!6uQ7G!HT]T`a. "IIm;M,Z^^-B-B5-IU^_M >> - - NN  e $%((/  # (?'??-2F2Fe?TL%.. 0D0DE.RK*9Z^^=R=R=a=aadooM'NN11@@=PM'34K<rorc*URU5nUR=(d 0UlURUlURb UROUR RUlUR R"UR4URURS.URD6nURURURS9nURSUl [UR[ 5(aR[#UR5UR:wa/[%S[#UR5SURS35eUR'XURS9UlUR+X#5 X4$)N)rrr^rz/You have passed a list of generators of length z+, but requested an effective batch size of z@. Make sure the batch size matches the length of the generators.rpr$)rrrr_r`rr preprocessrprrrjrrrhr$rxryr|rrr)r9rorrrps r.r(StableDiffusionXLVaeEncoderStep.__call__so**51 (3(E(E(K %'99 1<1B1B1NK--T^TbTbThTh **55    &1&8&8 @Q@Q U`UrUr  2 2+:K:KL!&Q  k++T 2 2s;;P;P7QU`UkUk7kA#kF[F[B\A]^'2233su  %)$:$::^i^s^s$:$t ! U0  r0r7N)rrrrrrrr:rrrJrrOr rYrWrX Generatorrrr"rrrr7r0r.rrCs&J [S[[  T-%8      Z(     d;&7   5<< EOO D ]]_!#C!M!^k!!r0rc\rSrSrSr\S\\4Sj5r\S\ 4Sj5r \S\\ 4Sj5r \S\\ 4Sj5rS\R S \R"4S jrS r\R("5S \S \S\4Sj5rSrg)&StableDiffusionXLInpaintVaeEncoderStepir4r5c [S[5[S[[SS05SS9[S[[SSS S S .5SS9/$) NrrrrrBrCmask_processorFT) do_normalizer do_binarizedo_convert_grayscalerr8s r.rJ:StableDiffusionXLInpaintVaeEncoderStep.expected_componentssb % / !!!#5q"9:(5    !!%*RVptu)6    r0cg)NzLVae encoder step that prepares the image and mask for the inpainting processr7r8s r.r:2StableDiffusionXLInpaintVaeEncoderStep.descriptions]r0c [S5[S5[SSS9[SSS9[S5[S[RS S 9[S 5/$) NrrrpTr mask_imagepadding_mask_cropr`zThe dtype of the model inputsrSr$)rrWr`r8s r.rO-StableDiffusionXLInpaintVaeEncoderStep.inputssT x w  w . |d 3 * + w%++Cb c { #  r0c [S[RSS9[S[RSS9[S[RSS9[S[[[ [ 4S S9/$) Nrz-The latents representation of the input imagerSmaskz*The mask to use for the inpainting processmasked_image_latentsz^The masked image latents to use for the inpainting process (only for inpainting-specifid unet) crops_coordszPThe crop coordinates to use for the preprocess/postprocess of the image and mask)r rWrXrrrr8s r.rY;StableDiffusionXLInpaintVaeEncoderStep.intermediate_outputssj 5<>((. 9 9jnn>S>S>`>`>l << (=(=(J(JKPPQRTUWXZ[\L :>>((- 8 8Z^^=R=R=^=^=j,,z~~'<'<'H'HINNqRSUVXYZK  >> - -KKME NN  EMM  2 i & &u{{1~..A!!6!6uQ7G!HT]T`a. "IIm;M,Z^^-B-B5-IU^_M >> - - NN  e $%((/  # (?'??-2F2Fe?TL%.. 0D0DE.RK*9TXX__=[=[[^iiM'NN11@@=PM'rc [RRRX%UR-XaR-4S9nUR XS9nUR SU:aWXBR S-S:Xd[SUSUR SS35eURXBR S-SSS5nUbUR SS:XaUn OSn UbU cUR XS9nURXU S 9n U R SU:aWXJR S-S:Xd[S USU R SS 35eU RXJR S-SSS5n U R XS9n X*4$) N)r@r^rzvThe passed mask and the required batch size don't match. Masks are supposed to be duplicated to a total batch size of z, but zo masks were passed. Make sure the number of masks that you pass is divisible by the total requested batch size.r!r rzyThe passed images and the required batch size don't match. Images are supposed to be duplicated to a total batch size of zq images were passed. Make sure the number of images that you pass is divisible by the total requested batch size.) rWnn functional interpolaterrjrr|rr) r9ror. masked_imagerrrr`r_r$r/s r.prepare_mask_latents;StableDiffusionXLInpaintVaeEncoderStep.prepare_mask_latentss xx"".. *"="==uHcHc?cd/ wwfw2 ::a=: % 1 -2 ..8\ 1 O^^ ;;zZZ]:Aq!DD  # (:(:1(=(B#/ #'  ##++fJ '+'='=jbk'='l$#))!,z9!$>$>q$AAQF$55?LG[GaGabcGdFefxx (<'B'B"<"B=V=V    $ $  " "              ! ! > : +: U0  r0r7N)rrrrrrrrrJrr:rrOr rYrWrXrrr9rr"rrrr7r0r.r r s&J  T-%8  &^S^^  Z(     d;&7  ( 5<< EOO H-*^ ]]_9!#C9!M9!^k9!9!r0r )Nr()4typingrrrrW transformersrrrr r configuration_utilsr guidersr rrrloadersrrmodelsrrr models.lorarutilsrrrrmodular_pipelinerrmodular_pipeline_utilsrrrr r" get_loggerrrrXrrr/r2rrr r7r0r.rOs)( .-DTJJ9 DWW>   H % ck TLL T-5eoo-F T\_ T[!%:[!|f!'<f!R h!&;h!VI!-BI!r0