+h]2SSKJr SSKJrJrJr SSKrSSKr SSK r SSK J r J r JrJr SSKJr SSKJr SSKJrJrJrJr SS KJr S S KJr \"5(a SSKJs Jr S r!OS r!\RD"\#5r$Sr%Sr&\"SS\55r'"SS\5r(g)) dataclass)ListOptionalUnionN)CLIPImageProcessorCLIPTextModelWithProjection CLIPTokenizerCLIPVisionModelWithProjection)PriorTransformer)UnCLIPScheduler) BaseOutputis_torch_xla_availableloggingreplace_example_docstring) randn_tensor)DiffusionPipelineTFav Examples: ```py >>> from diffusers import KandinskyPipeline, KandinskyPriorPipeline >>> import torch >>> pipe_prior = KandinskyPriorPipeline.from_pretrained("kandinsky-community/kandinsky-2-1-prior") >>> pipe_prior.to("cuda") >>> prompt = "red cat, 4k photo" >>> out = pipe_prior(prompt) >>> image_emb = out.image_embeds >>> negative_image_emb = out.negative_image_embeds >>> pipe = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1") >>> pipe.to("cuda") >>> image = pipe( ... prompt, ... image_embeds=image_emb, ... negative_image_embeds=negative_image_emb, ... height=768, ... width=768, ... num_inference_steps=100, ... ).images >>> image[0].save("cat.png") ``` a Examples: ```py >>> from diffusers import KandinskyPriorPipeline, KandinskyPipeline >>> from diffusers.utils import load_image >>> import PIL >>> import torch >>> from torchvision import transforms >>> pipe_prior = KandinskyPriorPipeline.from_pretrained( ... "kandinsky-community/kandinsky-2-1-prior", torch_dtype=torch.float16 ... ) >>> pipe_prior.to("cuda") >>> img1 = load_image( ... "https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main" ... "/kandinsky/cat.png" ... ) >>> img2 = load_image( ... "https://huggingface.co/datasets/hf-internal-testing/diffusers-images/resolve/main" ... "/kandinsky/starry_night.jpeg" ... ) >>> images_texts = ["a cat", img1, img2] >>> weights = [0.3, 0.3, 0.4] >>> image_emb, zero_image_emb = pipe_prior.interpolate(images_texts, weights) >>> pipe = KandinskyPipeline.from_pretrained("kandinsky-community/kandinsky-2-1", torch_dtype=torch.float16) >>> pipe.to("cuda") >>> image = pipe( ... "", ... image_embeds=image_emb, ... negative_image_embeds=zero_image_emb, ... height=768, ... width=768, ... num_inference_steps=150, ... ).images[0] >>> image.save("starry_cat.png") ``` c\rSrSr%Sr\\R\R4\ S'\\R\R4\ S'Sr g)KandinskyPriorPipelineOutputyz Output class for KandinskyPriorPipeline. Args: image_embeds (`torch.Tensor`) clip image embeddings for text prompt negative_image_embeds (`List[PIL.Image.Image]` or `np.ndarray`) clip image embeddings for unconditional tokens image_embedsnegative_image_embedsN) __name__ __module__ __qualname____firstlineno____doc__rtorchTensornpndarray__annotations____static_attributes__rp/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/pipelines/kandinsky/pipeline_kandinsky_prior.pyrrys9 bjj011 rzz!9::r&rc^\rSrSrSrS/rSrS\S\S\ S\ S\ S \ 4 U4S jjr \R"5\"\5SS \\\\R,R,\R.4S \\S \S\S\\\R6\\R64S\\R.S\\S\S\4Sjj55rSrSSjrSSjr\R"5\"\ 5S S\\\\4S\\\\\4S \S\S\\\R6\\R64S\\R.S\S\\S\!4Sjj55r"Sr#U=r$$)!KandinskyPriorPipelineaq Pipeline for generating image prior for Kandinsky This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods the library implements for all the pipelines (such as downloading or saving, running on a particular device, etc.) Args: prior ([`PriorTransformer`]): The canonical unCLIP prior to approximate the image embedding from the text embedding. image_encoder ([`CLIPVisionModelWithProjection`]): Frozen image-encoder. text_encoder ([`CLIPTextModelWithProjection`]): Frozen text-encoder. tokenizer (`CLIPTokenizer`): Tokenizer of class [CLIPTokenizer](https://huggingface.co/docs/transformers/v4.21.0/en/model_doc/clip#transformers.CLIPTokenizer). scheduler ([`UnCLIPScheduler`]): A scheduler to be used in combination with `prior` to generate image embedding. priorztext_encoder->prior image_encoder text_encoder tokenizer schedulerimage_processorc J>[TU]5 URUUUUUUS9 g)N)r+r-r.r/r,r0)super__init__register_modules)selfr+r,r-r.r/r0 __class__s r'r3KandinskyPriorPipeline.__init__s5  %'+  r&images_and_promptsweightsnum_images_per_promptnum_inference_steps generatorlatentsnegative_prior_promptnegative_promptguidance_scalec U =(d URn [U5[U5:wa$[S[U5S[U5S35e/n [X5GH!up[ U [ 5(aU"U UUUUUU S9R nO[ U [RR[R45(a[ U [RR5(aNURU SS9RSRS5RURR U S9n URU 5S nO[S [#U 535eU R%X-5 GM$ [R&"U 5R)SS S 9nU"UUUUUUU S9nUS :Xa UR*O UR n[-UUS9$)a Function invoked when using the prior pipeline for interpolation. Args: images_and_prompts (`List[Union[str, PIL.Image.Image, torch.Tensor]]`): list of prompts and images to guide the image generation. weights: (`List[float]`): list of weights for each condition in `images_and_prompts` num_images_per_prompt (`int`, *optional*, defaults to 1): The number of images to generate per prompt. num_inference_steps (`int`, *optional*, defaults to 25): The number of denoising steps. More denoising steps usually lead to a higher quality image at the expense of slower inference. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): One or a list of [torch generator(s)](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents, sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor will ge generated by sampling using the supplied random `generator`. negative_prior_prompt (`str`, *optional*): The prompt not to guide the prior diffusion process. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). negative_prompt (`str` or `List[str]`, *optional*): The prompt not to guide the image generation. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). guidance_scale (`float`, *optional*, defaults to 4.0): Guidance scale as defined in [Classifier-Free Diffusion Guidance](https://huggingface.co/papers/2207.12598). `guidance_scale` is defined as `w` of equation 2. of [Imagen Paper](https://huggingface.co/papers/2205.11487). Guidance scale is enabled by setting `guidance_scale > 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. Examples: Returns: [`KandinskyPriorPipelineOutput`] or `tuple` z`images_and_prompts` contains z items and `weights` contains z, items - they should be lists of same length)r;r:r<r=r?r@pt)return_tensorsr)dtypedevicerzq`images_and_prompts` can only contains elements to be of type `str`, `PIL.Image.Image` or `torch.Tensor` but is T)dimkeepdimrr)rElen ValueErrorzip isinstancestrrPILImager r!r0 pixel_values unsqueezetor,rDtypeappendcatsumrr)r5r8r9r:r;r<r=r>r?r@rEimage_embeddingscondweight image_embout_zerozero_image_embs r' interpolate"KandinskyPriorPipeline.interpolatesl&4;; ! "c'l 205G1H0IIghklshtguvbc  2%>%I%I4K]K]KdKdKoKopss!3!3!9!9t ++H5nE'..z1=r&c p[U[5(a [U5OSnURUSURRSSS9nUR nUR R5RU5n URUSSS9R n U RSURS:a[R"X5(dURRU SS2URRS- S245n [RS URRS U 35 USS2SURR24nURURU55n U R n U R"nU R%US S 9n UR%US S 9nU R%US S 9n U(GaUcS /U-nO['U5['U5La$[)S['U5S['U5S35e[U[*5(aU/nO2U[U5:wa![-SUS[U5SUSUS3 5eUnURUSURRSSS9nUR R5RU5nURUR RU55nUR nUR"nURSnUR/SU5nUR1Xc-U5nURSnUR/SUS5nUR1Xc-US5nUR%US S 9n[R2"UU /5n [R2"UU/5n[R2"UU /5n XU 4$)Nrf max_lengthTrB)paddingrp truncationrClongest)rqrCz\The following part of your input was truncated because CLIP can only handle sequences up to z tokens: r)rFrHz?`negative_prompt` should be the same type to `prompt`, but got z != .z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.)rMlistrJr.model_max_length input_idsattention_maskboolrSrar equal batch_decodeloggerwarningr- text_embedslast_hidden_staterepeat_interleaverT TypeErrorrNrKrjviewrV)r5promptrEr:do_classifier_free_guidancer?rk text_inputstext_input_ids text_maskuntruncated_ids removed_texttext_encoder_output prompt_embedstext_encoder_hidden_states uncond_tokens uncond_inputuncond_text_mask*negative_prompt_embeds_text_encoder_outputnegative_prompt_embeds!uncond_text_encoder_hidden_statesseq_lens r'_encode_prompt%KandinskyPriorPipeline._encode_prompt5s%/vt$<$>66q$..JiJilmJmprJrGr7stL NNNN334Il^M ,A/P1P1P/P,PQN"//0A0A&0IJ+77 %8%J%J"%778MST7U %?%Q%QRgmn%Q%o"//0E1/M &&!#z 1 fT/%::UVZ[jVkUlmV ~Q(OS11!0 1 s?33 )/)::J3K_J`ax/ |<33 !0 >>$>>::# *L ,::??ADDVL 9=9J9J 1`. Higher guidance scale encourages to generate images that are closely linked to the text `prompt`, usually at the expense of lower image quality. output_type (`str`, *optional*, defaults to `"pt"`): The output format of the generate image. Choose between: `"np"` (`np.array`) or `"pt"` (`torch.Tensor`). return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.ImagePipelineOutput`] instead of a plain tuple. Examples: Returns: [`KandinskyPriorPipelineOutput`] or `tuple` z2`prompt` has to be of type `str` or `list` but is Nz;`negative_prompt` has to be of type `str` or `list` but is rg?)rE)timestepproj_embeddingencoder_hidden_statesryrfr)rsampler< prev_timestepfinal_offload_hook)rBr"zBOnly the output types `pt` and `np` are supported not output_type=r"rI)'rMrNrvrKrT_execution_devicerJrr/ set_timesteps timestepsr+rh embedding_dimrcrD enumerate progress_barr rVpredicted_image_embeddingchunkrastep prev_sample XLA_AVAILABLExm mark_steppost_process_latentsrmrEmaybe_free_model_hookshasattrr prior_hookoffloadcpunumpyr)r5rr?r:r;r<r=r@rrrErkrrrrprior_timesteps_tensorritlatent_model_inputr predicted_image_embedding_uncondpredicted_image_embedding_textrrX zero_embedss r'__call__KandinskyPriorPipeline.__call__sdj fc " "XFFD))QRVW]R^Q_`a a os + +./OOT227RZ[_`o[pZqrs s  &-F/1O''[ 7 &4s&:#?C?R?R 1P_@ < 9 $$%8$H!%!9!9 ))77 &&  '       NN  d//0FGHDAq=XG9q=!9^e (, ",&@( )3) (' &+SlSrSrstSuP02P,L~25UUP-)1u.44Q77 $ 6q1u = nn)))#+ * k } AID**11':"  "--gmmA.>w~~-VK  ' ' ),<,B,B1,E ) kt122t7N7N7Z'') l *abmanop p $ /335;;= %//+113K$k2 2+9Ialmmr&r)rfNNNrH@N)rfN)N)NrfrNNrrBT)%rrrrr_exclude_from_cpu_offloadmodel_cpu_offload_seqr r rr r rr3r no_gradrEXAMPLE_INTERPOLATE_DOC_STRINGrrrNrOrPr!floatintr Generatorr^rcrmrEXAMPLE_DOC_STRINGrzrr% __classcell__)r6s@r'r)r)sb(") 1  5 2  !  #  , ( ]]_=> &'#%MQ*./3! #gj sCIIOOU\\'I!JKgjegj # gj ! gj E%//43H"HIJ gj%,,'gj (}gjgjgj?gjT  _DB ]]_12<@%&#%MQ*. #%) Wnc49n%Wn"%T#Y"78Wn # Wn ! Wn E%//43H"HIJ Wn%,,'WnWnc]WnWn3Wnr&r))) dataclassesrtypingrrrrr" PIL.ImagerOr transformersrrr r modelsr schedulersr utilsrrrrutils.torch_utilsrpipeline_utilsrtorch_xla.core.xla_modelcore xla_modelrr get_loggerrr}rrrr)rr&r'rs"(( vv&) ..))MM   H %<+"\  ;: ;  ;fn.fnr&