+h3SSKrSSKJr SSKJrJrJrJrJrJ r SSK r SSK r SSK r SSKJrJrJrJr SSKJrJr SSKJr SSKJr SS KJr SS KJrJrJrJ r SS K!J"r" SS K#J$r$ S SK%J&r&J'r'J(r( \"5(a SSK)J*s J+r, Sr-OSr-\R\"\/5r0Sr1\"SS\55r2"SS\&\'\(5r3S\ \ Rh\\ Rh44Sjr5S\ \ Rh\\ Rh\ RlRl\\ RlRl4S\\7\744Sjr8S\ \ Rh\\ Rh\ RlRl\\ RlRl4S\\7\744Sjr9g)N) dataclass)AnyDictListOptionalTupleUnion)CLIPImageProcessor CLIPTextModel CLIPTokenizerCLIPVisionModelWithProjection)PipelineImageInputVaeImageProcessor) AutoencoderKL) I2VGenXLUNet) DDIMScheduler) BaseOutputis_torch_xla_availableloggingreplace_example_docstring) randn_tensor)VideoProcessor)DeprecatedPipelineMixinDiffusionPipelineStableDiffusionMixinTFa Examples: ```py >>> import torch >>> from diffusers import I2VGenXLPipeline >>> from diffusers.utils import export_to_gif, load_image >>> pipeline = I2VGenXLPipeline.from_pretrained( ... "ali-vilab/i2vgen-xl", torch_dtype=torch.float16, variant="fp16" ... ) >>> pipeline.enable_model_cpu_offload() >>> image_url = ( ... "https://huggingface.co/datasets/diffusers/docs-images/resolve/main/i2vgen_xl_images/img_0009.png" ... ) >>> image = load_image(image_url).convert("RGB") >>> prompt = "Papers were floating in the air on a table in the library" >>> negative_prompt = "Distorted, discontinuous, Ugly, blurry, low resolution, motionless, static, disfigured, disconnected limbs, Ugly faces, incomplete arms" >>> generator = torch.manual_seed(8888) >>> frames = pipeline( ... prompt=prompt, ... image=image, ... num_inference_steps=50, ... negative_prompt=negative_prompt, ... guidance_scale=9.0, ... generator=generator, ... ).frames[0] >>> video_path = export_to_gif(frames, "i2v.gif") ``` c\rSrSr%Sr\\R\R\ \ \ RR4\ S'Srg)I2VGenXLPipelineOutputSa Output class for image-to-video pipeline. Args: frames (`torch.Tensor`, `np.ndarray`, or List[List[PIL.Image.Image]]): List of video outputs - It can be a nested list of length `batch_size,` with each sub-list containing denoised PIL image sequences of length `num_frames.` It can also be a NumPy array or Torch tensor of shape `(batch_size, num_frames, channels, height, width)` framesN)__name__ __module__ __qualname____firstlineno____doc__r torchTensornpndarrayrPILImage__annotations____static_attributes__r"j/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/pipelines/i2vgen_xl/pipeline_i2vgen_xl.pyrrSs5  %,, Dciioo1F,GG HHr0rc,^\rSrSrSrSrS\S\S\S\ S\ S \ S \ 4U4S jjr \S 5r\S 5rS+S\\R&S\\R&S\\4SjjrSrS,SjrSrS-SjrSrS,Sjr\R8"5\"\5S.S\\ \!\ 4S\"S\\S\\S\\S\S\S\#S \\\ \!\ 4S!\#S"\\S#\\S$\\\RH\!\RH4S%\\R&S\\R&S\\R&S&\\ S'\%S(\\&\ \'4S\\4(S)jj55r(S*r)U=r*$)/I2VGenXLPipelinecz0.33.1z&text_encoder->image_encoder->unet->vaevae text_encoder tokenizer image_encoderfeature_extractorunet schedulerc >[TU]5 URUUUUUUUS9 [USS5(a/S[ UR R R5S- -OSUl[URSS9Ul g)N)r5r6r7r8r9r:r;r5rF)vae_scale_factor do_resize) super__init__register_modulesgetattrlenr5configblock_out_channelsr?rvideo_processor) selfr5r6r7r8r9r:r; __class__s r1rBI2VGenXLPipeline.__init__~s  %'/  W^^bdikoVpVpc$((//*L*L&MPQ&Q Rvw-t?T?T`efr0cUR$N_guidance_scalerIs r1guidance_scaleI2VGenXLPipeline.guidance_scales###r0c URS:$)Nr=rNrPs r1do_classifier_free_guidance,I2VGenXLPipeline.do_classifier_free_guidances##a''r0 prompt_embedsnegative_prompt_embeds clip_skipc  Ub[U[5(aSnO3Ub![U[5(a [U5nOURSnUGcUR USUR R SSS9n U Rn UR USSS 9Rn U RS U RS :a[R"X5(djUR RU SS2UR R S- S 245n [RS UR R S U 35 [URRS 5(aAURRR (aU R"R%U5n OSn Uc%URU R%U5U S9nUSnOQURU R%U5U SS9nUS US-*nURR&R)U5nURbURR*nO0UR,bUR,R*nO UR*nUR%XS9nURunnnUR/SUS5nUR1X-US 5nUR2(GaUGcUcS/U-nOUb;[5U5[5U5La$[7S[5U5S[5U5S35e[U[5(aU/nO2U[U5:wa![9SUS[U5SUSUS3 5eUnURSnUR USUSSS9n[URRS 5(aAURRR (aUR"R%U5n OSn Uc/URURR%U5U S9nUSnO[URURR%U5U SS9nUS US-*nURR&R)U5nUR2(aFURSnUR%XS9nUR/SUS5nUR1X-US 5nXV4$)a Encodes the prompt into text encoder hidden states. Args: prompt (`str` or `List[str]`, *optional*): prompt to be encoded device: (`torch.device`): torch device num_videos_per_prompt (`int`): number of images that should be generated per prompt do_classifier_free_guidance (`bool`): whether to use classifier free guidance or not negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. If not defined, one has to pass `negative_prompt_embeds` instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, text embeddings will be generated from `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input argument. clip_skip (`int`, *optional*): Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. Nr=r max_lengthTpt)paddingrZ truncationreturn_tensorslongest)r\r^z\The following part of your input was truncated because CLIP can only handle sequences up to z tokens: use_attention_mask)attention_mask)rboutput_hidden_states)dtypedevicez?`negative_prompt` should be the same type to `prompt`, but got z != .z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.) isinstancestrlistrEshaper7model_max_length input_idsr(equal batch_decodeloggerwarninghasattrr6rFrarbto text_modelfinal_layer_normrdr:repeatviewrTtype TypeError ValueError)rIpromptrenum_videos_per_promptnegative_promptrVrWrX batch_size text_inputstext_input_idsuntruncated_ids removed_textrbprompt_embeds_dtypebs_embedseq_len_ uncond_tokensrZ uncond_inputs r1 encode_promptI2VGenXLPipeline.encode_promptsJ  *VS"9"9J  Jvt$<$<VJ&,,Q/J  ..$>>::# )K)22N"nnVYW[n\ffO$$R(N,@,@,DDU[[NN $~~::#At~~'F'F'JR'O$OP  778 ,Q t((//1EFF4K\K\KcKcKvKv!,!;!;!>!>v!F!% $ 1 1.2C2CF2K\j 1 k -a 0 $ 1 1"%%f-ncg!2! !.b 1IM2B C !% 1 1 < < M Mm \    ("&"3"3"9"9  YY ""&))// "/"5"5 %((/B(R ,22'1%,,Q0EqI %**8+KWVXY   + + +0F0N&!#z 1 #V D>$%# *Lt((//1EFF4K\K\KcKcKvKv!-!%>EX%>%h "%;%B%B1F[]^%_ "%;%@%@Acelnp%q "44r0c [URR55Rn[ U[ R 5(dTURRU5nURRU5nURUSSSSSS9RnURX$S9nURU5RnURS5nURupgnUR!SUS5nUR#Xc-US5nUR$(a-[ R&"U5n [ R("X/5nU$)NTFr[)images do_normalizedo_center_cropr@ do_rescaler^rerdr=r`)nextr8 parametersrdrhr(r)rH pil_to_numpy numpy_to_ptr9 pixel_valuesrs image_embeds unsqueezerkrvrwrT zeros_likecat) rIimagerer|rdimage_embeddingsrrrnegative_image_embeddingss r1 _encode_imageI2VGenXLPipeline._encode_imageKsAT''2245;;%..((55e:A--7 j%//!Q1a0889PRZdij  (F1gmmA.0ABA4I(JKRR e$CIIf!,EHHOOG,33E"3ekk!"oE dAg&&|4<rz7`height` and `width` have to be divisible by 8 but are z and rgzCannot forward both `prompt`: z and `prompt_embeds`: z2. Please make sure to only forward one of the two.zeProvide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined.z2`prompt` has to be of type `str` or `list` but is z'Cannot forward both `negative_prompt`: z and `negative_prompt_embeds`: zu`prompt_embeds` and `negative_prompt_embeds` must have the same shape when passed directly, but got: `prompt_embeds` z != `negative_prompt_embeds` z``image` has to be of type `torch.Tensor` or `PIL.Image.Image` or `List[PIL.Image.Image]` but is ) rzrhrirjrxrkr(r)r,r-)rIr{rrrr}rVrWs r1 check_inputsI2VGenXLPipeline.check_inputss A:?eai1nVW]V^^cdicjjklm m  -";08N}o^00 ^ 5w  FC)@)@TZ\`IaIaQRVW]R^Q_`a a  &+A+M9/9JK*++]_   $)?)K""&<&B&BB --:-@-@,AB.445Q85%,,//uciioo66ud++K=" ,70r0c URUS9nURRU5RR 5nXPRR R -nURS5n/n[US- 5HBnUS-US- - nUR[R"USS2SS2SS245U-5 MD U(a+[R"USS9n[R"XV/SS9nURUSSSS5nUR(a[R"U/S-5nU$)Nrerr=r)rsr5encode latent_distrrFrrrrr( ones_likerrvrT) rIrrerr| image_latentsframe_position_mask frame_idxscales r1prepare_image_latents&I2VGenXLPipeline.prepare_image_latentss '.::AAC %(F(FF &//2 !zA~.I]zA~6E  & &u}Q2A2X7N'ORW'W X/ "')),?Q"G !II}&JPQRM&,,-BAq!QO  + +!II}o&9:Mr0c 0UUUX@R-XPR-4n [U[5(a*[U5U:wa[ S[U5SUS35eU c [ XXvS9n OU R U5n XRR-n U $)Nz/You have passed a list of generators of length z+, but requested an effective batch size of z@. Make sure the batch size matches the length of the generators.)rrerd) r?rhrjrErzrrsr;init_noise_sigma) rIr~num_channels_latentsrrrrdrerrrks r1prepare_latents I2VGenXLPipeline.prepare_latentss   ++ + ** *   i & &3y>Z+GA#i.AQR&<'gi  ?"5fZGjj(GNN;;;r0r{rrr target_fpsrnum_inference_stepsrQr}rr|rrr output_type return_dictcross_attention_kwargsc  U=(d- URRRUR-nU=(d- URRRUR-nUR XX4XU5 Ub[ U[ 5(aSnO3Ub![ U[5(a [U5nOURSnURnXl URUUU U UUUS9unnUR(a[R"UU/5n[!X$U45n[#UUR$R&SUR$R&S45nUR)UUU 5n[!X$U45nUR*R-U5R/UUR0S9nUR3UUUU S9nUR(a'[R4"XU/5R/U5nO&[R4"U/5R/U5nUR7UU -S5R95nUR:R=UUS 9 UR:R>nURRR@nURCUU -UUUUUR0UU U5 nUREX5n[U5XpR:RF-- nURIUS 9n [KU5GHun!n"UR(a[R"U/S -5OUn#UR:RMU#U"5n#URU#U"UUUUUS S 9Sn$UR(aU$ROS 5un%n&U%UU&U%- --n$URunn'n(pCURQSS SSS5RSUU(-U'XC5nU$RQSS SSS5RSUU(-U'XC5n$UR:RT"U$U"U40UD6RVnUSSS24RSUU(U'XC5RQSS SSS5nU![U5S- :Xd)U!S-U:a0U!S-UR:RF-S:XaU RY5 [Z(dGM[\R^"5 GM SSS5 US:XaUn)O)URaXS9n*UR*RcU*US9n)URe5 U(dU)4$[gU)S9$!,(df  Nc=f)u The call function to the pipeline for image-to-video generation with [`I2VGenXLPipeline`]. Args: prompt (`str` or `List[str]`, *optional*): The prompt or prompts to guide image generation. If not defined, you need to pass `prompt_embeds`. image (`PIL.Image.Image` or `List[PIL.Image.Image]` or `torch.Tensor`): Image or images to guide image generation. If you provide a tensor, it needs to be compatible with [`CLIPImageProcessor`](https://huggingface.co/lambdalabs/sd-image-variations-diffusers/blob/main/feature_extractor/preprocessor_config.json). height (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`): The height in pixels of the generated image. width (`int`, *optional*, defaults to `self.unet.config.sample_size * self.vae_scale_factor`): The width in pixels of the generated image. target_fps (`int`, *optional*): Frames per second. The rate at which the generated images shall be exported to a video after generation. This is also used as a "micro-condition" while generation. num_frames (`int`, *optional*): The number of video frames to generate. num_inference_steps (`int`, *optional*): The number of denoising steps. guidance_scale (`float`, *optional*, defaults to 7.5): A higher guidance scale value encourages the model to generate images closely linked to the text `prompt` at the expense of lower image quality. Guidance scale is enabled when `guidance_scale > 1`. negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts to guide what to not include in image generation. If not defined, you need to pass `negative_prompt_embeds` instead. Ignored when not using guidance (`guidance_scale < 1`). eta (`float`, *optional*): Corresponds to parameter eta (η) from the [DDIM](https://huggingface.co/papers/2010.02502) paper. Only applies to the [`~schedulers.DDIMScheduler`], and is ignored in other schedulers. num_videos_per_prompt (`int`, *optional*): The number of images to generate per prompt. decode_chunk_size (`int`, *optional*): The number of frames to decode at a time. The higher the chunk size, the higher the temporal consistency between frames, but also the higher the memory consumption. By default, the decoder will decode all frames at once for maximal quality. Reduce `decode_chunk_size` to reduce memory usage. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for image generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor is generated by sampling using the supplied random `generator`. prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, text embeddings are generated from the `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, `negative_prompt_embeds` are generated from the `negative_prompt` input argument. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generated image. Choose between `PIL.Image` or `np.array`. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.stable_diffusion.StableDiffusionPipelineOutput`] instead of a plain tuple. cross_attention_kwargs (`dict`, *optional*): A kwargs dictionary that if specified is passed along to the [`AttentionProcessor`] as defined in [`self.processor`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py). clip_skip (`int`, *optional*): Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. Examples: Returns: [`pipelines.i2vgen_xl.pipeline_i2vgen_xl.I2VGenXLPipelineOutput`] or `tuple`: If `return_dict` is `True`, [`pipelines.i2vgen_xl.pipeline_i2vgen_xl.I2VGenXLPipelineOutput`] is returned, otherwise a `tuple` is returned where the first element is a list with the generated frames. Nr=r)rVrWrXrrr)rerr|r)totalrF)encoder_hidden_statesfpsrrrrrrlatent)r)rr)r!)4r:rF sample_sizer?rrhrirjrErk_execution_devicerOrrTr(r_center_crop_wide_resize_bilinearr9 crop_sizerrH preprocessrsrdrtensorrvravelr; set_timesteps timesteps in_channelsrrorder progress_bar enumeratescale_model_inputchunkrrr prev_sampleupdate XLA_AVAILABLExm mark_steprpostprocess_videomaybe_free_model_hooksr)+rIr{rrrrrrrQr}rr|rrrrVrWrrrrXr~re cropped_imager resized_imager fps_tensorrrrnum_warmup_stepsrrtlatent_model_input noise_prednoise_pred_uncondnoise_pred_textchannelr!r video_tensors+ r1__call__I2VGenXLPipeline.__call__s>zO499++77$:O:OOM))558M8MM &Xno  *VS"9"9J  Jvt$<$<VJ&,,Q/J'' .150B0B   ! '#91C1 - -  + +!II'=}&MNM *%@ ( D22<AAWgWmWmAn22 !"7 3   + +z&>?BB6JJzl366v>J&&z4I'I1MSSU  $$%8$HNN,,  $yy//;;&& . .           !::9Jy>,?..BVBV,VV   %8  9\!),1AEAaAaUYYy1}%=gn"%)^^%E%EFXZ[%\""YY&*7""/%5+A %'   339C9I9I!9L6%!2^YjGj5k!kJ>E]]: GVU!//!Q1a8@@fATV]_dm'//1aA>FFzTZGZ\cejs ..--j!WZHYZff"$'*22:vwPU^ffghjkmnpqstuI**A9I/IqSTuX\XfXfXlXlNlpqNq '') =LLNK-:R ( "E..w.\L((::[f:gE ##%8O%U33k: 9s F3T:T:: U)rOr?rH)NNNNrM)NNN)NNiir2g"@Ngr=r=NNNNpilTNr=)+r#r$r%r&_last_supported_versionmodel_cpu_offload_seqrr r r r rrrBpropertyrQrTrr(r)intrrrrrrrno_gradrEXAMPLE_DOC_STRINGr rirrr Generatorboolrrrr/ __classcell__)rJs@r1r3r3cs '&Eg g$g! g 5 g . gg!g2$$ ((049=#'g5   - g5!) 6g5C=g5R @.!.#/bDnr2 ]]_12)-$( ##$&#% #;?/0+,MQ*.049=%* ;?#$+e4c49n%e4"e4 e4 } e4 SM e4e4!e4e4"%T#Y"78e4e4 (}e4$C=e4E%//43H"HIJe4%,,'e4   -!e4"!) 6#e4$c]%e4&'e4(!)c3h 8)e4*C=+e43e4r0r3rc~[U[5(a9[US[R5(a[R"US5n[U[R5(aOUR S:XaUR S5n[R"U5n[R"U5nUnU$)Nrr) rhrjr(r)rndimrr pt_to_numpy numpy_to_pil)r image_numpy image_pils r1_convert_pt_to_pilrs%:eAh #E#E %#%&& ::?OOA&E'33E: %22;?  Lr0 resolutionc[U5n[U[5(a;UVs/sH,o"RU[R R 5PM. nnU$URU[R R 5nU$s snfrM)rrhrjresizer,r-BILINEAR)rrus r1rrsp u %E%CHI5a*cii&8&895I L Z););< LJs3Bc [U5n[U[5(a[USRSUS- USRSUS- 5nUVs/sHXo3R [ URU-5[ URU-54[RRS9PMZ nnUSRUS- S-nUSRUS- S-nUVs/sH o3RXEXAS-XQS-45PM" nnU$[URSUS- URSUS- 5nUR [ URU-5[ URU-54[RRS9nURUS- S-nURUS- S-nURXEXAS-XQS-45nU$s snfs snf)Nrr=)resampler) rrhrjminsizerroundrrr,r-BOXcrop)rrrrx1y1s r1rrs u %E%E!HMM!$z!}4eAhmmA6FTU6VWpuvpukl5E!12E!((e:K4LMX[XaXaXeXefpuvAhnnz!}, 2Ahoo 1 -! 3SXYSXam!3RQ-5GHISXY EJJqMJqM15::a=:a=3PQ eEKK5$895QVAV;WXcfclclcpcp qkkJqM)a /llZ]*q 0 BBA$6]8JKL w Zs AG/2'G4):r dataclassesrtypingrrrrrr numpyr*r,r( transformersr r r r image_processorrrmodelsrmodels.unets.unet_i2vgen_xlr schedulersrutilsrrrrutils.torch_utilsrrHrpipeline_utilsrrrtorch_xla.core.xla_modelcore xla_modelrr get_loggerr#rpr rr3r)rr-r rrr"r0r1r5ss!:: hhD#7' .-]]))MM   H %D  IZ I  IB 4B 4R eELL$u||2D$DE   tELL1399??DDYY Z hmnqsvnvhw  tELL1399??DDYY Zhmnqsvnvhwr0