+h SSKrSSKJrJrJrJrJrJrJr SSK r SSK J s J r SSKJrJrJrJr SSKJr SSKJrJrJrJr SSKJrJrJrJrJrJ r SSK!J"r" SS K#J$r$ SS K%J&r& SS K'J(r(J)r)J*r*J+r+J,r, SS K-J.r.J/r/ SS K0J1r1 SSK2J3r3 SSK4J5r5 SSK6J7r7J8r8 SSK9J:r: \)"5(a SSK;J Sr?OSr?\*R"\A5rBSrC"SS\7\8\\\\3\5\5 rDg)N)AnyCallableDictListOptionalTupleUnion)CLIPImageProcessor CLIPTextModel CLIPTokenizerCLIPVisionModelWithProjection)PipelineImageInput)FromSingleFileMixinIPAdapterMixinStableDiffusionLoraLoaderMixinTextualInversionLoaderMixin) AutoencoderKLControlNetModelImageProjectionMultiControlNetModelUNet2DConditionModelUNetMotionModel)adjust_lora_scale_text_encoder) MotionAdapter)KarrasDiffusionSchedulers)USE_PEFT_BACKENDis_torch_xla_availableloggingscale_lora_layersunscale_lora_layers)is_compiled_module randn_tensor)VideoProcessor) FreeInitMixin)AnimateDiffFreeNoiseMixin)DiffusionPipelineStableDiffusionMixin)AnimateDiffPipelineOutputTFaT Examples: ```py >>> import torch >>> from diffusers import ( ... AnimateDiffControlNetPipeline, ... AutoencoderKL, ... ControlNetModel, ... MotionAdapter, ... LCMScheduler, ... ) >>> from diffusers.utils import export_to_gif, load_video >>> # Additionally, you will need a preprocess videos before they can be used with the ControlNet >>> # HF maintains just the right package for it: `pip install controlnet_aux` >>> from controlnet_aux.processor import ZoeDetector >>> # Download controlnets from https://huggingface.co/lllyasviel/ControlNet-v1-1 to use .from_single_file >>> # Download Diffusers-format controlnets, such as https://huggingface.co/lllyasviel/sd-controlnet-depth, to use .from_pretrained() >>> controlnet = ControlNetModel.from_single_file("control_v11f1p_sd15_depth.pth", torch_dtype=torch.float16) >>> # We use AnimateLCM for this example but one can use the original motion adapters as well (for example, https://huggingface.co/guoyww/animatediff-motion-adapter-v1-5-3) >>> motion_adapter = MotionAdapter.from_pretrained("wangfuyun/AnimateLCM") >>> vae = AutoencoderKL.from_pretrained("stabilityai/sd-vae-ft-mse", torch_dtype=torch.float16) >>> pipe: AnimateDiffControlNetPipeline = AnimateDiffControlNetPipeline.from_pretrained( ... "SG161222/Realistic_Vision_V5.1_noVAE", ... motion_adapter=motion_adapter, ... controlnet=controlnet, ... vae=vae, ... ).to(device="cuda", dtype=torch.float16) >>> pipe.scheduler = LCMScheduler.from_config(pipe.scheduler.config, beta_schedule="linear") >>> pipe.load_lora_weights( ... "wangfuyun/AnimateLCM", weight_name="AnimateLCM_sd15_t2v_lora.safetensors", adapter_name="lcm-lora" ... ) >>> pipe.set_adapters(["lcm-lora"], [0.8]) >>> depth_detector = ZoeDetector.from_pretrained("lllyasviel/Annotators").to("cuda") >>> video = load_video( ... "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/animatediff-vid2vid-input-1.gif" ... ) >>> conditioning_frames = [] >>> with pipe.progress_bar(total=len(video)) as progress_bar: ... for frame in video: ... conditioning_frames.append(depth_detector(frame)) ... progress_bar.update() >>> prompt = "a panda, playing a guitar, sitting in a pink boat, in the ocean, mountains in background, realistic, high quality" >>> negative_prompt = "bad quality, worst quality" >>> video = pipe( ... prompt=prompt, ... negative_prompt=negative_prompt, ... num_frames=len(video), ... num_inference_steps=10, ... guidance_scale=2.0, ... conditioning_frames=conditioning_frames, ... generator=torch.Generator().manual_seed(42), ... ).frames[0] >>> export_to_gif(video, "animatediff_controlnet.gif", fps=8) ``` c8^\rSrSrSrSrSS/r/SQrSES\S \ S \ S \ \ \ 4S \S \ \\\\\\4S\S\\S\\4U4SjjjrSFS\\R2S\\R2S\\S\\4SjjrSGSjrSrSHS\4SjjrSr SISjr!SGSjr"SJS jr#\$S!5r%\$S"5r&\$S#5r'\$S$5r(\$S%5r)\$S&5r*\RV"5SSSSS'S(SS)SSSSSSSSS*S+SSSSSSSS,/S4S-\ \,\\,4S.\\S/\\S0\\S1\S2\S3\\ \,\\,4S4\\S5\S6\\ \RZ\\RZ4S,\\R2S\\R2S\\R2S7\\.S8\\.S9\\\.S:\\,S;\/S<\\0\,\14S=\ \\\4S>\/S?\ \\\4S@\ \\\4S\\SA\\2\\\0/S4SB\\,S\46SCjj5r3SDr4U=r5$)KAnimateDiffControlNetPipelinexa3 Pipeline for text-to-video generation with ControlNet guidance. This model inherits from [`DiffusionPipeline`]. Check the superclass documentation for the generic methods implemented for all pipelines (downloading, saving, running on a particular device, etc.). The pipeline also inherits the following loading methods: - [`~loaders.TextualInversionLoaderMixin.load_textual_inversion`] for loading textual inversion embeddings - [`~loaders.StableDiffusionLoraLoaderMixin.load_lora_weights`] for loading LoRA weights - [`~loaders.StableDiffusionLoraLoaderMixin.save_lora_weights`] for saving LoRA weights - [`~loaders.IPAdapterMixin.load_ip_adapter`] for loading IP Adapters Args: vae ([`AutoencoderKL`]): Variational Auto-Encoder (VAE) Model to encode and decode images to and from latent representations. text_encoder ([`CLIPTextModel`]): Frozen text-encoder ([clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14)). tokenizer (`CLIPTokenizer`): A [`~transformers.CLIPTokenizer`] to tokenize text. unet ([`UNet2DConditionModel`]): A [`UNet2DConditionModel`] used to create a UNetMotionModel to denoise the encoded video latents. motion_adapter ([`MotionAdapter`]): A [`MotionAdapter`] to be used in combination with `unet` to denoise the encoded video latents. scheduler ([`SchedulerMixin`]): A scheduler to be used in combination with `unet` to denoise the encoded image latents. Can be one of [`DDIMScheduler`], [`LMSDiscreteScheduler`], or [`PNDMScheduler`]. ztext_encoder->unet->vaefeature_extractor image_encoder)latents prompt_embedsnegative_prompt_embedsNvae text_encoder tokenizerunetmotion_adapter controlnet schedulerc >[T U]5 [U[5(a[R "XE5n[U[ [45(a [U5nURUUUUUUUUU S9 [USS5(a/S[URRR5S- -OSUl[!URS9Ul[!URSSS 9Ulg) N) r4r5r6r7r8r9r:r/r0r4r%r*)vae_scale_factorTF)r=do_convert_rgb do_normalize)super__init__ isinstancerr from_unet2dlisttuplerregister_modulesgetattrlenr4configblock_out_channelsr=r$video_processorcontrol_video_processor) selfr4r5r6r7r8r9r:r/r0 __class__s y/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/pipelines/animatediff/pipeline_animatediff_controlnet.pyrA&AnimateDiffControlNetPipeline.__init__s  d0 1 1"..tDD j4- 0 0-j9J %)!/'  W^^bdikoVpVpc$((//*L*L&MPQ&Q Rvw-t?T?TU'5!224V[( $r2r3 lora_scale clip_skipc  UbS[U[5(a>Xl[(d[ UR U5 O[ UR U5 Ub[U[5(aSn O3Ub![U[5(a [U5n OURSn UGc[U[5(aURXR5nURUSURRSSS9n U Rn URUSSS 9Rn U RS U RS :a[ R""X5(djURR%U SS2URRS- S 245n[&R)S URRS U35 [+UR R,S 5(aAUR R,R.(aU R0R3U5nOSnU c%UR U R3U5US9nUSnOQUR U R3U5USS9nUS U S-*nUR R4R7U5nUR bUR R8nO0UR:bUR:R8nO UR8nUR3UUS9nURunnnUR=SUS5nUR?UU-US 5nU(GaUGc|UcS/U -nOUb;[AU5[AU5La$[CS[AU5S[AU5S35e[U[5(aU/nO2U [U5:wa![ESUS[U5SUSU S3 5eUn[U[5(aURUUR5nURSnURUSUSSS9n[+UR R,S 5(aAUR R,R.(aUR0R3U5nOSnUR URR3U5US9nUSnU(aGURSnUR3UUS9nUR=SUS5nUR?X-US 5nUR b6[U[5(a![(a[GUR U5 Xg4$)a, Encodes the prompt into text encoder hidden states. Args: prompt (`str` or `List[str]`, *optional*): prompt to be encoded device: (`torch.device`): torch device num_images_per_prompt (`int`): number of images that should be generated per prompt do_classifier_free_guidance (`bool`): whether to use classifier free guidance or not negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts not to guide the image generation. If not defined, one has to pass `negative_prompt_embeds` instead. Ignored when not using guidance (i.e., ignored if `guidance_scale` is less than `1`). prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, text embeddings will be generated from `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs, *e.g.* prompt weighting. If not provided, negative_prompt_embeds will be generated from `negative_prompt` input argument. lora_scale (`float`, *optional*): A LoRA scale that will be applied to all LoRA layers of the text encoder if LoRA layers are loaded. clip_skip (`int`, *optional*): Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. Nr*r max_lengthTpt)paddingrU truncationreturn_tensorslongest)rWrYz\The following part of your input was truncated because CLIP can only handle sequences up to z tokens: use_attention_mask)attention_mask)r]output_hidden_states)dtypedevicez?`negative_prompt` should be the same type to `prompt`, but got z != .z`negative_prompt`: z has batch size z, but `prompt`: zT. Please make sure that passed `negative_prompt` matches the batch size of `prompt`.)$rBr _lora_scalerrr5r strrDrHshapermaybe_convert_promptr6model_max_length input_idstorchequal batch_decodeloggerwarninghasattrrIr\r]to text_modelfinal_layer_normr_r7repeatviewtype TypeError ValueErrorr!)rMpromptr`num_images_per_promptdo_classifier_free_guidancenegative_promptr2r3rRrS batch_size text_inputstext_input_idsuntruncated_ids removed_textr]prompt_embeds_dtypebs_embedseq_len_ uncond_tokensrU uncond_inputs rO encode_prompt+AnimateDiffControlNetPipeline.encode_promptsQV  !j7U&V&V) $#.t/@/@*M!$"3"3Z@  *VS"9"9J  Jvt$<$<VJ&,,Q/J  $ ;<<226>>J..$>>::# )K)22N"nnVYW[n\ffO$$R(N,@,@,DDU[[NN $~~::#At~~'F'F'JR'O$OP  778 ,Q t((//1EFF4K\K\KcKcKvKv!,!;!;!>!>v!F!% $ 1 1.2C2CF2K\j 1 k -a 0 $ 1 1"%%f-ncg!2! !.b 1IM2B C !% 1 1 < < M Mm \    ("&"3"3"9"9  YY ""&))// "/"5"5 %((/B6(R ,22'1%,,Q0EqI %**86K+KWVXY  '+A+I&!#z 1 #V D>$%# *Lt((//1EFF4K\K\KcKcKvKv!-! " &,2215G%;%>%>EXag%>%h "%;%B%B1F[]^%_ "%;%@%@Acelnp%q "    ($ >??DTDT#D$5$5zB44rQcd[URR55Rn[ U[ R 5(dURUSS9RnURX%S9nU(aqURUSS9RSnURUSS9nUR[ R"U5SS9RSnURUSS9nXg4$URU5RnURUSS9n[ R"U5n X4$) NrV)rYr`r_T)r^rdim)nextr0 parametersr_rBriTensorr/ pixel_valuesro hidden_statesrepeat_interleave zeros_like image_embeds) rMimager`rxr^r_image_enc_hidden_statesuncond_image_enc_hidden_statesruncond_image_embedss rO encode_image*AnimateDiffControlNetPipeline.encode_image~s?T''2245;;%..**5*FSSE4 &*&8&8UY&8&Z&h&hik&l #&=&O&OPekl&O&m #-1-?-?  'd.@.mB. *.L-]-]%1.^. *+J J--e4AAL'99:OUV9WL"'"2"2<"@ 4 4rQc /nU(a/nUGc&[U[5(dU/n[U5[URRR 5:waB[ S[U5S[URRR 5S35e[XRRR 5Hhup[U [5(+n URXSU 5upURU SSS245 U(dMPWRU SSS245 Mj OEUH?n U(a$U RS5upWRU 5 URU 5 MA /n[U5Hwup[R"U /U-SS9n U(a2[R"WU /U-SS9n [R"X/SS9n U RUS9n URU 5 My U$) NzK`ip_adapter_image` must have same length as the number of IP Adapters. Got z images and z IP Adapters.r*r%rrr`)rBrDrHr7encoder_hid_projimage_projection_layersrvziprrappendchunk enumeratericatro)rMip_adapter_imageip_adapter_image_embedsr`rxryrnegative_image_embedssingle_ip_adapter_imageimage_proj_layeroutput_hidden_statesingle_image_embedssingle_negative_image_embedsis rOprepare_ip_adapter_image_embeds=AnimateDiffControlNetPipeline.prepare_ip_adapter_image_embedss9 &$& ! " *.55$4#5 #$DII,F,F,^,^(__ abefvbwaxyEFIJNJSJSJdJdJ|J|F}E~~KL>A ))"<"<"T"T>9'+55E*W&W#DHDUDU+Q8KEA###$7a$@A..)001MdTUg1VW>(?#.H[HaHabcHdE0)001MN##$78 (? #%&/ &= "A"'))-@,ADY,Y_`"a */4yy:OPQ:R9SVk9kqr/s,&+ii1M0cij&k#"5"8"8"8"G  # * *+> ?'>'&rQdecode_chunk_sizecFSURRR- U-nURup4pVnUR SSSSS5R X5-XFU5n/n[ SURSU5H?n XX-n URRU 5Rn URU 5 MA [R"U5nUSSS24R X5S4URSS-5R SSSSS5nUR5nU$)Nr*rr%rr[) r4rIscaling_factorrepermutereshaperangedecodesamplerrirfloat) rMr1rr{channels num_framesheightwidthvideor batch_latentss rOdecode_latents,AnimateDiffControlNetPipeline.decode_latentssdhhoo444w>:A--7 j%//!Q1a0889PRZdijq'--*,=>A#(=>M HHOOM:AAM LL '?  % dAg&& 'CekkRSRTo'UV^^_`bcefhiklm  rQcnS[[R"URR5R R 55;n0nU(aX$S'S[[R"URR5R R 55;nU(aXS'U$)Neta generator)setinspect signaturer:steprkeys)rMrr accepts_etaextra_step_kwargsaccepts_generators rOprepare_extra_step_kwargs7AnimateDiffControlNetPipeline.prepare_extra_step_kwargss s7#4#4T^^5H5H#I#T#T#Y#Y#[\\  '*e $(3w/@/@ATAT/U/`/`/e/e/g+hh -6k *  rQ?c  ^^ US-S:wd US-S:wa[SUSUS35eUbW[U4SjU55(d=[STRSUV s/sHoTR;dMU PM sn 35eUbUb[S US US 35eUcUc [S 5eUb7[U[[ [ 45(d[S [U535eUbUb[SUSUS 35eUbCUb@URUR:wa&[SURSURS35e[TR[5(aW[U[ 5(aB[RS[TRR5S[U5S35 [[ S5=(a8 [TR["R$R&R(5n[TR[*5(d0U(a[TRR,[*5(aW[T [ 5(d[/S[T 535e[T 5U:wa[SUS[T 5<35eO[TR[5(d0U(a[TRR,[5(a[T [ 5(a[T S[ 5(d[/S[T 5<35e[T S5U:wa[SUS[T S5<35e[1U 4SjT 55(a [S5eOe[TR[*5(d0U(aJ[TRR,[*5(a![U [25(d [/S5eO[TR[5(d0U(a[TRR,[5(a[U [ 5(a#[1SU 55(a [S 5eOO[U [ 5(a7[U 5[TRR5:wa [S!5eOe[U [4[ 45(dU /n [U [4[ 45(dU /n [U 5[U 5:wa$[S"[U 5S#[U 5S$35e[TR[5(a[U 5[TRR5:wa[[S%U S&[U 5S'[TRR5S([TRR5S3 5e[7X5HHunnUU:a[S)US*US35eUS+:a[S)US,35eUS-:dM;[S.US/35e gs sn f)0Nr<rz7`height` and `width` have to be divisible by 8 but are z and rbc3@># UHoTR;v M g7fN)_callback_tensor_inputs).0krMs rO =AnimateDiffControlNetPipeline.check_inputs..sF 7Y!-- -7Ysz2`callback_on_step_end_tensor_inputs` has to be in z , but found zCannot forward both `prompt`: z and `prompt_embeds`: z2. Please make sure to only forward one of the two.zeProvide either `prompt` or `prompt_embeds`. Cannot leave both `prompt` and `prompt_embeds` undefined.z:`prompt` has to be of type `str`, `list` or `dict` but is z'Cannot forward both `negative_prompt`: z and `negative_prompt_embeds`: zu`prompt_embeds` and `negative_prompt_embeds` must have the same shape when passed directly, but got: `prompt_embeds` z != `negative_prompt_embeds` z You have z! ControlNets and you have passed z= prompts. The conditionings will be fixed across the prompts.scaled_dot_product_attentionz>For single controlnet, `image` must be of type `list` but got zExcepted image to have length z but got len(video)=zQFor multiple controlnets: `image` must be type list of lists but got type(video)=z$Expected length of image sublist as z but got len(video[0])=c3X># UHn[U5[TS5:gv M! g7f)rN)rH)rimgrs rOrr;s">3s8s58},s'*zDAll conditioning frame batches for multicontrolnet must be same sizezLFor single controlnet: `controlnet_conditioning_scale` must be type `float`.c3B# UHn[U[5v M g7fr)rBrD)rrs rOrrNsR4Qqz!T**4QszEA single batch of multiple conditionings are supported at the moment.zFor multiple controlnets: When `controlnet_conditioning_scale` is specified as `list`, it must have the same length as the number of controlnetsz`control_guidance_start` has z* elements, but `control_guidance_end` has zI elements. Make sure to provide the same number of elements to each list.z`control_guidance_start`: z has z elements but there are z- controlnets available. Make sure to provide zcontrol guidance start: z4 cannot be larger or equal to control guidance end: rz can't be smaller than 0.rzcontrol guidance end: z can't be larger than 1.0.)rvallrrBrdrDdictrtrer9rrlrmrHnetsrnFri_dynamo eval_frameOptimizedModuler _orig_modruanyrrEr)rMrwrrrrzr2r3"callback_on_step_end_tensor_inputsrcontrolnet_conditioning_scalecontrol_guidance_startcontrol_guidance_endr is_compiledstartends` ` rO check_inputs*AnimateDiffControlNetPipeline.check_inputss A:?eai1nVW]V^^cdicjjklm m - 9#F 7YF C C DTEaEaDbbn|^pH|^vwkokGkGbGpq|^pHoIJ   -";08N}o^00 ^ 5w   6Ct;L(M(MYZ^_eZfYghi i  &+A+M9/9JK*++]_   $)?)K""&<&B&BB --:-@-@,AB.445Q8 doo'; < <&$''DOO$8$8 9::[\_`f\g[hST a!?@ Z OOU]]55EEF  t 8 84??44oFFeT**"`aefkal`m noo5zZ' #A*MbWZ[`WaVc!dee( t(< = =4??446JKKeT***U1Xt2L2L"thlmrhsgu vww58} * #G |Sk]`afghai]j\l!mnn>>>> !ghh? 5 t 8 84??44oFF;UCC nooD t(< = =4??446JKK7>>R4QRRR$%lmmS94@@SIfEgkn$$lF!D 505$-@@&<%= ". >>$8#9 % &#.B*C C/4J0K/LLvwz|PxQwRR[\  doo'; < <)*c$//2F2F.GG 01G0HcRhNiMjkCDGHLHWHWH\H\D]C^^KLOPTP_P_PdPdLeKffgh4KJE3| .ug5ijminnops{ #;E7B[!\]]Sy #9#>X!YZZLYpHs Y2Yc |UR(aURXX4XVXxU 5 n [U[5(a*[ U5U:wa[ S[ U5SUS35eUUUX@R -XPR -4n U c [XXvS9n OU RU5n XRR-n U $)Nz/You have passed a list of generators of length z+, but requested an effective batch size of z@. Make sure the batch size matches the length of the generators.)rr`r_) free_noise_enabled_prepare_latents_free_noiserBrDrHrvr=r#ror:init_noise_sigma) rMr{num_channels_latentsrrrr_r`rr1res rOprepare_latents-AnimateDiffControlNetPipeline.prepare_latentsvs  " "66*eTZgnG i & &3y>Z+GA#i.AQR&<'gi    ++ + ** *   ?"5fZGjj(GNN;;;rQFc |URRXUS9R[RS9nUR SSSSS5R SS5nURSn U S:XaUn OUn URU SS9nURXgS 9nU(a!U (d[R"U/S-5nU$) N)rr)r_rr%r*rrrr) rLpreprocess_videororifloat32rflattenrerr) rMrrrr{num_videos_per_promptr`r_ry guess_modevideo_batch_size repeat_bys rO prepare_video+AnimateDiffControlNetPipeline.prepare_videos,,==eZ_=`cc--d  aAq!,44Q: ;;q> q "I.I'' q'94 &zIIugk*E rQcUR$r_guidance_scalerMs rOguidance_scale,AnimateDiffControlNetPipeline.guidance_scales###rQcUR$r) _clip_skiprs rOrS'AnimateDiffControlNetPipeline.clip_skip rQc URS:$)Nr*rrs rOry9AnimateDiffControlNetPipeline.do_classifier_free_guidances##a''rQcUR$r)_cross_attention_kwargsrs rOcross_attention_kwargs4AnimateDiffControlNetPipeline.cross_attention_kwargss+++rQcUR$r)_num_timestepsrs rO num_timesteps+AnimateDiffControlNetPipeline.num_timestepss"""rQcUR$r) _interruptrs rO interrupt'AnimateDiffControlNetPipeline.interruptr rQ2g@r*pilTr1rwrrrnum_inference_stepsrrzrrrrrconditioning_frames output_type return_dictrrrrrcallback_on_step_endrc[UR5(aURRO URn[U[5(d%[U[5(a[ U5U/-nO[U[5(d%[U[5(a[ U5U/-nOb[U[5(dM[U[5(d8[U[ 5(a[ UR5OSnUU/-UU/-nnU=(d- URRRUR-nU=(d- URRRUR-nSnURUUUUUUU U UUUUS9 X`l UUlUUlSUlUb[U["[$45(aSnO3Ub![U[5(a [ U5nOU R&SnUR(n[U[ 5(a.[U[*5(aU/[ UR5-n[U[,5(aURR.O"URSRR.n U=(d U nUbUR1SS5OSn!UR2(a/UR5UUUUUR6UU U U!UR8S9 upOeUR;UUUUR6UU U U!UR8S9 upUR6(a[<R>"X/5n U RAUSS 9n UcUb"URCUUUUU-UR65n"[U[,5(a2UREUUUUU-U-UUURFUR6US 9 nOg[U[ 5(aP/n#UHEn$UREU$UUUU-U-UUURFUR6US 9 n%U#RIU%5 MG U#nOeURJRMUUS 9 URJRNn&URRRPn'URSUU-U'UUUU RFUU U 5 n URUX5n(UcUbS W"0OSn)/n*[W[ U&55Hn+[YUU5V,V-s/sH>un,n-S [+U+[ U&5- U,:=(d U+S-[ U&5- U-:5- PM@ n.n,n-U*RI[U[,5(aU.SOU.5 M URZ(a UR\OSn/[WU/5GHon0URZ(a#UR_U U0UUU RFU 5un n&[ U&5Ul0[ U&5XPRJRb-- n1UReUR`S9n2[gU&5GHun+n3URh(aMUR6(a[<R>"U /S-5OU n4URJRkU4U35n4U(aDUR6(a3U n5URJRkU5U35n5U RmS5Sn6OU4n5U n6[U*U+[5(a([YUU*U+5V7V,s/sH un7n,U7U,-PM n8n7n,O$Un9[U9[5(aU9Sn9U9U*U+-n8[<Rn"U5SS5n5U5RqSU5R&SU5R&SU5R&S45n5URU5U3U6UU8USS9un:n;URU4U3U URrU)U:U;S9RtnU=UU>U=- --n 1`. negative_prompt (`str` or `List[str]`, *optional*): The prompt or prompts to guide what to not include in image generation. If not defined, you need to pass `negative_prompt_embeds` instead. Ignored when not using guidance (`guidance_scale < 1`). eta (`float`, *optional*, defaults to 0.0): Corresponds to parameter eta (η) from the [DDIM](https://huggingface.co/papers/2010.02502) paper. Only applies to the [`~schedulers.DDIMScheduler`], and is ignored in other schedulers. generator (`torch.Generator` or `List[torch.Generator]`, *optional*): A [`torch.Generator`](https://pytorch.org/docs/stable/generated/torch.Generator.html) to make generation deterministic. latents (`torch.Tensor`, *optional*): Pre-generated noisy latents sampled from a Gaussian distribution, to be used as inputs for video generation. Can be used to tweak the same generation with different prompts. If not provided, a latents tensor is generated by sampling using the supplied random `generator`. Latents should be of shape `(batch_size, num_channel, num_frames, height, width)`. prompt_embeds (`torch.Tensor`, *optional*): Pre-generated text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, text embeddings are generated from the `prompt` input argument. negative_prompt_embeds (`torch.Tensor`, *optional*): Pre-generated negative text embeddings. Can be used to easily tweak text inputs (prompt weighting). If not provided, `negative_prompt_embeds` are generated from the `negative_prompt` input argument. ip_adapter_image (`PipelineImageInput`, *optional*): Optional image input to work with IP Adapters. ip_adapter_image_embeds (`List[torch.Tensor]`, *optional*): Pre-generated image embeddings for IP-Adapter. It should be a list of length same as number of IP-adapters. Each element should be a tensor of shape `(batch_size, num_images, emb_dim)`. It should contain the negative image embedding if `do_classifier_free_guidance` is set to `True`. If not provided, embeddings are computed from the `ip_adapter_image` input argument. conditioning_frames (`List[PipelineImageInput]`, *optional*): The ControlNet input condition to provide guidance to the `unet` for generation. If multiple ControlNets are specified, images must be passed as a list such that each element of the list can be correctly batched for input to a single ControlNet. output_type (`str`, *optional*, defaults to `"pil"`): The output format of the generated video. Choose between `torch.Tensor`, `PIL.Image` or `np.array`. return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~pipelines.text_to_video_synthesis.TextToVideoSDPipelineOutput`] instead of a plain tuple. cross_attention_kwargs (`dict`, *optional*): A kwargs dictionary that if specified is passed along to the [`AttentionProcessor`] as defined in [`self.processor`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py). controlnet_conditioning_scale (`float` or `List[float]`, *optional*, defaults to 1.0): The outputs of the ControlNet are multiplied by `controlnet_conditioning_scale` before they are added to the residual in the original `unet`. If multiple ControlNets are specified in `init`, you can set the corresponding scale as a list. guess_mode (`bool`, *optional*, defaults to `False`): The ControlNet encoder tries to recognize the content of the input image even if you remove all prompts. A `guidance_scale` value between 3.0 and 5.0 is recommended. control_guidance_start (`float` or `List[float]`, *optional*, defaults to 0.0): The percentage of total steps at which the ControlNet starts applying. control_guidance_end (`float` or `List[float]`, *optional*, defaults to 1.0): The percentage of total steps at which the ControlNet stops applying. clip_skip (`int`, *optional*): Number of layers to be skipped from CLIP while computing the prompt embeddings. A value of 1 means that the output of the pre-final layer will be used for computing the prompt embeddings. callback_on_step_end (`Callable`, *optional*): A function that calls at the end of each denoising steps during the inference. The function is called with the following arguments: `callback_on_step_end(self: DiffusionPipeline, step: int, timestep: int, callback_kwargs: Dict)`. `callback_kwargs` will include a list of all tensors as specified by `callback_on_step_end_tensor_inputs`. callback_on_step_end_tensor_inputs (`List`, *optional*): The list of tensor inputs for the `callback_on_step_end` function. The tensors specified in the list will be passed as `callback_kwargs` argument. You will only be able to include variables listed in the `._callback_tensor_inputs` attribute of your pipeline class. Examples: Returns: [`~pipelines.animatediff.pipeline_output.AnimateDiffPipelineOutput`] or `tuple`: If `return_dict` is `True`, [`~pipelines.animatediff.pipeline_output.AnimateDiffPipelineOutput`] is returned, otherwise a `tuple` is returned where the first element is a list with the generated frames. r*) rwrrrrzrr2r3rrrrFNrscale) rwrr`rryrzr2r3rRrS)r2r3rRrS)repeatsr) rrrr{rr`r_ryrrrr)totalr%r[rr)encoder_hidden_statescontrolnet_condconditioning_scalerr)r#radded_cond_kwargsdown_block_additional_residualsmid_block_additional_residualr1r2r3latent)rr)frames)Hr"r9rrBrDrHrrr7rI sample_sizer=rrrr rrdrre_execution_devicerrglobal_pool_conditionsgetr_encode_prompt_free_noiseryrSrrirrrrr_rr: set_timesteps timesteps in_channelsrrrrfree_init_enabled_free_init_num_iters_apply_free_initrorder progress_barrrscale_model_inputr transposerrrr prev_samplelocalspopupdate XLA_AVAILABLExm mark_steprrKpostprocess_videomaybe_free_model_hooksr+)DrMrwrrrrrrzrrrr1r2r3rrrrrrrrrrrSrrrr9multr{r`r-text_encoder_lora_scalercond_prepared_videosframe_prepared_videor1rrr&controlnet_keeprsekeepsnum_free_init_itersfree_init_iternum_warmup_stepsr7tlatent_model_inputcontrol_model_inputcontrolnet_prompt_embedsc cond_scalecontrolnet_cond_scaledown_block_res_samplesmid_block_res_sample noise_prednoise_pred_uncondnoise_pred_textcallback_kwargsrcallback_outputsr video_tensorsD rO__call__&AnimateDiffControlNetPipeline.__call__s p3ET__2U2UT__..[_[j[j 0$77JG[]aBXAY%Y "0$77JG]_cBVAW#W 2D99*MacgBhBh+5jBV+W+W3z']^D.//,--%9 " O499++77$:O:OOM))558M8MM ! !+/Q'#9%*G#9!5   .#'=$  *Vc4["A"AJ  Jvt$<$<VJ&,,Q/J'' j"6 7 7JGdfk=9p ( "E..w8IJL((::[f:gE ##%8O(66WH&p'>=s-?Ae,%C(e8 e2 G e8-e82e88 f ) rr rrrcrrLr=rK)NN)NNNNNr)r)NNNNNrrr)FF)6__name__ __module__ __qualname____firstlineno____doc__model_cpu_offload_seq_optional_componentsrrr r r rrrrrrrrrr r rArirrintrrrrrrrrpropertyrrSryrrrno_gradrd Generatorrboolrrrr^__static_attributes__ __classcell__)rNs@rOr-r-xsY86/AT;?AE" " $" ! " (/9: " & " /4+@%BXZnno" -" $$67"  =>" " V049=&*#'t5  -t5!) 6t5UOt5C=t5n52+'\&!.#+/&)" J[\nrR%*@$$ ((,,## ]]_)-$& $##% #;?/0MQ*.049=9=@DBF%* ;?CF ?#L7$c]%L7&'L7(!)c3h 8)L7*(-UDK-?'@+L7,-L7.!&eT%[&8 9/L70$E4;$671L72C=3L74'xc40@$0F'GH5L76-1I7L789L7L7rQr-)Ertypingrrrrrrr ritorch.nn.functionalnn functionalr transformersr r r r image_processorrloadersrrrrmodelsrrrrrr models.lorarmodels.unets.unet_motion_modelr schedulersrutilsrrrr r!utils.torch_utilsr"r#rKr$free_init_utilsr&free_noise_utilsr'pipeline_utilsr(r)pipeline_outputr+torch_xla.core.xla_modelcore xla_modelr?r> get_loggerr`rlEXAMPLE_DOC_STRINGr-rQrOrsDDD hh1ww:;3nnA-+8D6))MM   H %?Df7"f7rQ