+h~PSSKJrJrJrJrJr SSKrSSKJr SSK rSSK J r J r SSK Jr SSKJr SSKJr SS KJrJr SS KJrJrJrJrJrJr SS KJrJr SS K J!r! SS K"J#r# SSK$J%r%J&r&J'r' SSK(J)r) \RT"\+5r,"SS\RZ5r."SS\!\ \5r/g))AnyDictOptionalTupleUnionN) ConfigMixinregister_to_config)UNet2DConditionLoadersMixin)logging)get_activation) Attention FeedForward)ADDED_KV_ATTENTION_PROCESSORSCROSS_ATTENTION_PROCESSORSAttentionProcessorAttnAddedKVProcessor AttnProcessorFusedAttnProcessor2_0)TimestepEmbedding Timesteps) ModelMixin)TransformerTemporalModel)UNetMidBlock3DCrossAttnget_down_block get_up_block)UNet3DConditionOutputc^\rSrSrSS\S\S\S\S\S\\S\4U4S jjjrS \ RS \ R4S jr S r U=r $)"I2VGenXLTransformerTemporalEncoder0dimnum_attention_headsattention_head_dim activation_fnupcast_attention ff_inner_dimdropoutc >[TU]5 [R"USSS9Ul[ UUUUSUSS9Ul[UUUSUSS9Ulg)NTh㈵>)elementwise_affineepsF) query_dimheadsdim_headr)biasr'out_bias)r)r& final_dropout inner_dimr1) super__init__nn LayerNormnorm1rattn1rff) selfr#r$r%r&r'r(r) __class__s _/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/models/unets/unet_i2vgen_xl.pyr6+I2VGenXLTransformerTemporalEncoder.__init__1se \\#$DI %'-   '"   hidden_statesreturncURU5nURUSS9nX1-nURS:XaURS5nUR U5nXA-nURS:XaURS5nU$)N)encoder_hidden_statesr)r9r:ndimsqueezer;)r<rAnorm_hidden_states attn_output ff_outputs r>forward*I2VGenXLTransformerTemporalEncoder.forwardOs"ZZ 6jj!34jP #3    ")11!4MGGM* !1    ")11!4Mr@)r:r;r9)gegluFNg)__name__ __module__ __qualname____firstlineno__intstrboolrr6torchTensorrK__static_attributes__ __classcell__r=s@r>r!r!0s %!&&*  !       sm   <|| r@r!cz^\rSrSrSrSr\S*S\\S\S\S\ \ S 4S \ \ S 4S \ \S 4S \S \\S\S\ \\ \4S\\ \\ \44U4Sjjj5r \ S\\ \44Sj5rS\ \\\ \444SjrS+S\\S\SS4SjjrSrSrSrSrSrSrS,S\R4S \ \R4\\4S!\R4S"\R4S#\\R4S$\\R4S%\\R4S&\\\ \4S'\S\ \\ \R444S(jjrS)r U=r!$)- I2VGenXLUNetaa I2VGenXL UNet. It is a conditional 3D UNet model that takes a noisy sample, conditional state, and a timestep and returns a sample-shaped output. This model inherits from [`ModelMixin`]. Check the superclass documentation for it's generic methods implemented for all models (such as downloading or saving). Parameters: sample_size (`int` or `Tuple[int, int]`, *optional*, defaults to `None`): Height and width of input/output sample. in_channels (`int`, *optional*, defaults to 4): The number of channels in the input sample. out_channels (`int`, *optional*, defaults to 4): The number of channels in the output. down_block_types (`Tuple[str]`, *optional*, defaults to `("CrossAttnDownBlock2D", "CrossAttnDownBlock2D", "CrossAttnDownBlock2D", "DownBlock2D")`): The tuple of downsample blocks to use. up_block_types (`Tuple[str]`, *optional*, defaults to `("UpBlock2D", "CrossAttnUpBlock2D", "CrossAttnUpBlock2D", "CrossAttnUpBlock2D")`): The tuple of upsample blocks to use. block_out_channels (`Tuple[int]`, *optional*, defaults to `(320, 640, 1280, 1280)`): The tuple of output channels for each block. layers_per_block (`int`, *optional*, defaults to 2): The number of layers per block. norm_num_groups (`int`, *optional*, defaults to 32): The number of groups to use for the normalization. If `None`, normalization and activation layers is skipped in post-processing. cross_attention_dim (`int`, *optional*, defaults to 1280): The dimension of the cross attention features. attention_head_dim (`int`, *optional*, defaults to 64): Attention head dim. num_attention_heads (`int`, *optional*): The number of attention heads. FN sample_size in_channels out_channelsdown_block_types.up_block_typesblock_out_channelslayers_per_blocknorm_num_groupscross_attention_dimr%r$c $ >[TU]5 U n [U5[U5:wa[SUSUS35e[U5[U5:wa[SUSUS35e[ U [ 5(d*[U 5[U5:wa[SU SUS35e[ R"X"-USSS S 9Ul[S U USS US 9Ul [ R"[ R"S US -SS S9[ R"5[ R"US -US -SS S S9[ R"5[ R"US -USS S S95Ul [USUS -USS9Ul[ R"[ R"S US -SS S9[ R"5[ R "S5[ R"US -US-SSS S9[ R"5[ R"US-U SSS S95UlUSS -n [%USSS5UlUSn [)XSS9Ul[ R"[ R,"X5[ R"5[ R,"XU-55Ul[ R"[ R,"X5[ R"5[ R,"X55Ul[ R2"/5Ul[ R2"/5Ul[ U [ 5(aU 4[U5-n USn[9U5HTunnUnXonU[U5S - :Hn[;UUUUU U(+SSUU XS SS9 nUR4R=U5 MV [?USU SSS U U SUSS9 Ul SUl![E[GU55n[E[GU 55nUSn[9U5HunnU[U5S - :HnUnUUnU[IUS -[U5S - 5nU(dSnU=RBS - sl!OSn[KUUS -UUUU USSUU UUSUS9nUR6R=U5 UnM [ RL"USUSS9Ul'[QS5Ul)[ R"USUSS S 9Ul*g)Nz\Must provide the same number of `down_block_types` as `up_block_types`. `down_block_types`: z. `up_block_types`: .zbMust provide the same number of `block_out_channels` as `down_block_types`. `block_out_channels`: z. `down_block_types`: zdMust provide the same number of `num_attention_heads` as `down_block_types`. `num_attention_heads`: rrr) kernel_sizepadding)r$r%r^ num_layersrdrE)ri)striderir gelu)r#r$r(r%r&) rnTsilu)act_fnr+F) rkr^r_ temb_channelsadd_downsample resnet_eps resnet_act_fn resnet_groupsrer$downsample_paddingdual_cross_attention) r^rrrtruoutput_scale_factorrer$rvrx) rkr^r_prev_output_channelrr add_upsamplertrurvrer$rxresolution_idx) num_channels num_groupsr-)+r5r6len ValueError isinstancerRr7Conv2dconv_inrtransformer_in SequentialSiLUimage_latents_proj_inr!image_latents_temporal_encoderAdaptiveAvgPool2dimage_latents_context_embeddingr time_projrtime_embeddingLinearcontext_embedding fps_embedding ModuleList down_blocks up_blocks enumeraterappendr mid_blocknum_upsamplerslistreversedminr GroupNorm conv_norm_outrconv_actconv_out)r<r]r^r_r`rarbrcrdrer%r$time_embed_dimtimestep_input_dimoutput_channelidown_block_type input_channelis_final_block down_blockreversed_block_out_channelsreversed_num_attention_heads up_block_typer{r|up_blockr=s r>r6I2VGenXLUNet.__init__~s2 1  C$7 7nooAAUVdUeefg  ! "c*:&; ;tvHuII_`p_qqrs -s33.1*2.( ! 3 3B 7)!&   '+84F+G&H#'+H5H,I'J$4Q7 ). 9 A}#&8"9A"==N"0 8;N7AE3GYCZ]^C^8_`M"# ##q(#$ #+a/)+$7,) $-$7$@$C%* H NN ! !( +"0 ?!:D \\7I!7LYhnst&v.  "4Q"7ST^_` r@rBc^0nS[S[RRS[[[ 44U4SjjmUR 5Hup#T"X#U5 M U$)z Returns: `dict` of attention processors: A dictionary containing all attention processors used in the model with indexed by its weight name. namemodule processorsc>[US5(aUR5X S3'UR5Hup4T"USU3XB5 M U$)N get_processor .processorrg)hasattrrnamed_children)rrrsub_namechildfn_recursive_add_processorss r>rAI2VGenXLUNet.attn_processors..fn_recursive_add_processorsHsZv//282F2F2H V:./#)#8#8#:+tfAhZ,@%T$; r@)rSrUr7Modulerrr)r<rrrrs @r>attn_processorsI2VGenXLUNet.attn_processors=sb  c 588?? X\]`bt]tXu !//1LD 'j A2r@ processorc d^[URR55n[U[5(a-[U5U:wa[ S[U5SUSUS35eS[ S[RR4U4SjjmUR5Hup4T"X4U5 M g) a Sets the attention processor to use to compute attention. Parameters: processor (`dict` of `AttentionProcessor` or only `AttentionProcessor`): The instantiated processor class or a dictionary of processor classes that will be set as the processor for **all** `Attention` layers. If `processor` is a dict, the key needs to define the path to the corresponding cross attention processor. This is strongly recommended when setting trainable attention processors. z>A dict of processors was passed, but the number of processors z0 does not match the number of attention layers: z. Please make sure to pass z processor classes.rrc >[US5(aJ[U[5(dURU5 O#URUR US355 UR 5Hup4T"USU3XB5 M g)N set_processorrrg)rrdictrpopr)rrrrrfn_recursive_attn_processors r>rDI2VGenXLUNet.set_attn_processor..fn_recursive_attn_processorlssv//!)T22((3(($z7J)KL#)#8#8#:+tfAhZ,@%S$;r@N) rrkeysrrrrSrUr7rr)r<rcountrrrs @r>set_attn_processorI2VGenXLUNet.set_attn_processorWsD((--/0 i & &3y>U+BPQTU^Q_P`a005w6QRWQXXkm   Tc T588?? T!//1LD 'i @2r@ chunk_sizer#c^US;a[SU35eU=(d SnS[RRS[S[4U4SjjmUR 5H nT"X1U5 M g) a Sets the attention processor to use [feed forward chunking](https://huggingface.co/blog/reformer#2-chunked-feed-forward-layers). Parameters: chunk_size (`int`, *optional*): The chunk size of the feed-forward layers. If not specified, will run feed-forward layer individually over each tensor of dim=`dim`. dim (`int`, *optional*, defaults to `0`): The dimension over which the feed-forward computation should be chunked. Choose between dim=0 (batch) or dim=1 (sequence length). )rrz-Make sure to set `dim` to either 0 or 1, not rrrr#c>[US5(aURXS9 UR5H nT"X1U5 M gNset_chunk_feed_forward)rr#rrchildrenrrr#rfn_recursive_feed_forwards r>rGI2VGenXLUNet.enable_forward_chunking..fn_recursive_feed_forward=v788---M*)%SA+r@N)rrUr7rrRr)r<rr#rrs @r>enable_forward_chunking$I2VGenXLUNet.enable_forward_chunkingzsn f LSERS S _1  Behhoo B3 BUX BmmoF %f# >&r@c^S[RRS[S[4U4SjjmUR 5H nT"USS5 M g)Nrrr#c>[US5(aURXS9 UR5H nT"X1U5 M grrrs r>rHI2VGenXLUNet.disable_forward_chunking..fn_recursive_feed_forwardrr@r)rUr7rrRr)r<rrs @r>disable_forward_chunking%I2VGenXLUNet.disable_forward_chunkingsH Behhoo B3 BUX BmmoF %fdA 6&r@c ~[SURR555(a [5nOr[SURR555(a [ 5nO8[ S[ [URR55535eURU5 g)zU Disables custom attention processors and sets the default attention implementation. c3F# UHoR[;v M g7fN)r=r.0procs r> :I2VGenXLUNet.set_default_attn_processor..siKh4~~!>>Kh!c3F# UHoR[;v M g7fr)r=rrs r>rrshJg$#==JgrzOCannot call `set_default_attn_processor` when attention processors are of type N) allrvaluesrrrnextiterr)r<rs r>set_default_attn_processor'I2VGenXLUNet.set_default_attn_processors i4K_K_KfKfKhi i i,.I h$J^J^JeJeJgh h h%IabfgklpmAmAmHmHmJhKcLbMN   *r@c[UR5H9upV[USU5 [USU5 [USU5 [USU5 M; g)aEnables the FreeU mechanism from https://huggingface.co/papers/2309.11497. The suffixes after the scaling factors represent the stage blocks where they are being applied. Please refer to the [official repository](https://github.com/ChenyangSi/FreeU) for combinations of values that are known to work well for different pipelines such as Stable Diffusion v1, v2, and Stable Diffusion XL. Args: s1 (`float`): Scaling factor for stage 1 to attenuate the contributions of the skip features. This is done to mitigate the "oversmoothing effect" in the enhanced denoising process. s2 (`float`): Scaling factor for stage 2 to attenuate the contributions of the skip features. This is done to mitigate the "oversmoothing effect" in the enhanced denoising process. b1 (`float`): Scaling factor for stage 1 to amplify the contributions of backbone features. b2 (`float`): Scaling factor for stage 2 to amplify the contributions of backbone features. s1s2b1b2N)rrsetattr)r<rrrrrupsample_blocks r> enable_freeuI2VGenXLUNet.enable_freeusJ$"+4>>!: A ND" - ND" - ND" - ND" - ";r@c1Skn[UR5H9up#UH.n[X45(d[X4S5cM"[ X4S5 M0 M; g)zDisables the FreeU mechanism.>rrrrN)rrrgetattrr)r< freeu_keysrrks r> disable_freeuI2VGenXLUNet.disable_freeusH- !*4>>!: A>--D1Q1]Nt4 ";r@cSUlURR5H3upS[URR 5;dM*[ S5e URUlUR5H)n[U[5(dMURSS9 M+ UR[55 g)u Enables fused QKV projections. For self-attention modules, all projection matrices (i.e., query, key, value) are fused. For cross-attention modules, key and value projection matrices are fused. This API is 🧪 experimental. NAddedzQ`fuse_qkv_projections()` is not supported for models having added KV projections.T)fuse) original_attn_processorsritemsrSr=rNrmodulesrrfuse_projectionsrr)r<_attn_processorrs r>fuse_qkv_projections!I2VGenXLUNet.fuse_qkv_projectionss)-%!%!5!5!;!;!= A#n66??@@ !tuu">)-(<(<%llnF&),,''T'2%  5 78r@cVURbURUR5 gg)umDisables the fused QKV projection if enabled. This API is 🧪 experimental. N)rr)r<s r>unfuse_qkv_projections#I2VGenXLUNet.unfuse_qkv_projectionss)  ( ( 4  # #D$A$A B 5r@sampletimestepfps image_latentsimage_embeddingsrD timestep_condcross_attention_kwargs return_dictc N^&URuppnSUR-m&SnSn[U&4SjURSS55(a[R S5 SnUn[ R "U5(dURRS:HnURRS :Hn[U[5(a/U(dU(a[ RO[ RnO.U(dU(a[ RO[ Rn[ R"U/UURS 9nO7[!UR5S :XaUSR#UR5nUR%URS 5nUR'U5nUR#UR(S 9nUR+UU5nUR%URS 5nUR-UR'U5R#UR(S 95nUU-nUR/U S URS U -S 9nUR1U S UR2R45n[ R6"UU/SS9nUSS2SS2SS2SS24nUR9S SSSS5R;URS URS-URSURSURS5nUR=U5nURunnnnUR9S SSS5R;UUU-U5n[ R6"UU/SS9nUR?U5nURASUR2RBUR2R45n[ R6"UU/SS9nUR/U S URS U -S 9nUR9S SSSS5R;URS URS-URSURSURS5nUREU5nUSSS24R;XXU5R9S SSSS5R;X-U-X5nURGU5nUR;XXU 5R9S SSSS5n[ R6"X/SS9nUR9S SSSS5R;URS U -S4URSS-5nURIU5nURKUU USS9S nU4n URLHCn![OU!S5(aU!RP(aU!"UUUU US9unn"O U!"UUU S9unn"U U"- n ME URRbURSUUUU US9n[UURV5Hun#n$U#[!URV5S- :Hn%U [!U$RX5*Sn"U S[!U$RX5*n U%(dU(aU SRSSn[OU$S5(aU$RP(aU$"UUU"UUU US9nMU$"UUU"UU S9nM UR[U5nUR]U5nUR_U5nUSSS24R;SU 4URSS-5R9S SSSS5nU (dU4$[aUS9$)a The [`I2VGenXLUNet`] forward method. Args: sample (`torch.Tensor`): The noisy input tensor with the following shape `(batch, num_frames, channel, height, width`. timestep (`torch.Tensor` or `float` or `int`): The number of timesteps to denoise an input. fps (`torch.Tensor`): Frames per second for the video being generated. Used as a "micro-condition". image_latents (`torch.Tensor`): Image encodings from the VAE. image_embeddings (`torch.Tensor`): Projection embeddings of the conditioning image computed with a vision encoder. encoder_hidden_states (`torch.Tensor`): The encoder hidden states with shape `(batch, sequence_length, feature_dim)`. cross_attention_kwargs (`dict`, *optional*): A kwargs dictionary that if specified is passed along to the `AttentionProcessor` as defined under `self.processor` in [diffusers.models.attention_processor](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py). return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~models.unets.unet_3d_condition.UNet3DConditionOutput`] instead of a plain tuple. Returns: [`~models.unets.unet_3d_condition.UNet3DConditionOutput`] or `tuple`: If `return_dict` is True, an [`~models.unets.unet_3d_condition.UNet3DConditionOutput`] is returned, otherwise a `tuple` is returned where the first element is the sample tensor. r FNc32># UH oT-S:gv M g7f)rN)rsdefault_overall_up_factors r>r'I2VGenXLUNet.forward../sM;La,,1;Lsz9Forward upsample size to force interpolation output size.Tmpsnpu)dtypedevicer)r)r# output_sizer)r#rrEry) num_framesr rhas_cross_attention)rAtembrDrr )rArr)rDrr )rArres_hidden_states_tuplerD upsample_sizerr )rArrrr)r)1shaperanyloggerinforU is_tensorrtyperfloatfloat32float64int32int64tensorrtoexpandrrrrrepeat_interleave new_zerosconfigrecatpermutereshaperrviewr^rrrrrrrrrrresnetsrrrr)'r<rrr r r rDr r r batch_sizechannelsrheightwidthforward_upsample_sizer timestepsis_mpsis_npurt_embfps_embemb context_embimage_latents_for_context_embdsimage_latents_context_embs _batch_size _channels_height_width image_embdown_block_res_samplesdownsample_block res_samplesrrrrs' @r>rKI2VGenXLUNet.forwardsL;A,,7 j% %&t':':$:!!& M6<<;LM M M KKS T$( ! y))]]''50F]]''50F)U++*0F (.& u{{ i[fmmTI  !Q &!$**6==9I$$V\\!_5 y) tzz*##E=9jj1&$$T^^C%8%;%;$**%;%MNgo##JA399Q'> @_@_` iii 8a@ !33JAS^SdSdefSgjtSt3u %--aAq!<DD    "]%8%8%; ;    "    "    "  22=A $' " WZXu E WQ1a # WZ(50* G  ;;MJ %--j%U]^ffghjkmnpqstu F2:1aA.66 Q*8TVX7Y\b\h\hijik\l7lmf%$$ !#9 %  #) $ 0 0 ')>??DTDhDh&6"(*5)+A '# '7VRUbl&m# "k 1 "!1 >> %^^&1%'= $F"+4>>!: A~#dnn"5"99N0#n6L6L2M1M1OPK%;CYCY?Z>Z%[ ""&; 6r : @ @ D ~'<==.BdBd'"(,7*5"/)+A("(,7"/) -";>##F+v&v&a(("j)9FLLr[r[as|4(-$&*- + /E !)+#'57@D-|ac]|a|a |a  S/ |ac3h|a""#s(O#|a$%|a&"#'|a(!)|a*"#uSz/2+|a,&eCsO&<=-|a|a|c+=&=!>0 AE2Dd3PbKbFc2c,d AF?(3-?S?Y]?< 7 + .2594 C&488<04;? S4 S4 eS01S4\\ S4 || S4 #5<<0 S4 ( 5S4  -S4!)c3h 8S4S4 $eELL&99 :S4S4r@r[)0typingrrrrrrUtorch.nnr7torch.utils.checkpointconfiguration_utilsr r loadersr utilsr activationsr attentionrrattention_processorrrrrrr embeddingsrrmodeling_utilsr!transformers.transformer_temporalrunet_3d_blocksrrrunet_3d_conditionr get_loggerrNr"rr!r[rr@r>res54 B2(.6'H 5   H %..bo 4:{,Go 4r@