+hSSKrSSKJrJr SSKrSSKJr SSKJrJr SSK J r SSK J r SSK Jr SS KJrJrJr SS KJr SS KJr SS KJr \R2"\5rS rSSjr"SS\R<5r"SS\R<5r "SS\R<5r!"SS\R<5r""SS\ \5r#"SS\ \5r$g)N)OptionalUnion)nn) ConfigMixinregister_to_config) ModelMixin) FeedForward) Attention)TimestepEmbedding Timestepsget_2d_sincos_pos_embed)Transformer2DModelOutput) AdaLayerNorm)loggingcSnXSU-- :d XSU--:a[RS5 [R"5 U"X1- U- 5nU"XA- U- 5nUR SU-S- SU-S- 5 UR 5 UR U[R"S5-5 URU5 URX4S9 UsSSS5 $!,(df  g=f)NchS[R"U[R"S5- 5-S- $)N?@)matherfsqrt)xs g/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/pipelines/unidiffuser/modeling_uvit.pynorm_cdf(_no_grad_trunc_normal_..norm_cdfs(dhhq499S>122c99zjmean is more than 2 std from [a, b] in nn.init.trunc_normal_. The distribution of values may be incorrect.r)minmax) loggerwarningtorchno_graduniform_erfinv_mul_rradd_clamp_)tensormeanstdabrlus r_no_grad_trunc_normal_r2s: 1s7{1s7{ 2 ;  ah#% & ah#% & A 1q519-   C$))C.() D  ! #+ s BC  C.c[XX#U5$)aFills the input Tensor with values drawn from a truncated normal distribution. The values are effectively drawn from the normal distribution :math:`\mathcal{N}(\text{mean}, \text{std}^2)` with values outside :math:`[a, b]` redrawn until they are within the bounds. The method used for generating the random values works best when :math:`a \leq \text{mean} \leq b`. Args: tensor: an n-dimensional `torch.Tensor` mean: the mean of the normal distribution std: the standard deviation of the normal distribution a: the minimum cutoff value b: the maximum cutoff value Examples: >>> w = torch.empty(3, 5) >>> nn.init.trunc_normal_(w) )r2)r+r,r-r.r/s r trunc_normal_r49s "& ::rcH^\rSrSrSrSU4SjjrSrSrU=r$) PatchEmbedLz2D Image to Patch Embeddingc >[T U]5 X-X#--n XplX`l[R "XEX34X8S9UlU(a[R"USSS9UlOSUlXl UR(aF[U[U S-5SS9n URSU R5RS 5SS 9 gg) N) kernel_sizestridebiasFgư>)elementwise_affineeps?pt) output_type pos_embedr) persistent)super__init__flatten layer_normrConv2dproj LayerNormnorm use_pos_embedrintregister_bufferfloat unsqueeze) selfheightwidth patch_size in_channels embed_dimrFrEr;rK num_patchesrA __class__s rrDPatchEmbed.__init__Os +0CD  $II 0HQ[    Y5dSDIDI*   / 3{C?O;P^bcI  ioo.?.I.I!.LY^ _ rcURU5nUR(a!URS5RSS5nUR(aUR U5nUR (aXR -$U$)Nrr)rHrE transposerFrJrKrA)rPlatents rforwardPatchEmbed.forwardnsb6" <<^^A&00A6F ??YYv&F   NN* *Mr)rErFrJrHrK) r^rFTTT) __name__ __module__ __qualname____firstlineno____doc__rDr\__static_attributes__ __classcell__rWs@rr6r6Ls3% `>  rr6c6^\rSrSrS\4U4SjjrSrSrU=r$) SkipBlockzdimc>[TU]5 [R"SU-U5Ul[R "U5Ulg)Nr)rCrDrLinear skip_linearrIrJ)rPrlrWs rrDSkipBlock.__init__{s7 99QWc2LL% rcrUR[R"X/SS95nURU5nU$)Nrl)ror$catrJ)rPrskips rr\SkipBlock.forwards1   UYYyb9 : IIaLr)rJro) rarbrcrdrLrDr\rfrgrhs@rrjrjzs&C&rrjc^\rSrSrSrSS\S\S\S\\S\S\\S \S \S \S \S \S\S\S\4U4Sjjjr SSjr Sr U=r $)UTransformerBlocka A modification of BasicTransformerBlock which supports pre-LayerNorm and post-LayerNorm configurations. Parameters: dim (`int`): The number of channels in the input and output. num_attention_heads (`int`): The number of heads to use for multi-head attention. attention_head_dim (`int`): The number of channels in each head. dropout (`float`, *optional*, defaults to 0.0): The dropout probability to use. cross_attention_dim (`int`, *optional*): The size of the encoder_hidden_states vector for cross attention. activation_fn (`str`, *optional*, defaults to `"geglu"`): Activation function to be used in feed-forward. num_embeds_ada_norm (:obj: `int`, *optional*): The number of diffusion steps used during training. See `Transformer2DModel`. attention_bias (:obj: `bool`, *optional*, defaults to `False`): Configure if the attentions should contain a bias parameter. only_cross_attention (`bool`, *optional*): Whether to use only cross-attention layers. In this case two cross attention layers are used. double_self_attention (`bool`, *optional*): Whether to use two self-attention layers. In this case no cross attention layers are used. upcast_attention (`bool`, *optional*): Whether to upcast the query and key to float32 when performing the attention calculation. norm_elementwise_affine (`bool`, *optional*): Whether to use learnable per-element affine parameters during layer normalization. norm_type (`str`, defaults to `"layer_norm"`): The layer norm implementation to use. pre_layer_norm (`bool`, *optional*): Whether to perform layer normalization before the attention and feedforward operations ("pre-LayerNorm"), as opposed to after ("post-LayerNorm"). Note that `BasicTransformerBlock` uses pre-LayerNorm, e.g. `pre_layer_norm = True`. final_dropout (`bool`, *optional*): Whether to use a final Dropout layer after the feedforward network. rlnum_attention_headsattention_head_dimcross_attention_dim activation_fnnum_embeds_ada_normattention_biasonly_cross_attentiondouble_self_attentionupcast_attentionnorm_elementwise_affine norm_typepre_layer_norm final_dropoutc >[TU]5 XlUSL=(a U S:HUlXlU S;aUc[ SU SU S35e[ UUUUUU (aUOSU S9UlUcU (a[ UU (dUOSUUUUU S9UlOSUlUR(a[X5Ul O[R"XS9Ul UcU (a6UR(a [X5O[R"XS9Ul OSUl [R"XS9Ul[XXoS 9Ulg Nada_norm)r ada_norm_zeroz`norm_type` is set to zw, but `num_embeds_ada_norm` is not defined. Please make sure to define `num_embeds_ada_norm` if setting `norm_type` to .) query_dimheadsdim_headdropoutr;r|r)rr|rrrr;r)r<)rr}rrCrDruse_ada_layer_normr ValueErrorr attn1attn2rnorm1rrInorm2norm3r ffrPrlrzr{rr|r}r~rrrrrrrrrWs rrDUTransformerBlock.__init__N$ $8!#6d#B"_ U_H_, 5 5:M:U( 4KKT+UVX  %'7K 3QU-    *.C"?T$7Z^)+#!1DJDJ  " "%c?DJcVDJ  *.C **S6\\#R J DJ\\#R c-mrcUR(a5UR(aURX5nOURU5nOUnUbUO0nUR"U4UR(aUOSUS.UD6n UR(d4UR(aURX5n OURU 5n X-nUR bUR(a4UR(aUR X5OUR U5nOUnUR "U4UUS.UD6n UR(d3UR(aUR X5OUR U 5n X-nUR(aURU5nOUnURU5n UR(dURU 5n X-nU$N)encoder_hidden_statesattention_mask rrrrrrrrr) rP hidden_statesrrencoder_attention_masktimestepcross_attention_kwargs class_labelsnorm_hidden_states attn_output ff_outputs rr\UTransformerBlock.forwards   &&%)ZZ %H"%)ZZ %>"!. [TU]5 XlUSL=(a U S:HUlXlU S;aUc[ SU SU S35e[ UUUUUU (aUOSU S9UlUcU (a[ UU (dUOSUUUUU S9UlOSUlUR(a[X5Ul O[R"XS9Ul UcU (a6UR(a [X5O[R"XS9Ul OSUl [R"XS9Ul[XXoS 9Ulgrrrs rrDUniDiffuserBlock.__init__rrrcUR(a4UR(aURX5nOURU5nUbUO0nUR"U4UR(aUOSUS.UD6nX-nUR(d4UR(aURX5nOURU5nUR bUR(a3UR(aUR X5OUR U5nUR "U4UUS.UD6nX-nUR(d3UR(aUR X5OUR U5nUR(aURU5nURU5n X-nUR(dURU5nU$rr) rPrrrrrrrrrs rr\UniDiffuserBlock.forwards   && $ = C $ = 9