+h=SSKJrJr SSKrSSKJr SSKJrJr SSK J r SSK J r SSK JrJrJr SS KJrJr SS KJr SS KJr SS KJrJr \ R6"\5r"S S\R<5r"SS\\5r g))DictUnionN) ConfigMixinregister_to_config)logging) FeedForward) AttentionAttentionProcessorCogVideoXAttnProcessor2_0)&CogView3CombinedTimestepSizeEmbeddingsCogView3PlusPatchEmbed)Transformer2DModelOutput) ModelMixin)AdaLayerNormContinuous%CogView3PlusAdaLayerNormZeroTextImagec ^\rSrSrSrSS\S\S\S\4U4SjjjrS\RS \RS \RS \R4S jr S r U=r $)CogView3PlusTransformerBlock"a Transformer block used in [CogView](https://github.com/THUDM/CogView3) model. Args: dim (`int`): The number of channels in the input and output. num_attention_heads (`int`): The number of heads to use for multi-head attention. attention_head_dim (`int`): The number of channels in each head. time_embed_dim (`int`): The number of channels in timestep embedding. dimnum_attention_headsattention_head_dimtime_embed_dimc >[TU]5 [XAS9Ul[ UUUUSSSS[ 5S9 Ul[R"USSS9Ul [R"USSS9Ul [XS S 9Ul g) N) embedding_dimrT layer_normFư>) query_dimheadsdim_headout_dimbiasqk_normelementwise_affineeps processorgh㈵>)r%r&zgelu-approximate)rdim_out activation_fn) super__init__rnorm1r r attn1nn LayerNormnorm2 norm2_contextr ff)selfrrrr __class__s p/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/models/transformers/transformer_cogview3plus.pyr+%CogView3PlusTransformerBlock.__init__1s :a %' $/1   \\#%TJ \\#%TR#BTU hidden_statesencoder_hidden_statesembreturnc URS5nURXU5u nnnnn n n n n nURXZS9unnXRS5U--nX+RS5U--nUR U5nUSUSS2S4--USS2S4-nUR U5n U SU SS2S4--U SS2S4-n [ R"X/SS9nURU5nXRS5USS2US24--nX.RS5USS2SU24--nUR[ R:XaURSS5nUR[ R:XaURSS5nX4$)N)r8r9)ri i) sizer,r- unsqueezer0r1torchcatr2dtypefloat16clip)r3r8r9r:text_seq_lengthnorm_hidden_statesgate_msa shift_mlp scale_mlpgate_mlpnorm_encoder_hidden_states c_gate_msa c_shift_mlp c_scale_mlp c_gate_mlpattn_hidden_statesattn_encoder_hidden_states ff_outputs r5forward$CogView3PlusTransformerBlock.forwardMs 044Q7 JJ}S A      &    :>,:D: 66&(:(:1(=@R(RR 58L8LQ8ORl8l l"ZZ 6/1yD7I3IJYWXZ^W^M__%)%7%78M%N"%?1{STVZSZG[C[%\_jklnrkr_s%s"#YY(B'W]^_GG./ %(:(:1(= !_M]J]@^(^^ 58L8LQ8OR[\]_o`o_o\oRp8p p   %-- /)..vu=M & &%-- 7$9$>$>vu$M !33r7)r-r2r,r0r1)i @() __name__ __module__ __qualname____firstlineno____doc__intr+r@TensorrS__static_attributes__ __classcell__r4s@r5rr"s  #%"$! V V!V V  VV804||04 %||04\\ 04  0404r7rc^\rSrSrSrSrSS/rSS/r\S!S\ S \ S \ S \ S \ S \ S\ S\ S\ S\ S\ 4U4Sjjj5r \ S\ \ \44Sj5rS\\\ \ \444SjrS"S\R&S\R&S\R(S\R&S\R&S\R&S\S\\R&\44SjjrS rU=r$)#CogView3PlusTransformer2DModela! The Transformer model introduced in [CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusion](https://huggingface.co/papers/2403.05121). Args: patch_size (`int`, defaults to `2`): The size of the patches to use in the patch embedding layer. in_channels (`int`, defaults to `16`): The number of channels in the input. num_layers (`int`, defaults to `30`): The number of layers of Transformer blocks to use. attention_head_dim (`int`, defaults to `40`): The number of channels in each head. num_attention_heads (`int`, defaults to `64`): The number of heads to use for multi-head attention. out_channels (`int`, defaults to `16`): The number of channels in the output. text_embed_dim (`int`, defaults to `4096`): Input dimension of text embeddings from the text encoder. time_embed_dim (`int`, defaults to `512`): Output dimension of timestep embeddings. condition_dim (`int`, defaults to `256`): The embedding dimension of the input SDXL-style resolution conditions (original_size, target_size, crop_coords). pos_embed_max_size (`int`, defaults to `128`): The maximum resolution of the positional embeddings, from which slices of shape `H x W` are taken and added to input patched latents, where `H` and `W` are the latent height and width respectively. A value of 128 means that the maximum supported height and width for image generation is `128 * vae_scale_factor * patch_size => 128 * 8 * 2 => 2048`. sample_size (`int`, defaults to `128`): The base resolution of input latents. If height/width is not provided during generation, this value is used to determine the resolution as `sample_size * vae_scale_factor => 128 * 8 => 1024` T patch_embednormrr patch_size in_channels num_layersrr out_channelstext_embed_dimr condition_dimpos_embed_max_size sample_sizec >[T U]5 X`lXT-UlSU -Ul[ UURUUU S9Ul[UU URURS9Ul[R"[U5V s/sHn [URUUUS9PM sn 5Ul [URUSSS9Ul[R "URX-UR-SS 9UlSUlgs sn f) N)rh hidden_sizergtext_hidden_sizerm)rrlpooled_projection_dim timesteps_dim)rrrrFr)rconditioning_embedding_dimr%r&T)r#)r*r+rj inner_dimrsrrertime_condition_embedr. ModuleListrangertransformer_blocksrnorm_outLinearproj_outgradient_checkpointing)r3rgrhrirrrjrkrrlrmrn_r4s r5r+'CogView3PlusTransformer2DModel.__init__s (,A&+]%:"1#!+1  %K('"&"<"<.. % !#%--z* +A -(;'9#1  +  # /..'5$    $..*2IDL]L]2]dhi &+#' s D r;c^0nS[S[RRS[[[ 44U4SjjmUR 5Hup#T"X#U5 M U$)z Returns: `dict` of attention processors: A dictionary containing all attention processors used in the model with indexed by its weight name. namemodule processorsc>[US5(aUR5X S3'UR5Hup4T"USU3XB5 M U$)N get_processor .processor.)hasattrrnamed_children)rrrsub_namechildfn_recursive_add_processorss r5rSCogView3PlusTransformer2DModel.attn_processors..fn_recursive_add_processorssZv//282F2F2H V:./#)#8#8#:+tfAhZ,@%T$; r7)strr@r.Modulerr r)r3rrrrs @r5attn_processors.CogView3PlusTransformer2DModel.attn_processorssb  c 588?? X\]`bt]tXu !//1LD 'j A2r7r'c d^[URR55n[U[5(a-[U5U:wa[ S[U5SUSUS35eS[ S[RR4U4SjjmUR5Hup4T"X4U5 M g) a Sets the attention processor to use to compute attention. Parameters: processor (`dict` of `AttentionProcessor` or only `AttentionProcessor`): The instantiated processor class or a dictionary of processor classes that will be set as the processor for **all** `Attention` layers. If `processor` is a dict, the key needs to define the path to the corresponding cross attention processor. This is strongly recommended when setting trainable attention processors. z>A dict of processors was passed, but the number of processors z0 does not match the number of attention layers: z. Please make sure to pass z processor classes.rrc >[US5(aJ[U[5(dURU5 O#URUR US355 UR 5Hup4T"USU3XB5 M g)N set_processorrr)r isinstancedictrpopr)rrr'rrfn_recursive_attn_processors r5rVCogView3PlusTransformer2DModel.set_attn_processor..fn_recursive_attn_processorssv//!)T22((3(($z7J)KL#)#8#8#:+tfAhZ,@%S$;r7N) lenrkeysrr ValueErrorrr@r.rr)r3r'countrrrs @r5set_attn_processor1CogView3PlusTransformer2DModel.set_attn_processorsD((--/0 i & &3y>U+BPQTU^Q_P`a005w6QRWQXXkm   Tc T588?? T!//1LD 'i @2r7r8r9timestep original_size target_size crop_coords return_dictc$URSSupURSn URX5nURX4XVUR5n USS2SU 24nUSS2U S24n[ UR 5HRup[ R"5(a)UR(aURU UUU 5upMHU "UUU S9upMT URX5nURU5nURRnX-nX-n URURSXURX4S9n[ R "SU5nURURSURX-X-4S9nU(dU4$[#US9$) a The [`CogView3PlusTransformer2DModel`] forward method. Args: hidden_states (`torch.Tensor`): Input `hidden_states` of shape `(batch size, channel, height, width)`. encoder_hidden_states (`torch.Tensor`): Conditional embeddings (embeddings computed from the input conditions such as prompts) of shape `(batch_size, sequence_len, text_embed_dim)` timestep (`torch.LongTensor`): Used to indicate denoising step. original_size (`torch.Tensor`): CogView3 uses SDXL-like micro-conditioning for original image size as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). target_size (`torch.Tensor`): CogView3 uses SDXL-like micro-conditioning for target image size as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). crop_coords (`torch.Tensor`): CogView3 uses SDXL-like micro-conditioning for crop coordinates as explained in section 2.2 of [https://huggingface.co/papers/2307.01952](https://huggingface.co/papers/2307.01952). return_dict (`bool`, *optional*, defaults to `True`): Whether or not to return a [`~models.transformer_2d.Transformer2DModelOutput`] instead of a plain tuple. Returns: `torch.Tensor` or [`~models.transformer_2d.Transformer2DModelOutput`]: The denoised latents using provided inputs as conditioning. Nr=)r8r9r:r)shapeznhwcpq->nchpwq)sample)rrerwrB enumeraterzr@is_grad_enabledr~_gradient_checkpointing_funcr{r}configrgreshaperjeinsumr)r3r8r9rrrrrheightwidthrEr: index_blockblockrgoutputs r5rS&CogView3PlusTransformer2DModel.forwardsL&++BC0 /55a8((  ''[n[no -a1A/1A.A B%a)9&9: "+D,C,C"D K$$&&4+F+F7;7X7X!) 84 48="/*?84 4#E m9  m4 [[++ %#%-- &&q)6$:K:KZd.  %5}E && &&q)4+<+QSXSef' 9 'v66r7) r~rvr{rjrersr}rwrz) r rVrUrirWrdrd)T)rXrYrZr[r\ _supports_gradient_checkpointing _skip_layerwise_casting_patterns_no_split_modulesrr]r+propertyrrr rrrr@r^ LongTensorboolrrSr_r`ras@r5rcrcs D(,$(5v'>$79QR"$#%"! "%9,9,9, 9,  9, ! 9,9,9,9,9, 9,9,9,vc+=&=!>0 AE2Dd3PbKbFc2c,d AT!S7||S7 %||S7"" S7 || S7 \\ S7\\S7S7 u||55 6S7S7r7rc)!typingrrr@torch.nnr.configuration_utilsrrutilsr attentionr attention_processorr r r embeddingsrrmodeling_outputsrmodeling_utilsr normalizationrr get_loggerrXloggerrrrcr7r5rs`  B#ZZW7'Y   H %[4299[4|r7Zr7r7