cCiJSSKJrJr SSKrSSKJr SSKJr SSKJr SSK J r SSK J r SS K Jr S S KJrJrJrJr S S KJr \ R,"\5r"SS\SS9r"SS\R45r"SS\5r"SS\5r"SS\5r"SS\5r/SQr g))Optional TypedDictN)nn)ACT2FN)Cache)Unpack)logging)deprecate_kwarg)GraniteMoeDecoderLayerGraniteMoeForCausalLMGraniteMoeModelGraniteMoePreTrainedModel)GraniteMoeSharedConfigc\rSrSr%Sr\R \S'\R \S'\\S'\\S'\R\S'Sr g ) GraniteFlashAttentionKwargs&aR Keyword arguments for advanced Flash Attention, causal-conv1d, and mamba_ssm kernel usage. Use cases include padding-free training and fewer `torch.compile` graph breaks. Attributes: cu_seq_lens_q (`torch.LongTensor`) Gets cumulative sequence length for query state. cu_seq_lens_k (`torch.LongTensor`) Gets cumulative sequence length for key state. max_length_q (`int`): Maximum sequence length for query state. max_length_k (`int`): Maximum sequence length for key state. seq_idx (`torch.IntTensor): Index of each packed sequence. cu_seq_lens_q cu_seq_lens_k max_length_q max_length_kseq_idxN) __name__ __module__ __qualname____firstlineno____doc__torch LongTensor__annotations__int IntTensor__static_attributes__rw/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/granitemoeshared/modular_granitemoeshared.pyrr&s7"###### __r'rF)totalcn^\rSrSrSrS\4U4SjjrS\RS\R4Sjr Sr U=r $) GraniteMoeSharedMLP?zj MLP layer for shared experts Args: config: Configuration object with model hyperparameters. configcX>[TU]5 URUlURUl[ UR Ul[R"URURS-SS9Ul [R"URURSS9Ul g)Nr F)bias) super__init__ hidden_size input_sizeshared_intermediate_sizer hidden_act activationrLinear input_linear output_linearselfr- __class__s r(r1GraniteMoeSharedMLP.__init__Hs  ,,!:: !2!23IIdoot7G7G!7KRWXYYt'7'7uUr' hidden_statesreturncURU5nURSSS9nURUS5US-nURU5nU$)Nr )dimrr)r8chunkr6r9)r;r>chunked_hidden_statess r(forwardGraniteMoeSharedMLP.forwardQs^))-8 - 3 3A2 3 >(=a(@ADYZ[D\\ **=9 r')r6r2r8r3r9) rrrrr rr1r!TensorrEr& __classcell__r<s@r(r+r+?s7V5VU\\ellr'r+c^\rSrSrS\S\4U4Sjjr\"SSSS9SS \RS \ \RS \ \RS\ \ S \ \ S \ \ S\ \RS\ \ S\ \\R\R4S\\S\\R"\ \\R"\R"444Sjj5rSrU=r$)GraniteMoeSharedDecoderLayerYr- layer_idxct>[TU]X5 URS:XaSUlg[U5Ulg)Nr)r0r1r4r+ shared_mlpr;r-rMr<s r(r1%GraniteMoeSharedDecoderLayer.__init__Zs1 +"("A"AQ"F$L_`fLgr'past_key_valuepast_key_valuesz4.58)new_nameversionr>attention_mask position_idsoutput_attentions use_cachecache_positionoutput_router_logitsposition_embeddingskwargsr?c tUn URU5nUR"SUUUUUUUU S.U D6upXUR--nUn URU5nUR U5upUR cU nOXR U5-nA XUR--nU4nU(aX4- nU(aX4- nU$)aq Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch_size, sequence_length)` if flash attention is used or `(batch_size, 1, query_sequence_length, key_sequence_length)` if default attention is used. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_values (`Cache`, *optional*): cached past key and value projection states cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence output_router_logits (`bool`, *optional*): Whether or not to return the logits of all the routers. They are useful for computing the router loss, and should not be returned during inference. position_embeddings (`tuple[torch.FloatTensor, torch.FloatTensor]`, *optional*): Tuple containing the cosine and sine positional embeddings of shape `(batch_size, seq_len, head_dim)`, with `head_dim` being the embedding dimension of each attention head. kwargs (`dict`, *optional*): Arbitrary kwargs. Can be used to provide `GraniteFlashAttentionKwargs` for padding-free training and/or improve torch.compile performance. )r>rVrWrSrXrYrZr\r)input_layernorm self_attnresidual_multiplierpost_attention_layernormblock_sparse_moerO)r;r>rVrWrSrXrYrZr[r\r]residualself_attn_weightsmoe_hidden_states router_logitsoutputss r(rE$GraniteMoeSharedDecoderLayer.forward^sN!,,]; ,0>> , ')%+/) 3 ,  , ( !43K3K#KK !55mD +/+@+@+O( ?? "-M- 0NNM  43K3K#KK  "  + +G  ' 'Gr')rO)NNNFFNFN)rrrrrr$r1r r!rGrr"rbooltupler r FloatTensorrEr&rHrIs@r(rKrKYsHh5h#h%0A6R2637+/,1$)59/4KOO||O!.Ou//0 O "% O $D> OD>O!!1!12O'tnO&eELL%,,,F&GHO45O u  (51B1BEDUDU1U+V"WW XOSOr'rKc&\rSrSr%\\S'S/rSrg)GraniteMoeSharedPreTrainedModelr-rKrN)rrrrrr#_no_split_modulesr&rr'r(rnrns ""78r'rnc0^\rSrSrS\4U4SjjrSrU=r$)GraniteMoeSharedModelr-c >[TU]U5 [R"[ UR 5Vs/sHn[ X5PM sn5Ulgs snfN)r0r1r ModuleListrangenum_hidden_layersrKlayersrPs r(r1GraniteMoeSharedModel.__init__sI  mmNSTZTlTlNm nNm )& [TU]U5 [U5UlUR 5 gru)r0r1rrmodel post_initr:s r(r1$GraniteMoeSharedForCausalLM.__init__s&  *62  r')r) rrrr_tied_weights_keysrr1r&rHrIs@r(r|r|s*+5r'r|)r|rrrn)!typingrrr!r activationsr cache_utilsrprocessing_utilsr utilsr utils.deprecationr granitemoe.modeling_granitemoer rrrconfiguration_granitemoesharedr get_loggerrloggerrModuler+rKrnrrr|__all__rr'r(rs ' ! &0 C   H %)52"))4U#9Up9&?9  O "7 fr'