cCiSrSSKJrJrJr SSKrSSKJr SSKJr SSK J r J r SSK J r SS KJr SS KJr SS KJrJrJr SS KJrJr SS KJrJr SSKJrJr SSKJ r SSK!J"r"J#r#J$r$J%r% SSK&J'r' SSK(J)r) \$"5(a SSK*J+r+ SSK,J-r- \%R\"\/5r0"SS\Rb5r2Sr3S3Sjr4"SS\Rb5r5S4S\RbS\RlS\RlS\RlS \\RlS!\7S"\74S#jjr8"S$S%\Rb5r9"S&S'\5r:\""S(S)\55r;\""S*S+\;55r<"S,S-\;\ 5r="S.S/\\;5r>"S0S1\\;5r?/S2Qr@g)5zPyTorch Persimmon model.)CallableOptionalUnionN)nn)ACT2FN)Cache DynamicCache)GenerationMixin)AttentionMaskConverter)FlashAttentionKwargs) GenericForSequenceClassificationGenericForTokenClassificationGradientCheckpointingLayer)BaseModelOutputWithPastCausalLMOutputWithPast)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack)auto_docstringcan_return_tupleis_torch_flex_attn_availablelogging)deprecate_kwarg)PersimmonConfig) BlockMask)make_flex_block_causal_maskc^\rSrSr%\R \S'SS\4U4Sjjjr\R"5\ S55r Sr U=r $)PersimmonRotaryEmbedding;inv_freqconfigc>[TU]5 [US5(aZ[UR[ 5(a;URR SURR S55UlOSUlURUl URUl Xl [URUl URURU5uo0lURSUSS9 UR Ulg)N rope_scaling rope_typetypedefaultr$F) persistent)super__init__hasattr isinstancer'dictgetr(max_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenr%r rope_init_fnattention_scalingregister_bufferr$original_inv_freq)selfr%devicer$ __class__s j/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/persimmon/modeling_persimmon.pyr-!PersimmonRotaryEmbedding.__init__>s  6> * *z&:M:Mt/T/T#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E'E!((--[`J`ahhmmfk ^^ U C&,,.1F1L1L1NNYYZ[]^_E))UN3C'')d444C'')d444C D vvAGGv$cff177f&;;; D Cs $BF  F.)r6r%r3r8r4r5r(N)__name__ __module__ __qualname____firstlineno__rNTensor__annotations__rr-no_gradrrZ__static_attributes__ __classcell__r;s@r<r"r";s@ll///" ]]_ < r"cUSSURSS-24nUSURSS-S24n[R"U*U4SS9$)z*Rotates half the hidden dims of the input..Nr@rErF)rKrNrQ)rTx1x2s r< rotate_halfrj`sZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r>cURU5nURU5nX-[U5U--nX-[U5U--nXg4$)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezerj)qkrRrSrU unsqueeze_dimq_embedk_embeds r<apply_rotary_pos_embrrhsS( -- &C -- &Cw;q>C/0Gw;q>C/0G  r>c.^\rSrSrU4SjrSrSrU=r$) PersimmonMLPc>[TU]5 [R"URUR 5Ul[R"UR UR5Ul[URUl gr\) r,r-rLinear hidden_sizeintermediate_size dense_h_to_4h dense_4h_to_hr hidden_actactr9r%r;s r<r-PersimmonMLP.__init__s^ YYv'9'96;S;STYYv'?'?ASAST&++,r>clURU5nURU5nURU5nU$r\)rzr}r{)r9 hidden_statess r<rZPersimmonMLP.forwards6**=9 / **=9 r>)r}r{rz)r]r^r_r`r-rZrdrerfs@r<rtrts- r>rtmodulequerykeyvalueattention_maskscalingdropoutc [R"XRSS55U-nUb"USS2SS2SS2SURS24n X-n[R R US[RS9RUR5n[R RXURS9n[R"X5n U RSS5R5n X4$)NrErr@)rGrH)ptrainingr) rNmatmulrPrKr functionalsoftmaxfloat32rLrHrr contiguous) rrrrrrrkwargs attn_weights causal_mask attn_outputs r<eager_attention_forwardrs<<}}Q':;gEL!$Q1o " o%=> #1 ==((2U]](SVVW\WbWbcL==((6??([L,,|3K''1-88:K  $$r>c ^\rSrSrSrSS\S\\4U4SjjjrS\ RS\ \ R\ R\ R44Sjr \ "S S S S 9SS \ RS\\ RS\\ RS \\S\S\S\\ RS\\ \ R\ R4S\\S\ \ R\\ R\\ \ R44Sjj5rSrU=r$)PersimmonAttentionz=Multi-headed attention from 'Attention Is All You Need' paperr% layer_idxc>[TU]5 XlX lUc-[R SUR RS35 URUlURUl URUR-Ul URUl [URUR-5UlSUlURUR-UR:wa&[#SURSURS35e[$R&"URSUR-SS9Ul[$R&"URUR-URSS9UlUR,UlURS -UlUR,(ax[$R0"URUR-UR2SS 9Ul[$R0"URUR-UR2SS 9Ul[$R8"UR:5Ul[=URS 9Ulg) NzInstantiating z without passing a `layer_idx` is not recommended and will lead to errors during the forward call if caching is used. Please make sure to provide a `layer_idx` when creating this class.Tz?hidden_size must be divisible by num_heads (got `hidden_size`: z and `num_heads`: z).rbiasg)epselementwise_affiner%) r,r-r%rlogger warning_oncer;r]rxnum_attention_heads num_headshead_dim rope_thetaintpartial_rotary_factor rotary_ndims is_causal ValueErrorrrwquery_key_valuedense qk_layernormr LayerNormlayer_norm_eps q_layernorm k_layernormDropoutattention_dropoutr" rotary_embr9r%rr;s r<r-PersimmonAttention.__init__s  "      !8!8 9:,,  "--33((DNN:  ++ 0L0L LM MMDNN *t/?/? ?QRVRbRbQc$T^^$4B8  "yy)9)91t?O?O;OVZ[YYt~~ =t?O?OVZ[ "//}}d*   !||""dnn4&:O:Odh D  "||""dnn4&:O:Odh D "$F,D,D!E2$++Fr> fused_qkvreturncURup#nURX#URSUR5nUSSSS24USSSS24USSSS244$)a Split the last dimension into (num_heads, head_dim) without making any copies, results share same memory storage as `fused_qkv` Args: fused_qkv (`torch.tensor`): [batch_size, seq_length, num_heads * 3 * head_dim] Returns: query: [batch_size, seq_length, num_heads, head_dim] key: [batch_size, seq_length, num_heads, head_dim] value: [batch_size, seq_length, num_heads, head_dim] r.rNrrE)rKviewrr)r9r batch_size seq_lengththree_times_hidden_sizes r< _split_headsPersimmonAttention._split_headss^;D//7 7NN:4>>1dmm\ a#YsAqy%99S!QY;OOOr>past_key_valuepast_key_values4.58new_nameversionrrrUoutput_attentions use_cachecache_positionposition_embeddingsrc  0UR5upn URU5n URU 5upnUR(a"UR U5nUR U5nUR SS5nUR SS5nUR SS5nUunnUSSUR24USURS24nnUSSUR24USURS24nn[UUUU5unn[R"UU4SS9n[R"UU4SS9nUb2UUURUS.nURUUURU5unn[nURRS:wa[ URRnU"UUUUU4UR"(dSOURR$UR&S .U D6unnUR)XS5nUR+U5nU(dSnUU4$) NrrE.r@rF)rSrRpartial_rotation_sizereager)rr)sizerrrrrrPrrrrNrQupdaterrr%_attn_implementationrrrrreshaper)r9rrrUrrrrrrbszq_len_r query_states key_states value_statesrRrS query_rot query_passkey_rotkey_pass cache_kwargsattention_interfacerrs r<rZPersimmonAttention.forwardsO&**, A((7 483D3DY3O0<   ++L9L))*5J$--a3 #--a3 ))!Q/ &S 1 1 111 2 d//11 2 s/d//// 0 sD--// 0 2)Wc3O 7yy)Z!8bA YY2;  &)-):):"0 L (7'='=j,X\XfXfht'u $J (? ;; + +w 6"9$++:Z:Z"[ $7      % $}}C$++2O2OLL %  % ! \"))#b9 jj-  LL((r>)rr%rrrxrrrrrrrrrrrr\NNNFFNN)r]r^r_r`__doc__rrrr-rNratuplerr LongTensorr boolrr rZrdrerfs@r<rrslG$G$G8C=$G$GLPellPuU\\5<Q5RR SN)SN)r>rc^\rSrSrS\S\4U4Sjjr\"SSSS9SS \RS \ \RS \ \RS\ \ S \ \ S \ \ S\ \RS\ \\R\R4S\\S\\R"\ \\R"\R"444Sjj5rSrU=r$)PersimmonDecoderLayeri4r%rc>[TU]5 URUl[XS9Ul[ U5Ul[R"URURS9Ul [R"URURS9Ul [R"UR5Ulg)N)r%rr)r,r-rxr self_attnrtmlprrrinput_layernormpost_attention_layernormrhidden_dropoutrrs r<r-PersimmonDecoderLayer.__init__5s !--+6O'!||F,>,>FDYDYZ(* V5G5GVMbMb(c%zz&"7"78 r>rrrrrrrUrrrrrrc Un URU5nUR"SUUUUUUUUS.U D6upX-nUn URU5nURU5nUR U5nX-nU4n U(aX4- n U $)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. position_ids (`torch.LongTensor` of shape `({0})`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. [What are position IDs?](../glossary#position-ids) past_key_values (`Cache`, *optional*): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence position_embeddings (`tuple[torch.FloatTensor, torch.FloatTensor]`, *optional*): Tuple containing the cosine and sine positional embeddings of shape `(batch_size, seq_len, head_dim)`, with `head_dim` being the embedding dimension of each attention head. )rrrUrrrrr)rrrrr) r9rrrUrrrrrrresidualself_attn_weightsoutputss r<rZPersimmonDecoderLayer.forward>sJ!,,]; ,0>> , ')%+/) 3 ,  , ( !0 !55mD /  ]3 %0  "  + +Gr>)rrxrrrrr)r]r^r_r`rrr-rrNrarrr rrrr FloatTensorrZrdrerfs@r<rr4s29939%0A6R2637+/,1$)59KOC||C!.Cu//0 C "% C $D> CD>C!!1!12C&eELL%,,,F&GHC-.C u  (51B1BEDUDU1U+V"WW XCSCr>rcH\rSrSr%\\S'SrSrS/rSr Sr Sr Sr Sr SrSrg ) PersimmonPreTrainedModelir%modelTrrcURRn[U[R5(aWUR R RSUS9 URb%URR R5 gg[U[R5(adUR R RSUS9 URb2UR R URR5 gg[U[R5(aJUR R RS5 URR R5 gg)Nr)meanstdg?)r%initializer_ranger/rrwweightdatanormal_rzero_ Embedding padding_idxrfill_)r9rrs r< _init_weights&PersimmonPreTrainedModel._init_weightsskk++ fbii ( ( MM   & &CS & 9{{&   &&('  - - MM   & &CS & 9!!- ""6#5#56<<>.  - - MM   $ $S ) KK   " " $.r>rN)r]r^r_r`rrbbase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_can_compile_fullgraph_supports_sdpa_supports_flash_attn_supports_attention_backendrrdrr>r<rrs? &*#01"3!N"& %r>rc^\rSrSrSrS\4U4Sjjr\\SS\ \ RS\ \ RS\ \ RS\ \ S \ \ RS \ \S \ \S \ \S \ \ RS\\S\4Sjj55rSS\\ RS4S\ RS \ RS\ S \4 Sjjr\S\ RS\S\S\ R0S \ RS\4 Sj5rSrU=r$)PersimmonModeliz Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`PersimmonDecoderLayer`] Args: config: PersimmonConfig r%c >[TU]U5 URUlURUl[ R "URURUR5Ul[ R"[UR5Vs/sHn[X5PM sn5Ul [ R"URURS9Ul[#US9UlSUlUR)5 gs snf)NrrF)r,r- pad_token_idr vocab_sizerrrx embed_tokens ModuleListrangenum_hidden_layersrlayersrrfinal_layernormr"rgradient_checkpointing post_initrs r<r-PersimmonModel.__init__s  !.. ++LL):):F,>FDYDYZ2&A&+#  hsD  input_idsrrUr inputs_embedsrroutput_hidden_statesrrrc UbUOURRnUbUOURRnUbUOURRnUSLUSL- (a [ S5eUR (a/UR (aU(a[RS5 SnU(aUc[URS9nUcURU5nU cDUbUR5OSn [R"XURS-URS9n UcU R!S5nUR#X%XU5n Un UR%X5nU(aSOSnU(aSOSnUR&H7nU(aX4- nU"U 4U UUUUU US .U D6nUSn U(dM.UUS4- nM9 UR)U 5n U(aX4- n[+U UUUS 9$) Nz:You must specify exactly one of input_ids or inputs_embedszZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...Frrrr:r)rrUrrrrr)last_hidden_staterr attentions)r%rrrrrrrrr rget_seq_lengthrNarangerKr:rl_update_causal_maskrrrr)r9rrrUrrrrrrrpast_seen_tokensrrrall_hidden_statesall_self_attns decoder_layer layer_outputss r<rZPersimmonModel.forwards2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKKO   --i8M  !CRC^==?de "\\ ]5H5H5K"KTaThThN  )33A6L.. >L] & #oomJ#7BD0d![[M#!%55!) *) /"3#-$7  M*!,M  =#3"55')*,,];  !1 1 &+++%   r>r input_tensorc URRS:XaUbUS:HR5(aU$gURRS:Xa,[U[R 5(a [ U5nU$UbUR5OSnUb UROSnURRS:Xa5U(d.U(d'[R"UUUURS9(agURnURSn U(aUR5n O5[U[R 5(aURS OXi-S-n URUU U UUURSS 9n URRS:XaZUbWUR R"S ;a=U(d6[R$"U5R&n [R("X5n U $) Nflash_attention_2rflex_attentionrFsdpa)rpast_key_values_length is_trainingrr@)sequence_length target_lengthrHrr)cudaxpunpu)r%ranyr/rNrar r$is_compileabler _ignore_causal_mask_sdparrHrKget_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positionr:r)finfomin_unmask_unattended) r9rr-rrrr'using_compilable_cacherHr4r5r min_dtypes r<r&"PersimmonModel._update_causal_masks ;; + +/B B)~/D.I.I.K.K%% ;; + +/? ?.%,,77!<^!L! ! @O?Z?99;`aCRC^!?!?di ;; + +v 5>T]n%>>*'7 MM  ""&,,Q/ !+??AMnell;;$$R(%7!; PP +')#))!, Q  KK , , 6*%%**.DD%  E*..I0CCK[Kr>r4r5rHrc UbUR5S:XaUnU$[R"U5Rn[R"X4XUR S9nUS:wa[R "USS9nU[R"X$R S9URSS5:-nUSSSS2SS24RUSSS5nUbUR5nURSn USS2SS2SS2SU 24USS2SSSS24RUR 5-n U S:Hn USS2SS2SS2SU 24RX5USS2SS2SS2SU 24'U$) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. N) fill_valuerHr:r)diagonalr!r@r)rGrNr>r?fullr:triur%rrJclonerKrL masked_fill) rr4r5rHrrrrrB mask_length padding_masks r<r=DPersimmonModel._prepare_4d_causal_attention_mask_with_cache_position\s}>  %.*<*<*>!*C(K*' E*..I** 0Y\j\q\qK!##jjqA 5<< >S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c 6 Aq!\k\12r>)rrrrrrr) NNNNNNNNN)F)r]r^r_r`rrr-rrrrNrrar rrrr rrZrr& staticmethodrrHr=rdrerfs@r<rrs"151537+/59$(,0/359X E,,-X !.X u//0 X "% X   1 12 X D>X $D>X 'tnX !!1!12X -.X  !X X B#( BellK78BllB B  B  BH4 444{{ 4  4  44r>rch^\rSrSrS/rU4Sjr\\SS\\ RS\\ RS\\ RS\\ S\\ RS \\ RS \\S \\S \\S \\ RS\\\ R4S\4Sjj55rSrU=r$)PersimmonForCausalLMizlm_head.weightc>[TU]U5 [U5UlURUl[ R "URURSS9UlUR5 g)NFr) r,r-rrrrrwrxlm_headrr~s r<r-PersimmonForCausalLM.__init__sU  #F+  ++yy!3!3V5F5FUS  r>rrrUrrlabelsrrrrlogits_to_keeprc UbUOURRnU bU OURRn UR"SUUUUUUUU U S. U D6n U Rn[ U [ 5(a [U *S5OU nURUSS2USS245nSnUb*UR"UU4SURR0U D6n[UUU RU RU RS9$)u labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, PersimmonForCausalLM >>> model = PersimmonForCausalLM.from_pretrained("adept/persimmon-8b-base") >>> tokenizer = AutoTokenizer.from_pretrained("adept/persimmon-8b-base") >>> prompt = "human: Hey, what should I eat for dinner?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] 'human: Hey, what should I eat for dinner?\n\ncat: 🐱\n\nhuman: 😐\n\n' ```N) rrrUrrrrrrr)losslogitsrrr#r)r%rrrr"r/rslicerS loss_functionrrrrr#)r9rrrUrrrUrrrrrVrrr slice_indicesrYrXs r<rZPersimmonForCausalLM.forwards)P2C1N-TXT_T_TqTq$8$D $++JjJj  ,0:: , )%+'/!5) ,  ,  11 8B>SV8W8W~ot4]k mA}a,?@A  %% ;;11 D&#33!//))   r>)rSrr) NNNNNNNNNNr)r]r^r_r`_tied_weights_keysr-rrrrNrrar rrrrrrZrdrerfs@r<rQrQs2*+151537+/59-1$(,0/35934M E,,-M !.M u//0 M "% M   1 12 M ))*M D>M $D>M 'tnM !!1!12M c5<</0M  M M r>rQc\rSrSrSrg)"PersimmonForSequenceClassificationirNr]r^r_r`rdrr>r<r`r`sfir>r`c\rSrSrSrg)PersimmonForTokenClassificationirNrarr>r<rcrcs`cr>rc)rQrrr`rc)Nr)r)ArtypingrrrrNr activationsr cache_utilsr r generationr modeling_attn_mask_utilsr modeling_flash_attention_utilsr modeling_layersrrrmodeling_outputsrrmodeling_rope_utilsrrmodeling_utilsrrprocessing_utilsrutilsrrrrutils.deprecationrconfiguration_persimmonr!torch.nn.attention.flex_attentionrintegrations.flex_attentionr get_loggerr]rModuler"rjrrrtrarIrrrrrrQr`rc__all__rr>r<rws(,, !.)>B LF&\\04 !!;J   H %!