bCi.W JSSKrSSKJrJr SSKrSSKJr SSKJ r J r J r SSK J r SSKJr SSKJr \ "5(aSSKr\"\5rS S S S .S SS.S.\ S\ S0S\ S0S.r\"\SR/55r"SS\5r"SS5r"SS\5r"SS\5r"SS\5r"SS \5r"S!S"\5r"S#S$\5r "S%S&\5r!"S'S(\5r""S)S*\5r#\\\\\\\\ \!\"\"\#S+. r$S,r%S2S-\\&S.\\'S/\&4S0jjr(S3S1jr)g)4N) NamedTupleOptional)tqdm)GGUF_CONFIG_MAPPINGGGUF_TOKENIZER_MAPPING_gguf_parse_value)is_torch_available)is_gguf_available) get_loggerversion tensor_countkv_count)r rr file_typequantization_version)rr)GGUFgeneral tokenizertokenizer_config)ignoreconfigrrrcH\rSrSr%\R \S'\\S'\\S'Sr g) GGUFTensor7weightsnamemetadataN) __name__ __module__ __qualname____firstlineno__npndarray__annotations__strdict__static_attributes__rb/home/james-whalen/.local/lib/python3.13/site-packages/transformers/modeling_gguf_pytorch_utils.pyrr7s ZZ INr)rc$\rSrSrSSjrSrSrg)TensorProcessor=Nc$U=(d 0UlgNr)selfrs r*__init__TensorProcessor.__init__>s l r)c [X05$r/rr1rrkwargss r*processTensorProcessor.processAs',,r)r0r/)rr r!r"r2r8r(rr)r*r,r,=s #-r)r,c ^\rSrSrS U4SjjrSrS S\RS\S\ \S\R4Sjjr S r U=r $) LlamaTensorProcessorEc >[TU]US9 gNr0superr2r1r __class__s r*r2LlamaTensorProcessor.__init__F 'r)c &SU;dSU;azURRS5nURRS5nSXE4;a [X05$SU;aURXU5nOSU;aURXU5n[X05$)Nz.attn_k.z.attn_q.num_attention_headsnum_key_value_heads)rgetr_reverse_permute_weights)r1rrr7 num_heads num_kv_headss r*r8LlamaTensorProcessor.processIs  t!3 (=>I;;??+@AL 00!'44T!77IVt#77LY',,r)rn_headrKreturncUbX#:waUnURSU-S-nUR"X$S/URSSQ76nURSS5RUR5$)Nrr)shapereshapeswapaxes)r1rrMrKdimws r*rI-LlamaTensorProcessor._reverse_permute_weightsVsl  #(>!FmmA&(A- OOF ?W]]12-> ?zz!Q'' 66r)rr/) rr r!r"r2r8r#r$intrrIr( __classcell__rBs@r*r;r;EsJ( -OS 7zz 7+. 7>Fsm 7  7 7r)r;cj^\rSrSrS U4SjjrSrS\RS\\ \4S\ S\4Sjr S r U=r $) Qwen2MoeTensorProcessorcc >[TU]US9 gr>r?rAs r*r2 Qwen2MoeTensorProcessor.__init__drDr)c SU;aHURS5nURS5nU(aURXX$5 [US05$SU;a[R"USS9n[X05$)N_exptensor_key_mappingparsed_parametersffn_gate_inp_shexpraxis)rH_split_moe_expert_tensorrr# expand_dims)r1rrr7rarbs r*r8Qwen2MoeTensorProcessor.processgsr T>!',@!A  & +> ? !--g$c!'444 4 'nnW15G',,r)rrbrracXCnURRSS5n[SU5HMnURSSUS35nXn[R "[ R"U55USU'MO g)N num_experts<r mlp.experts..tensors)rrHrangereplacetorch from_numpyr#copy) r1rrbrra w_counteri temp_name exp_weights r*rf0Qwen2MoeTensorProcessor._split_moe_expert_tensortsu "'KKOOM26 q)$A ^|A3a5HII J6;6F6FrwwzGZ6[ i ( 3%r)rr/) rr r!r"r2r8r#r$r'r&rfr(rXrYs@r*r[r[csI( - \zz \6:39o \MP \fj \ \r)r[c^\rSrSrS U4SjjrSrS\RS\S\4Sjr S\RS\S\4Sjr S r U=r $) BloomTensorProcessorc >[TU]US9 gr>r?rAs r*r2BloomTensorProcessor.__init__rDr)c SU;aIURSnURSnSU;aURXU5nOURXU5n[X05$)Nattn_qkvrM hidden_sizeweight)r_reverse_reshape_weights_reverse_reshape_biasr)r1rrr7rJn_embeds r*r8BloomTensorProcessor.processs_   H-Ikk-0G477GT44WQ',,r)rrMrc[R"USSS9upEnURX#U-U5nURX#U-U5nURX#U-U5n[R"XEU/SS9nURUS-X2--U5$)Nrrdr)r# array_splitrRstack)r1rrMrqkv qkv_weightss r*r-BloomTensorProcessor._reverse_reshape_weightss..!!4a IIf/ 9 IIf/ 9 IIf/ 9hhayq1 ""6A:1B#CWMMr)c[R"US5upEnURX#U-5nURX#U-5nURX#U-5n[R"XEU/SS9R 5nU$)Nrrrd)r#rrRrflatten)r1rrMrq_biask_biasv_biasqkv_biass r*r*BloomTensorProcessor._reverse_reshape_biassu"$!;6(9:6(9:6(9:88VV41=EEGr)rr/) rr r!r"r2r8r#r$rWrrr(rXrYs@r*rzrzsS(- N NC NRU N RZZ  s  r)rzc2^\rSrSrSU4SjjrSrSrU=r$)T5TensorProcessorc >[TU]US9 gr>r?rAs r*r2T5TensorProcessor.__init__rDr)c SnURS5H%nUR5(dM[U5n O [XSU05$)Nrmbid)splitisdigitrWr)r1rrr7rchunks r*r8T5TensorProcessor.processsCZZ_E}}%j%'%66r)rr/rr r!r"r2r8r(rXrYs@r*rrs(77r)rc2^\rSrSrSU4SjjrSrSrU=r$)GPT2TensorProcessorc >[TU]US9 gr>r?rAs r*r2GPT2TensorProcessor.__init__rDr)c SU;dSU;d SU;dSU;a URnUS:XaFSnURS05n[R"[R "U55USU'Sn[ X05$) Nzattn_qkv.weightzffn_down.weightz ffn_up.weightzattn_output.weight output.weightzlm_head.weightrbrn)TrHrqrrr#rsr)r1rrr7rbs r*r8GPT2TensorProcessor.processs  % D($&#t+iiG ? "$D & +> C 161A1A"'''BR1S i ( .D',,r)rr/rrYs@r*rrs(--r)rc2^\rSrSrSU4SjjrSrSrU=r$)MambaTensorProcessorc >[TU]US9 gr>r?rAs r*r2MambaTensorProcessor.__init__rDr)c SU;a[R"USS9nSU;a[R"U*5n[X05$)Nzssm_conv1d.weightrrdssm_a)r#rglogrr6s r*r8MambaTensorProcessor.processsB $ &nnW15G d?ffgX&G',,r)rr/rrYs@r*rrs( - -r)rc2^\rSrSrSU4SjjrSrSrU=r$)NemotronTensorProcessorc >[TU]US9 gr>r?rAs r*r2 NemotronTensorProcessor.__init__rDr)c 0SU;aUS- n[X05$Nz norm.weightrr5r6s r*r8NemotronTensorProcessor.process D kG',,r)rr/rrYs@r*rrs(--r)rc2^\rSrSrSU4SjjrSrSrU=r$)Gemma2TensorProcessorc >[TU]US9 gr>r?rAs r*r2Gemma2TensorProcessor.__init__rDr)c 0SU;aUS- n[X05$rr5r6s r*r8Gemma2TensorProcessor.processrr)rr/rrYs@r*rrs( --r)rc2^\rSrSrSU4SjjrSrSrU=r$)Lfm2TensorProcessorc >[TU]US9 gr>r?rAs r*r2Lfm2TensorProcessor.__init__rDr)c PSU;a[R"USS9n[X05$)Nzshortconv.conv.weightrrd)r#rgrr6s r*r8Lfm2TensorProcessor.processs' "d *nnW15G',,r)rr/rrYs@r*rrs(--r)r) llamaqwen2moeqwen3moebloomt5 t5encodergpt2mambanemotrongemma2gemma3lfm2cXR;a/$URUnURVs/sH&n[URUUR5PM( sn$s snfr/)fieldsdatar partstypes)readerfieldvalue _data_indexs r* read_fieldrsT MM! MM% EX]XbXb cXb ekk+6 DXb cc cs-A model_type num_layers qual_namec [5(a[5(a SSKJnJn O [ R S5 [S5eUcURROUnUcURROUnUS:XaSnO#US:XaS nOUS :XaS nOUS :XaS nOUS:XaSnSnUR5HupxX:XdM Un O Uc[SUS35eU"Xb5n 0n UR5n U Hn US;aSU ;a[R"SSU 5n U SpU R!S5(dU R!S5(aU R#SS5upSU-nU R%U 5nUcMX<-XU-'M UR'5=n(a[UHUun n[)UXUU S3S9nUR5VVs0sHunnUU ;dMUU_M nnnU R+U5 MW U $s snnf)a= GGUF uses this naming convention for their tensors from HF checkpoint: `blk.N.BB.weight` and `blk.N.BB.bias` where N signifies the block number of a layer, and BB signifies the attention/mlp layer components. See "Standardized tensor names" in https://github.com/ggerganov/ggml/blob/master/docs/gguf.md for details. r)MODEL_ARCH_NAMESget_tensor_name_mapLoading a GGUF checkpoint in PyTorch, requires both PyTorch and GGUF>=0.10.0 to be installed. Please see https://pytorch.org/ and https://github.com/ggerganov/llama.cpp/tree/master/gguf-py for installation instructions.KPlease install torch and gguf>=0.10.0 to load a GGUF checkpoint in PyTorch.Ncoherez command-r qwen2_moer qwen3_moer gemma3_textrumt5rzUnknown gguf model_type: z in gguf-py. This might because you're using an outdated version of gguf-py package, you can install `gguf` package from source refer to https://github.com/ggerganov/llama.cpp/tree/master/gguf-py#development)rrrlzmlp.experts.\d+.z.weightz.biasrmr)r)r r ggufrrloggererror ImportErrorrrnum_hidden_layersitemsNotImplementedError state_dictresubendswithrsplitget_namenamed_childrenget_gguf_hf_weights_mapupdate)hf_modelrrrrrarchkeyrname_mapgguf_to_hf_name_maprhf_namersuffix gguf_namerchildsub_maprrs r*rrs>133>>  A ghh/9/A++zJ6@6H22jJX { " { " } $ v  D&,,.   D / |!' |4U U  #44H$$&J 1 1n6Off0.'JGf   I & &'*:*:7*C*C">>#q1LD6\F%%d+   2;2E./$"0022~2)KD%-eZZcYdeidjjkWlmG(/ X11DW;Wtq!tGX  & &w / * Ys  H!Hc, ^&^'[5(a[5(a SSKJnJn O [ R S5 [S5eU"U5nURn[UR55n[Vs0sHo0_M n n[US5Sn [US5n Sn SU ;a S U ;aS n OXS U ;dS U ;aJS U S S'U (a)SU SR5;aSn S U ;a S/U S S'OS U ;a S/U S S'S n OU n SU ;aSn OSU ;aSn SU ;aX1Skm&Sm'[U&4SjUR55n [U'4SjUR55nXS S'U(+U S S'U [ ;aU [ ;a[#SU S35eS S!/n[%S"UR55=(d X;U S S#'URR'5GHHunnUR)X5nUR+S$5nUSnS$R-US%S5nUR.Vs/sH&n[1UR2UUR45PM( nn[7U5S%:XaUSn[9U[:5(aU U;aUR)X5n[R'5HMunnUU;dMUUU;dMUUUnUS&:XaM)UbUU UU'UU;dM<UR=U5 MO UU;dGM-[ R?S'US(U35 GMK U S S)S*:XaS+U S S)'U S S)S,:XaPU S S-n[AU5U S S-'S.U S S/'[CU5VVs/sHunnUS:dMUPM snnU S S0'S1U S ;a5U S2nS3U;a[7US35U S S1'O[ RES45 U(a0U S5'[GU5nU RIS 05n[JRIU [L5n U "US69n![OURS7S89Hn"U"RPn#U"U"R.U"RR5n$U!RUU$U#UU S99n%U%RVn$U%RPn#U#U;aM^UU#n#[XRZ"[\R^"U$55U S5U#'M [7U5S:a[ R?S:U35 U $s snfs snfs snnf);a Load a GGUF file and return a dictionary of parsed parameters containing tensors, the parsed tokenizer and config attributes. Args: gguf_checkpoint_path (`str`): The path the to GGUF file to load return_tensors (`bool`, defaults to `False`): Whether to read the tensors from the file and return them. Not doing so is faster and only loads the metadata in memory. r) GGUFReader dequantizerrzgeneral.architecturez general.nameNrmistralrrTr is_gated_actrUMT5EncoderModel architecturesT5EncoderModelrrrrstablelm> attn_k.bias attn_q.bias attn_v.biasffn_normc3T># UHnTHo"UR;v M M g7fr/r).0tensor bias_nameattn_bias_names r* 'load_gguf_checkpoint..s$mnF^lQZFKK/^l/ns%(c3B># UHnTUR;v M g7fr/r)rr ffn_norm_names r*rrs#^~VMV[[$@~s use_qkv_biasuse_parallel_residualzGGUF model with architecture z is not supported yet.falconrc3@# UHnSUR:gv M g7f)rNr)rrs r*rrs HvOv{{ *stie_word_embeddingsrmrz1Some keys were not parsed and added into account z | rrrrrGFblock_auto_adjust_ff_dimfull_attn_idxs vocab_sizertokenszCan't find a way to retrieve missing config vocab_size from tokenizer parameters. This will use default value from model config class and cause unexpected behavior.rnr0z,Converting and de-quantizing GGUF tensors...)desc)rrrarbz0Some keys of the GGUF file were not considered: )0r r rr r rrrrlistkeysGGUF_TO_TRANSFORMERS_MAPPINGrloweranyrnGGUF_SUPPORTED_ARCHITECTURES ValueErrorallrrprjoinrr rrlen isinstancer&removeinfomax enumeratewarningrrHTENSOR_PROCESSORSr,rr tensor_typer8rrqrrr#rs)(gguf_checkpoint_pathreturn_tensors model_to_loadr r rr reader_keysrrb architecture model_nameupdated_architecturerr! exceptionsgguf_keyrrprefix config_keyrr parameterparameter_renamesrenamed_config_keygguf_num_key_value_headsrurKtokenizer_parametersrarProcessorClass processorrrrresultrrs( @@r*load_gguf_checkpointrPks133//  A ghh , -F ]]Fv{{}%K(DE(D1B(DEf&<=a@LFN3J,9 #:(   !<6:(#N3 &JqM$7$7$99#) l*@R?S!(+O<l*@P?Q!(+O<#' +\!* | #* \!F" mfnnmm ##^v~~#^ ^6>(#N3CX?X(#$;<77z%J"%+%1GL%i01CD{*&&x0-Q { " KKKH:UXY^X_` a71<"<0H<4A(#L1"<0F:#4X#>?T#U =@AY=Z(#$9:BG(#$>? &//G%H9 %H/!\L[\L\A%H9 (#$45 ,X660= + +8;>0^_F;;D f.@.@AG&&#5"3 'FnnG;;D--%d+D161A1A"'''BR1S i ( .'`* ;! F{mTU aFviH9 s? T>-T  TT)NNr)FN)*rtypingrrnumpyr# tqdm.autor integrationsrrr utilsr utils.import_utilsr utils.loggingr rqrrr-r+r,r0rr,r;r[rzrrrrrrr;rr&rWrrPrr)r*rXs  ' &1% H  !*" "-F\] "5kBC$&<=O&PQ  $$@$J$O$O$QR --7?7<\o\<$?$N 7 7-/-4 -? - -o- -O --/-"'' ! "  !'##   d!% $ O O O Odir)