h#_SSKrSSKrSSKJr SSKJr SSKJr SSKJ r SSK J r J r J r JrJr SSKrSSKJr SSKJr SSKJr SS KJrJrJrJrJr SS KJr S S KJ r J!r! \"5(aSSK"r"SS K"J#r#J$r$J%r%J&r& \ (aSSK'J(r( SSK)J*r* SSKJ+r+ SSK,J-r- \ \!4r.\ "SS55r/S\/0r0S1S\S\S\\ SS\\1S\2\\44 Sjjr3S2S\S\S\4S\\1S\2\\\5\144 Sjjr6S3Sjr7S4Sjr8S4Sjr9S3S jr:\S5S\S!S"S#S$\;4S%jj5r"S(S)5r?S\S*\\@S\4S+jrAS,rBS\S-\S.S/\S\4S0jrCg)7N)Callable)contextmanager)deepcopy) dataclass) TYPE_CHECKINGAnyLiteralOptionalUnion) is_peft_model)version) AddedToken AutoTokenizerPreTrainedModelPreTrainedTokenizerTrainingArguments)is_peft_available)!AutoModelForCausalLMWithValueHead"AutoModelForSeq2SeqLMWithValueHead) PeftConfig PeftModelget_peft_modelprepare_model_for_kbit_training) Accelerator)DeepSpeedEngine)Module)DistributedDataParallelc\rSrSr%SrSr\\S'Sr\\S'Sr \\S'\ S5r \ S 5r \ S 5r \ S 5rS rg )ChatMlSpecialTokens4ziDataclass for special tokens used in ChatML, including system, user, assistant, bos, eos, and pad tokens.z <|im_start|> bos_tokenz <|im_end|> eos_token pad_tokenc URS3$)Nsystemr"selfs J/home/james-whalen/.local/lib/python3.13/site-packages/trl/models/utils.pyr&ChatMlSpecialTokens.system<s..!((c URS3$)Nuserr'r(s r*r.ChatMlSpecialTokens.user@s..!&&r,c URS3$)N assistantr'r(s r*r1ChatMlSpecialTokens.assistantDs..!++r,cVSURSURSURS3$)Nz {% for message in messages %}{{'z2' + message['role'] + ' ' + message['content'] + 'z7' + ' '}}{% endfor %}{% if add_generation_prompt %}{{ 'z ' }}{% endif %})r"r#r1r(s r* chat_template!ChatMlSpecialTokens.chat_templateHsA NN##VW[WeWeVfg^^$%   r,N)__name__ __module__ __qualname____firstlineno____doc__r"str__annotations__r#r$propertyr&r.r1r4__static_attributes__r6r,r*r r 4sqs#Is#!Is!!Is! ))'',,  r,r chatmlmodel tokenizerformatresize_to_multiple_ofreturnc[R"S[5 URb [ S5eU[ ;a#[ SUS[ R 535e[ U"5nURUlURUlURUl URSURUR/05 URUlUR[UR5UbUOSS9 [USS5bQURUR lUR"UR lUR$UR l[US S5bQUR"UR&lUR$UR&lURUR&lX4$) a Setup chat format by adding special tokens to the tokenizer, setting the correct format, and extending the embedding layer of the model based on the new special tokens. This function is deprecated and will be removed in version 0.26.0. Please use [`clone_chat_template`] instead. If the model already has a chat template, this will throw an error. If you want to overwrite it, please set `tokenizer.chat_template` to `None`. Args: model (`~transformers.PreTrainedModel`): The model to be modified. tokenizer (`~transformers.PreTrainedTokenizer`): The tokenizer to be modified. format (`Optional[Literal["chatml"]]`): The format to be set. Defaults to "chatml". resize_to_multiple_of (`int` or `None`): Number to resize the embedding layer to. Defaults to None. Returns: model (`~transformers.PreTrainedModel`): The modified model. tokenizer (`~transformers.PreTrainedTokenizer`): The modified tokenizer. zThe `setup_chat_format` function is deprecated and will be removed in version 0.26.0. Please use `clone_chat_template` instead.NzcChat template is already added to the tokenizer. If you want to overwrite it, please set it to NonezFormat z" not available. Please use one of additional_special_tokensnew_num_tokenspad_to_multiple_ofconfiggeneration_config)warningswarnDeprecationWarningr4 ValueErrorFORMAT_MAPPINGkeysr#r$r"add_special_tokensresize_token_embeddingslenvocabgetattr pad_token_idrK bos_token_id eos_token_idrL)rArBrCrD chat_formats r*setup_chat_formatr\Ws> MM ) * q  ^#76(*L^M`M`MbLcdee (*K&//I%//I%//I   "= @U@UWbWlWl?m!no)77I !!9??+4I4U0[_ "uh%1$-$:$: !$-$:$: !$-$:$: !u)40rzAVocabulary size mismatch after resizing: tokenizer vocab size is z, but model embedding size is zB. This indicates an internal error in the token alignment process.)rfrom_pretrainedget_chat_templater4added_tokens_decodervaluescontentrV add_tokensr#rZrKrLrTrU vocab_sizerappend RuntimeErrorconvert_tokens_to_ids) rArBr]rDtokenizer_sourcetoken added_tokensidx dummy_tokenis_addeds r*clone_chat_templaterps^%445JK/@@BI,@@GGII%]]bkbqbqMqI&+44I ) 6 6ELL+4+A+AE( !!9??+4I4U0[_ " C   S1 1 :cU!!45 '' 4 q q=    ,   S1 1 9??u///OPST]TcTcPdOef!!&!1!1 22t v  0<<|eMM|L<22<@L \ ))KF=sG9)G9 G>c[US5(dgURb2[URS5(aURRnO%URb URnO [S5e[ UR SS9HnUR R5 M URHnUR5 M URHnUR5 M /Ul/Ul g)z:Removes the optimizer hooks from a DeepSpeed ZeRO-3 model. optimizerNparameter_offload8The model optimizer is None, which is not yet supported.Trecurse) hasattrrrrsrh iter_paramsmoduleds_active_sub_modulesclear forward_hooksremovebackward_hooks)rAoptimizer_offloadparamhooks r* remove_hooksrs 5+ & & "wu@S'T'T!OO==  $!OOUVV.55tD ##))+E"// 0!00 1')#')$r,cf[R"URUS9UR55$)Nru) itertoolschainnamed_parametersds_external_parameters) sub_modulervs r*get_all_parametersrs* ??:66w6GIjIjIl mmr,cN[X5VVs/sHup#UPM snn$s snnfN)r)ryrv_rs r*rxrxs#"4V"E F"EhaE"E FF Fs!cSSKn[US5(dgURb2[URS5(aURRnO%URb URnO [ S5e[ R "UR5[ R "S5:aURUR5 gURUR5 g)z7Adds the optimizer hooks from a DeepSpeed ZeRO-3 model.rNrrrsrtz0.16.4) deepspeedrwrrrsrhr parse __version___register_deepspeed_modulery_register_hooks_recursively)rArrs r* add_hooksr!s 5+ & & "wu@S'T'T!OO==  $!OOUVV}}Y**+w}}X/FF445F5M5MN556G6N6NOr,)rr acceleratorrgather_deepspeed3_paramsc#:# URU5nURnU(aUR5 URRbURRR S:Xa{U(dURU5v OdSSKnURRUR55 [U5 URU5v [U5 SSS5 OUv U(aUR5 gg!,(df  N'=f7f)a Context manager to unwrap distributed or accelerated models for generation tasks. Args: model (`Union[DistributedDataParallel, DeepSpeedEngine]`): Model to be unwrapped. accelerator (`~accelerate.Accelerator`): Accelerator instance managing the model. gather_deepspeed3_params (`bool`, *optional*, defaults to `True`): Whether to gather weights for DeepSpeed ZeRO Stage 3 models. If `False`, skips parameter gathering, which can be more memory-efficient but may lead to slower generation times. Yields: Unwrapped model. Example: ```python with unwrap_model_for_generation(model, accelerator) as unwrapped_model: generated_outputs = unwrapped_model.generate(input_ids) ``` Nr) unwrap_modelis_gradient_checkpointinggradient_checkpointing_disablestatedeepspeed_plugin zero_stagerzeroGatheredParameters parametersrrgradient_checkpointing_enable)rArrunwrapped_modelrrs r*unwrap_model_for_generationr4s6"..u5O / I I 668))5+:K:K:\:\:g:gkl:l'**51 1 2253C3C3EFU#!..u55% GF  557! GFsB8D:*D $&D DDcSSKnURRn[UR5nUSSnUb{[ UR SS5(a[UR R5O[ UR SS5nUb&US:Xa URXf-SU-S U-U-S .5 US:waSUSS'URXS 9tpUR5 U$) aPrepares the model for DeepSpeed inference or evaluation by initializing it with the appropriate configuration. Adapted from accelerate: https://github.com/huggingface/accelerate/blob/739b135f8367becb67ffaada12fe76e3aa60fefd/src/accelerate/accelerator.py#L1473 rNzero_optimizationstage hidden_sizes hidden_sizer g?)z$zero_optimization.reduce_bucket_sizez4zero_optimization.stage3_param_persistence_thresholdz-zero_optimization.stage3_prefetch_bucket_size)rArK) rrrrdeepspeed_configrWrKmaxrupdate initializeeval)rArrr config_kwargsrrrs r*prepare_deepspeedrcs "((99->>?M - .w 7E u||^T::  )) *}d;   "uz  !>!c+BcBc&22 + < <*AA"-"@"@!,!>!> + < <*::(66*::!,!>!>$++  U%f% JJL Lr,c \rSrSrSrS\R S\R S\S\S\4 Sjr S\R S\R S S 4S jr S\R S\R S S 4S jr S r g )_ForwardRedirectionia9Implements the `forward-redirection`. Taken from Pytorch-lightning: https://github.com/Lightning-AI/pytorch-lightning/blob/02311d03fb982560246eead7c08104481fac9579/src/lightning/pytorch/strategies/strategy.py#L602 A method call to a wrapped module gets rerouted through the wrapper's `forward` method instead. wrapper_moduleoriginal_modulemethodargsrc^^^^^TRmS[S[S[4UUUUU4SjjnUTlT"U0UD6nTRTT5 U$)ajReroutes a method call through the `wrapper_module`'s `forward` method. Args: wrapper_module: The module that has `original_module` wrapped. original_module: The module that was wrapped inside `wrapper_module`. method: The method that should be called on the `original_module` after inputs get redirected through the `wrapper_module`'s `forward` method. *args: The positional arguments to the `method`. They will get passed to a patched `forward` method instead. **kwargs: The keyword arguments to the `method`. They will get passed to a patched `forward` method instead. _args_kwargsrEcJ>TTlT"U0UD6nTRTT5 U$r)forwardon_after_inner_forward)rroutroriginal_forwardrr)rs r*wrapped_forward5_ForwardRedirection.__call__..wrapped_forwards1'7O #%+7+C  ' ' HJr,)rron_after_outer_forward) r)rrrrrrwrapper_outputrs ```` @r*__call___ForwardRedirection.__call__s_ +22 C C C  #2'88 ##NODr,rENcgrr6r)rrs r*r*_ForwardRedirection.on_after_inner_forward r,cgrr6rs r*r*_ForwardRedirection.on_after_outer_forwardrr,r6) r7r8r9r:r;nnrrrrrrr?r6r,r*rrs  ii :<)) MU ^a mp D RYY QSQZQZ _c  RYY QSQZQZ _c r,rgradient_checkpointing_kwargscL[U5(aURR5 OUR5 U=(d 0nSU;=(d USnU(aE[US5(aUR 5 U$SnUR 5R U5 U$)z-Enables gradient checkpointing for the model. use_reentrantenable_input_require_gradsc&URS5 g)NT)requires_grad_)ryinputoutputs r*make_inputs_require_grad?enable_gradient_checkpointing..make_inputs_require_grads%%d+r,)r base_modelrrwrget_input_embeddingsregister_forward_hook)rArrrs r*enable_gradient_checkpointingrs U 668 ++-$A$GR!<<n@]^m@n 56 7 7  , , . L  ,  & & ( > >?W X Lr,c^UR5Humn[U[RR5(dST;a!UR [R 5nMV[U4SjS55(dMr[US5(dMURR[R :XdMUR [R5nM g)Nnormc3,># UH oT;v M g7frr6).0xnames r* .peft_module_casting_to_bf16..sN$Mqd$Ms)lm_head embed_tokenswtewpeweight) named_modulesrtorchr LayerNormtofloat32anyrwrdtypebfloat16)rAryrs @r*peft_module_casting_to_bf16r s++- f fehh00 1 1Vt^YYu}}-F N$MN N Nvx((==&&%--7#YYu~~6F .r, peft_configrrc[5(d [S5e[U[5(aUbUR 5n[ USS5=(d [ USS5nSn[ USS5(aXUR 5HDupVURRS:XdM!URRRS;n O U(a7U(d0[UURUR=(d 0S9nSUl O&UR(a[XR5nUbk[ R""[$R&5[ R""S5:a$[ USS5(aU(a [)XSS 9nO [)X5nUR*(a$[ USS5(aU(d [-U5 U$) z#Prepares a model for PEFT training.z=PEFT is required to use a peft model. Run `pip install peft`.is_loaded_in_4bitFis_loaded_in_8bit Params4bit>cpumeta)use_gradient_checkpointingrz0.12)autocast_adapter_dtype)r ImportErrorrrmerge_and_unloadrWr __class__r7datartypergradient_checkpointingrrr rpeftrrbf16r )rAr ris_qlorais_sharded_qlorarrs r*prepare_peft_modelrs   YZZ%## (?&&(u159gWUL_af=gHu)511..0HA''<7#(::#4#4#9#9_#L 1 (/ '+'B'B*.*L*L*RPR  ',# $ $-e5W5WX MM$** +w}}V/D D2E:: "5eTE"56E yyWU$7??HX#E* Lr,)r@N)@)rArrEN)F)T)rArrr)DrrMcollections.abcr contextlibrcopyr dataclassesrtypingrrr r r rtorch.nnraccelerate.utilsr packagingr transformersrrrrrtransformers.utilsrmodeling_value_headrrrrrrr acceleraterdeepspeed.runtime.enginerrtorch.nn.parallel.distributedrSUPPORTED_ARCHITECTURESr rQinttupler\r<listrprrrxrboolrrrrdictrr rr6r,r*r3s$%!?? *kk0f[[&8E&&     >/0 +3+/ K K"K WX& 'K$C= K  ?/ /0 Kd,. Z* Z*"Z*Z*$C= Z*  ?/c :; Z*z*.nGP&&*+8 = >+8+8#+8+8\%P:0 0 f ;CD>:72 2)1,)?2GX22r,