f:i SrSSKJr SSKrSSKJr SSKJr SSKJrJ r J r J r J r J r JrJr SSKJrJrJrJrJrJrJrJrJrJrJrJrJrJ r JrJrJrJrJrJ r J r J!r!J"r"J#r#J$r$J%r%JrJ&r&J'r'J(r(J)r)JrJ*r* SSK&r&SSK7 SSK+J,r,J-r- SS K.J/r/ SSKrSSK0r1SS K2J3r3 SSKJr SS K4J5r5J6r7 SS K8J9r9 SSK:r:SS K;J\R~"SS\>S9S5r@S\RS\AS\AS\R4SjrBS\RS\RS\AS\AS\R4 SjrCS\RS\AS\R4SjrD\,"S S!\55rE"S"S#\5rF"S$S%\F5rGg)&z; 2025.10.10 2025.10.9 4.56.2 0.23.0 __UNSLOTH_VERSIONING__ )TensorN) functional)AnyListOptionalTupleUnionDictSetCallable)!rAutoModelForCausalLMBaseImageProcessorr DataCollatorDataCollatorForChatMLDatasetEvalPredictionFFeatureExtractionMixin GKDConfig GKDTrainerGenerationConfigr PeftConfigPreTrainedModelPreTrainedTokenizerBaseProcessorMixin SFTTrainerTrainerCallbackr disable_dropout_in_model empty_cachegenerate_model_cardget_comet_experiment_urlis_wandb_availablennosprepare_deepspeedrandomtextwraptorchunwrap_model_for_generation)*) dataclassfield)Version) nullcontext)DataCollatorForSeq2SeqDataCollatorForLanguageModeling) ParallelMode) MethodTypecF^[R"T5U4Sj5nU$)Nc8>[US5(a5[URS5(aURR5 T"U/UQ70UD6n[US5(a5[URS5(aURR5 U$)Nmodel for_training for_inference)hasattrr5r6r7)selfargskwargsoutputfs >/home/james-whalen/unsloth_compiled_cache/UnslothGKDTrainer.pywrapper*prepare_for_training_mode..wrapper0sx 4 ! !gdjj.&I&I JJ # # %4)$)&) 4 ! !gdjj/&J&J JJ $ $ & ) functoolswraps)r=r?s` r>prepare_for_training_moderD/s%__Q NrATF)epilogue_fusion max_autotune shape_paddingz trace.enabledztriton.cudagraphs)dynamic fullgraphoptionscd[R"URSURS5SSS9n[R"URS5SSS9n/n[ X#5HupVUR [R 5n[R"USURS5S9RS5n[R"USS9nXx- n URU 5 M [R"U5nURURSURS45nU$)Nr)chunksdim)rOindexrO) r(chunkreshapeshapeziptofloat32gather unsqueezesqueeze logsumexpappendconcat) logitsrPchunked_logits chunked_indexall_per_token_logps chunk_logits chunk_indexselected_logitslogsumexp_valuesper_token_logpss r>chunked_selective_log_softmaxrhEs[[FLL4D!EPQYZ[N[[r!2QaHM%(%G! #u}}5 ,,|2{G\G\]_G`aiijlm ??calculate_pad_tokens_in_promptrsWsX++STTq"2N?"223N"2L#''A'. rAcompletion_input_idsleft_pad_tokens_per_prompt max_left_padcURupEURnX!- n[R"XVS9R S5nXR S5:n X:gn X-n U $)a) Given that we have a sequence, [p,p,p,c,c,c,pad,pad,pad] Where p are extra prompt tokens we got from slicing the torch tensor, c is completion tokens and pad are pad tokens, this function would make a completion mask that would 0 out the pad and p tokens. so in this example [0,0,0,1,1,1,0,0,0] )devicerrR)rUrxr(arangerZ) rtrurvrk batch_sizecompletion_lenrxnum_tokens_to_maskindices shift_masknon_padding_mask final_masks r> create_completion_attention_maskrjsi"6!;!;J ! ( (F%Bll>9CCAFG88;;J,<.J rAtensorpad_idclX:gn[R"USSSS9n[R"USU5nU$)zD Moves all padding tokens in each sequence of a batch to the right. rRT)rO descendingstable)r(argsortrY)rrmasksorted_indices packed_tensors r>left_pack_paddingrs8  D]]4Q4MNLLN;M rAc^\rSrSr%Sr\"SSS0S9r\\\ S'\"SSS 0S9r \\ \ S '\"SSS 0S9r \\ \ S 'SU4S jjr SrU=r$)UnslothGKDConfiga Configuration class for [`GKDTrainer`]. This class includes only the parameters that are specific to GKD training. For a full list of training arguments, please refer to the [`~transformers.TrainingArguments`] and [`SFTConfig`] documentation. Args: temperature (`float`, *optional*, defaults to `0.9`): Temperature for sampling. The higher the temperature, the more random the completions. lmbda (`float`, *optional*, defaults to `0.5`): Lambda parameter that controls the student data fraction (i.e., the proportion of on-policy student-generated outputs). beta (`float`, *optional*, defaults to `0.5`): Interpolation coefficient between `0.0` and `1.0` of the Generalized Jensen-Shannon Divergence loss. When beta is `0.0`, the loss is the KL divergence. When beta is `1.0`, the loss is the Inverse KL Divergence. max_new_tokens (`int`, *optional*, defaults to `128`): Maximum number of tokens to generate per completion. teacher_model_name_or_path (`str` or `None`, *optional*, defaults to `None`): Model name or path of the teacher model. If `None`, the teacher model will be the same as the model being trained. teacher_model_init_kwargs (`dict[str, Any]]` or `None`, *optional*, defaults to `None`): Keyword arguments to pass to `AutoModelForCausalLM.from_pretrained` when instantiating the teacher model from a string. disable_dropout (`bool`, *optional*, defaults to `True`): Whether to disable dropout in the model. seq_kd (`bool`, *optional*, defaults to `False`): Seq_kd parameter that controls whether to perform Sequence-Level KD (can be viewed as supervised FT on teacher-generated output). NhelpzvLLM SamplingParams)defaultmetadatavllm_sampling_paramsrLz8Chunk size to reduce memory usage. -1 is most efficient.unsloth_num_chunksz'Maximum sequence length to truncate to.max_seq_lengthc p>US:a[SUS35 US:a[SUS35 UcU#S:Xa U$S:XaSnS n#Wc$S S KJn [[ U"5S -S 5S5n[ R RSS5S:XaS SKJ n U(a WcS SKJ n UnWS ::a [S5eWS:a [S5e[TU]4"S0SU_SU_SU_SU_SU_SU_SU_SU_SU _S U _S!U _S"U _S#U _S$U_S%U_S&U_S'U_S(U_S)U_S*U_S+U_S,U_S-U_S.U_S/U_S0U_S1U_S2U_S3U_S4U_S5U_S6U _S7U!_S8U"_S9U#_S:U$_S;U%_SU(_S?U)_S@U*_SAU+_SBU,_SCU-_SDU._SEU/_SFU0_SGU1_SHU2_SIU3_SJU4_SKU5_SLU6_SMU7_SNU8_SOU9_SPU:_SQU;_SRU<_SSU=_STU>_SUU?_SVW@_SWWA_SXWB_SYWC_SZWD_S[WE_S\WF_S]WG_S^WH_S_WI_S`WJ_SaWK_SbWL_ScWM_SdWN_SeWO_SfWP_SgWQ_ShWR_SiWS_SjWT_SkWU_SlWV_SmWW_SnWX_SoWY_SpWZ_SqW[_SrW\_SsW]_StW^_SuW__SvW`_SwWa_SxWb_SyWc_SzWd_S{We_S|Wf_S}Wg_S~Wh_SWi_SWj_SWk_SWl_SWm_SWn_SWo_SWp_SWq_SWr_SWs_SWt_SWu_SWv_SWw_SWx_SWy_SWz_SW{_SW|_SW}_SW~_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_WD6 WUlWUlWUlg)NgHz>z Unsloth: Your learning rate of `zi` is too small and less than 1e-7! Consider increasing it, otherwise gradient updates will be close to 0!rRza` is way too larger > 1! Consider decreasing it to 1e-1, otherwise gradient updates will explode!stepsunsloth_training_checkpointsnor) cpu_countrM@UNSLOTH_ENABLE_FLEX_ATTENTION01)HAS_FLEX_ATTENTION)FLEX_ATTENTION_BLOCK_SIZEzUUnsloth: Please set a positive non-zero temperature since your results will be wrong. zgUnsloth: Please set a positive non-zero temperature less than 10, since sampling will be quite erratic. output_diroverwrite_output_dirdo_traindo_eval do_predict eval_strategyprediction_loss_onlyper_device_train_batch_sizeper_device_eval_batch_sizeper_gpu_train_batch_sizeper_gpu_eval_batch_sizegradient_accumulation_stepseval_accumulation_steps eval_delaytorch_empty_cache_steps learning_rate weight_decay adam_beta1 adam_beta2 adam_epsilon max_grad_normnum_train_epochs max_stepslr_scheduler_type warmup_ratio warmup_steps log_levellog_level_replicalog_on_each_node logging_dirlogging_strategylogging_first_step logging_stepslogging_nan_inf_filter save_strategy save_stepssave_total_limitsave_safetensorssave_on_each_nodesave_only_model'restore_callback_states_from_checkpointno_cudause_cpuuse_mps_deviceseed data_seed jit_mode_evaluse_ipexbf16fp16fp16_opt_levelhalf_precision_backendbf16_full_evalfp16_full_evaltf32 local_rank ddp_backend tpu_num_corestpu_metrics_debugdebugdataloader_drop_last eval_stepsdataloader_num_workersdataloader_prefetch_factor past_indexrun_name disable_tqdmremove_unused_columns label_namesload_best_model_at_endmetric_for_best_modelgreater_is_betterignore_data_skipfsdpfsdp_min_num_params fsdp_config"fsdp_transformer_layer_cls_to_wrapaccelerator_configparallelism_config deepspeedlabel_smoothing_factoroptim optim_args adafactorgroup_by_lengthlength_column_name report_toddp_find_unused_parametersddp_bucket_cap_mbddp_broadcast_buffersdataloader_pin_memorydataloader_persistent_workersskip_memory_metricsuse_legacy_prediction_loop push_to_hubresume_from_checkpoint hub_model_id hub_strategy hub_tokenhub_private_repohub_always_push hub_revisiongradient_checkpointinggradient_checkpointing_kwargsinclude_inputs_for_metricseval_do_concat_batches fp16_backendpush_to_hub_model_idpush_to_hub_organizationpush_to_hub_token mp_parametersauto_find_batch_sizefull_determinism torchdynamo ray_scope ddp_timeout torch_compiletorch_compile_backendtorch_compile_modeinclude_tokens_per_secondinclude_num_input_tokens_seenneftune_noise_alphaoptim_target_modulesbatch_eval_metrics eval_on_startuse_liger_kernelliger_kernel_configeval_use_gather_objectaverage_tokens_across_devicesmodel_init_kwargschat_template_pathdataset_text_fielddataset_kwargsdataset_num_proc eos_token pad_token max_lengthpackingpacking_strategy padding_freepad_to_multiple_of eval_packingcompletion_only_lossassistant_only_loss loss_typeactivation_offloading temperaturelmbdabetamax_new_tokensteacher_model_name_or_pathteacher_model_init_kwargsdisable_dropoutseq_kd)printmultiprocessingrminmaxr$environgetunsloth_zoo.flex_attentionrr MathErrorsuper__init__rrr)r9rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r r r r rrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r(r)r*r+r,r-r.r/r0r1r2r3r4r5r6r7r8r9r:r;r<rrrr;rrr __class__s r>rGUnslothGKDConfig.__init__sB 4 )I-YB(C"D 1 e&F}oVw%xy  -7":zS?P7J M  # 1"3y{1}a#8"=  ::>>93 ?3 F E!&8&@P%>" ! st t B FG G Z &#Z Z & Z & Z & $ Z & * Z &$8Z &+FZ &*DZ &(@Z &'>Z &+FZ &'>Z &$Z &'>Z & *!Z &"(#Z &$$%Z &&$'Z &(()Z &**+Z &, 0-Z &."/Z &0!21Z &2(3Z &4(5Z &6"7Z &8!29Z &: 0;Z &<&=Z &> 0?Z &@"4AZ &B*CZ &D&?g@rLlinear皙?rpassivewarningTNrFrRFrrNTFFFFFFO rQFFFFO1autoFFNrLNNFFNrNrLNNTNFNNFrTrNNNNN adamw_8bitNFFlengthNNNNTFTFFNN every_saveNNFNTNFTrSNNNrTFFNlastiFNNFFNNFFFNFTNNtextNNNNiFbfdFNNNFnllFrK?r]NNTFNrLN)__name__ __module__ __qualname____firstlineno____doc__r,rrr__annotations__rintrrG__static_attributes__ __classcell__rHs@r>rrsY>+012+(3-*/VW*#&+EF&NXc] #$&'%&#'"&&'"#"%$%""!&!27!'!$!"%) $!& $  -1!!!$%%) $ $(-"%*!%#!%(,%*!%##' $  $!$)(-"#" "!&(, !# !## %%)$(#}R-R-rArc^\rSrSrSrSS/rS!S\\\\ R\ 4S\\\ R\ 4S\\ S \\ S \\S \\\\\ \44S \\\\\\4S \\\/\4S\\\S\\R4R6\R4R8R:4S\\\R<\R</\R<4S\SS\\4U4Sjjjr\ S"Sj5r!S#Sjr"\ S$Sj5r#S$S\ RS\\ \\R<\$44S\\%S\R<4U4Sjjjr&S%S\\ S\\ S\\ \\ S44Sjjr'S r(U=r)$)&_UnslothGKDTraineria Trainer for Generalized Knowledge Distillation (GKD) of language models. For details on GKD, see the paper: [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://huggingface.co/papers/2306.13649). Args: model ([`~transformers.PreTrainedModel`] or `torch.nn.Module` or `str`, *optional*): Model to be trained, or the string identifier of the model to be instantiated from a pretrained model. teacher_model ([`~transformers.PreTrainedModel`] or `torch.nn.Module` or `str`, *optional*): Teacher model for knowledge distillation, or the string identifier of the model to be instantiated from a pretrained model. args ([`GKDConfig`], *optional*): Training arguments. data_collator ([`~transformers.DataCollator`], *optional*): Data collator to batch samples from the dataset. It defaults to a [`DataCollatorForChatML`] using the `processing_class`. train_dataset ([`~datasets.Dataset`], *optional*): Dataset for training. eval_dataset ([`~datasets.Dataset`] or `dict` of [`~datasets.Dataset`], *optional*): Dataset for evaluation. processing_class ([`~transformers.PreTrainedTokenizerBase`], [`~transformers.BaseImageProcessor`], [`~transformers.FeatureExtractionMixin`] or [`~transformers.ProcessorMixin`], *optional*): Class to process the data. compute_metrics (`Callable`, *optional*): Function to compute metrics at evaluation. Must take in an [`~transformers.EvalPrediction`] and return a dictionary string to float. callbacks (`list` of [`~transformers.TrainerCallback`], *optional*): Callbacks to use during training. optimizers (`tuple` of `torch.optim.Optimizer` and `torch.optim.lr_scheduler.LambdaLR`, *optional*, defaults to `(None, None)`): Tuple containing the optimizer and the learning rate scheduler to use for training. preprocess_logits_for_metrics (`Callable`, *optional*): Function to preprocess the logits before computing the metrics. Must take in the `logits` and `labels` and return the logits to be used for metrics computation. peft_config ([`~peft.config.PeftConfig`], *optional*): PEFT configuration to use PEFT for training. If `None`, PEFT is not used. If provided, the `model` will be wrapped with the specified PEFT adapter. formatting_func (`Callable`, *optional*): Function to format the dataset. Must take in an example and return an example. trlgkdNr5 teacher_modelr: data_collator train_dataset eval_datasetprocessing_classcompute_metrics callbacks optimizerspreprocess_logits_for_metrics peft_configrformatting_funccP>SUlUc[XsRS9nURc SS0UlOSURS'SUlUR (a,[ URSURSS9Ul SUl[TU]-UUUUUUUU U U U U S9 URc0nOP[U[5(d [S5eURnUS S ;aUS O[!["US 5US '[U[5(a[$R&"U40UD6nUR((a[+UR,5 UR.(a R25UlOUR2R7USS 9UlUR8UlURUlURUlUR:Ul[=UR>URSS UR@(aSOSURBRDS 9Ul#[IUR,RFS5(aRUR,RFRJb0UR,RFRJURFl%ggg)NF) tokenizerr+skip_prepare_datasetT)r7 ignore_indexr5compiled) r:rnrorprqrrrsrtrurvrwzfYou passed teacher_model_init_kwargs to the GKDConfig, but your teacher_model is already instantiated.dtype)rSN)evaluation_moder)r8r5 do_sampletop_k use_cacherk eos_token_id)&rrr+r'use_liger_gkd_lossr LigerFusedLinearJSDLossr7r5liger_jsd_lossrFrGr: isinstancestrrngetattrr(r from_pretrainedr;rr5is_deepspeed_enabledr% acceleratorrm prepare_modelr6r<rr8r rqrkgeneration_configr8r)r9r5rmr:rnrorprqrrrsrtrurvrwr:rHs r>rG_UnslothGKDTrainer.__init__:sx&&+"  1D   6 7#(  "9YY! ,, #D  '+D #  ''%-+!*G#+    ) ) 1(* %M3//x )-(F(F %-W5G*'2U$=g$FG &g . mS ) )0@@lRklM    $TZZ 0  $ $!2=BRBR!SD !%!1!1!?!? _c!?!dD ZZ II ++kk !1..((#::e..;; "  DJJ00. A A ,,99E26**2N2N2[2[D " " /F BrAc pX- nX- n[R"USS9n[R"USS9nUS:Xa[R"XgSSS9nOUS:Xa[R"XvSSS9nO[R"X6R S9n[R "[R"U[R"SU- 5-U[R"U5-/5SS9n [R"XSSS9n [R"XSSS9n X:-SU- U --nUb US :gn XnUS :XaXUb!UR5W R5- $UR5URS5URS5-- $US :XaUR5$US :XaUR5$U$) a  Compute the generalized Jensen-Shannon Divergence loss for knowledge distillation using F.kl_div. See Eq. (1) of https://huggingface.co/papers/2306.13649 for the definition. Args: student_logits: Tensor of shape (batch_size, sequence_length, vocab_size) teacher_logits: Tensor of shape (batch_size, sequence_length, vocab_size) labels: Tensor of shape (batch_size, sequence_length) with -100 for padding tokens to ignore when computing loss beta: Interpolation coefficient between 0 and 1 (default: 0.5) temperature: Softmax temperature (default: 1.0) reduction: Specifies the reduction to apply to the output (default: 'batchmean') Returns: loss: Scalar tensor with the generalized JSD loss rLrQrnoneT) reduction log_targetrR)r~r{ batchmeanromean) r log_softmaxkl_divr(rr~r\stacklogrosizer) student_logitsteacher_logitslabelsr7r5rstudent_log_probsteacher_log_probsjsdmixture_log_probs kl_teacher kl_studentrs r>generalized_jsd_loss'_UnslothGKDTrainer.generalized_jsd_losss8(5'5MM.bAMM.bA 19((,6^bcC QY((,6^bcC<<,C,CDD % .1t81DDFWZ_ZcZcdhZiFijk! "3RXeijJ"3RXeijJ#q4x:&==C  T>D)C  #-3-?3779txxz) lSWWYRURZRZ[\R]`c`h`hij`kRkEl l % 779  & 88: JrAc ~UR(Ga%URRU5n[US5(a"UR 5bUR 5nO[ U[ USS5U5nU"USUSSSS9nUR R5 URRUR 5n[US5(a"UR 5bUR 5n O[ U[ USS5U5n [R"5 U "USUSSSS9n SSS5 URSS2SS 24R5n W RSS2SS 24R5n US S :gn [R"XS[R"USS 55nUSS2S S24R5nUR5nUR5nURU UR U UR U[ US S5[ US S5S9nOU"USUSS9nUR R5 [R"5 UR USUSS9n SSS5 USR"S nUR$SS2US - S 2SS24nW R$SS2US - S 2SS24nUS SS2US24nUR'UUUUR(S9n[+5 U(aUU4$U$!,(df  GN=f!,(df  N=f)N get_decoderbase_model_prefixr5riattention_maskTF)riroutput_hidden_statesrrLrr{rRbias) student_inputstudent_weight teacher_inputteacher_weight true_labels student_bias teacher_bias)rirprompts)rrrr7)rr unwrap_modelr8rrrmevalr(no_gradlast_hidden_state contiguouswhere full_likeget_output_embeddingsrweightrUr_rr7r)r9r5inputsreturn_outputsnum_items_in_batchunwrapped_student base_studentstudent_outputsunwrapped_teacher base_teacherteacher_outputsstudent_hiddenteacher_hidden labels_maskmasked_input_idsr student_head teacher_headlossprompt_lengthsshifted_student_logitsshifted_teacher_logitsshifted_labelss r> compute_loss_UnslothGKDTrainer.compute_losssn  " " " $ 0 0 = =e D (-88=N=Z=Z=\=h0<<> &%w/@BUW^'_ar  + -%&67%) O    # # % $ 0 0 = =d>P>P Q (-88=N=Z=Z=\=h0<<> &%w/@BUW^'_ar  ".$[1#)*:#;)-# #!->>q#2#vFQQSN,>>q#2#vFQQSN!*d2K${{K0%//&BUW[2\  +1ab51<<>K-BBDL,BBDL&&,+22,+22'$\64@$\64@'D$ -%&67O    # # %"&"4"4$[1#)*:#;#5#!$I.44Q7N%4%;%;A~PQ?QTV?VXYgenerate_on_policy_outputs-_UnslothGKDTrainer.generate_on_policy_outputsIs"NNY'!::&=tD/$( + -66"__-=>%++-   #59J\1 2CD <? @Z??rArrrlcn>UR(al[URUR5nUR XBUR UR R5upVnSSS5 WUS'WUS'WUS'[R"5UR::aa[XR5nUR XBUR UR R5upVnSSS5 WUS'WUS'WUS'[T U]-XU5nU$!,(df  N=f!,(df  N@=f)a9 Perform a training step for the Generalized Knowledge Distillation (GKD) model. This method implements the on-policy learning approach described in the GKD paper. With probability `self.lmbda`, it generates new responses using the student model, which are then used for training instead of the original inputs. Nrirr) r<r)rmrrrrqrkr&r6rF training_step) r9r5rrunwrapped_model new_input_idsrrrrHs r>r _UnslothGKDTrainer.training_step`s ;;,T-?-?AQAQRVe@D@_@_#T-C-CTEZEZEgEgA= :S#0F; '9F# $)F8  ==?djj (,U4D4DE@D@_@_#T-C-CTEZEZEgEgA= :F#0F; '9F# $)F8 w$U4FG #SRFEs5D75D& D#& D4 model_name dataset_nametagscUR5(dg[URRS5(ac[R R URRR5(d!URRRnOSnUc [5nO$[U[5(aU1nO [U5n[URRS5(aURS5 S[R;aURS5 URUR5 [R "S5n[#UUUR$UU['5(a+[(R*b[(R*R,OS[/5SUS S S 9 nUR1[R R3UR4R6S 55 g) a Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *optional*, defaults to `None`): Name of the model. dataset_name (`str` or `None`, *optional*, defaults to `None`): Name of the dataset used for training. tags (`str`, `list[str]` or `None`, *optional*, defaults to `None`): Tags to be associated with the model card. N _name_or_pathunsloth_versionunslothJOB_IDhf_jobsan @inproceedings{agarwal2024on-policy, title = {{On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes}}, author = {Rishabh Agarwal and Nino Vieillard and Yongchao Zhou and Piotr Stanczyk and Sabela Ramos Garea and Matthieu Geist and Olivier Bachem}, year = 2024, booktitle = {The Twelfth International Conference on Learning Representations, {ICLR} 2024, Vienna, Austria, May 7-11, 2024}, publisher = {OpenReview.net}, url = {https://openreview.net/forum?id=3zKtaqxLhW}, }GKDzPOn-Policy Distillation of Language Models: Learning from Self-Generated Mistakesz 2306.13649) base_modelrrrr wandb_url comet_url trainer_nametrainer_citation paper_titlepaper_idz README.md)is_world_process_zeror8r5configr$pathisdirrsetrraddrBupdate _tag_namesr'dedentr rr"wandbrunurlr!savejoinr:r)r9rrrrcitation model_cards r>create_model_card$_UnslothGKDTrainer.create_model_card~sn"))++  4::$$o 6 6rww}}TZZM^M^MlMl?m?m**88JJ <5D c " "6Dt9D 4::$$&7 8 8 HHY  rzz ! HHY  DOO$??$ )!!**%'9';'; @Ueiimm[_.0%j!    TYY%9%9;GHrA)r7rrr6r<rmr5r) NNNNNNNNN)NNNNN)Nr]rLr)FN)N)NNN)*r_r`rarbrcrrr rr#Modulerrrrdictrrrrr rlistrtupler(r Optimizer lr_schedulerLambdaLRrrG staticmethodrrrrrerrrfrgrhs@r>rjrjs%NJCG@D$(04+/EI FJ59Vbhl.2.2!h\oryy#=>?h\_bii<=h\y! h\  - h\  ( h\uWd3<.@%@ABh\# )+=?UWee f h\"(N+;T+A"BCh\D12h\%++//1I1I1R1RRSh\(0%,, 9UW\WcWc9c0d'eh\l+h\ "(+!h\h\TZeDDL\C|@@.rvYY(,S% c8I2J-J(Kaijman @%)&*,0 BISMBIsmBICcD() BIBIrArjcH^\rSrSrSrSU4SjjrSrU=r$)UnslothGKDTraineria Trainer for Generalized Knowledge Distillation (GKD) of language models. For details on GKD, see the paper: [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://huggingface.co/papers/2306.13649). Args: model ([`~transformers.PreTrainedModel`] or `torch.nn.Module` or `str`, *optional*): Model to be trained, or the string identifier of the model to be instantiated from a pretrained model. teacher_model ([`~transformers.PreTrainedModel`] or `torch.nn.Module` or `str`, *optional*): Teacher model for knowledge distillation, or the string identifier of the model to be instantiated from a pretrained model. args ([`GKDConfig`], *optional*): Training arguments. data_collator ([`~transformers.DataCollator`], *optional*): Data collator to batch samples from the dataset. It defaults to a [`DataCollatorForChatML`] using the `processing_class`. train_dataset ([`~datasets.Dataset`], *optional*): Dataset for training. eval_dataset ([`~datasets.Dataset`] or `dict` of [`~datasets.Dataset`], *optional*): Dataset for evaluation. processing_class ([`~transformers.PreTrainedTokenizerBase`], [`~transformers.BaseImageProcessor`], [`~transformers.FeatureExtractionMixin`] or [`~transformers.ProcessorMixin`], *optional*): Class to process the data. compute_metrics (`Callable`, *optional*): Function to compute metrics at evaluation. Must take in an [`~transformers.EvalPrediction`] and return a dictionary string to float. callbacks (`list` of [`~transformers.TrainerCallback`], *optional*): Callbacks to use during training. optimizers (`tuple` of `torch.optim.Optimizer` and `torch.optim.lr_scheduler.LambdaLR`, *optional*, defaults to `(None, None)`): Tuple containing the optimizer and the learning rate scheduler to use for training. preprocess_logits_for_metrics (`Callable`, *optional*): Function to preprocess the logits before computing the metrics. Must take in the `logits` and `labels` and return the logits to be used for metrics computation. peft_config ([`~peft.config.PeftConfig`], *optional*): PEFT configuration to use PEFT for training. If `None`, PEFT is not used. If provided, the `model` will be wrapped with the specified PEFT adapter. formatting_func (`Callable`, *optional*): Function to format the dataset. Must take in an example and return an example. c  2>Uc [5n[USS5n[U5[LaSn[USS5n[U5[LaSnSn[R R SS5S:HnU(d1[R R SS5S:Xa [S5 S n[R R S S 5n[URS S5=(d [URS S5nUcUR5RnSSK J n U"U5nU[R:HnU(dU(aU(a [S5eU(dU(dU(a [S5eU(a"SUlSUlS[R S'OCU(d<U(d5US :Xa/UUlU(+UlU(aSOS[R S'[USS5b-[USS5S:XaSUl[USS5cSUl[USS5nUb/US:a)SSKJn [-U5[-S5::a [S5 [USS5S:waL[USS5nUS:Xa!UR.U:aUR.Ul[US S5c UbUUl[US!S5n[U5[LaSn[US"S5n[U5[LaSnUR (aU(aSUlS UlUR"(aU(aS UlSUlU(aSUlSUlOc[R R S S 5S#:XaS UlSUlO0U(d)U(d"UR"UlUR UlSn S$S5bS n S%S5bS nU(aS[R S&'S'[95;a[;US'5(dOD[US'S5n[US'S5nUc'Ub$UR<n[;US'5(aUUlUb![;US(5(aUR?5 S)[95;a [;[@S*5(a S+[@l!S,[95;aU[;US*5(aS+Ul![;US)5(a,[;UR@S*5(aS+UR@l!S,[95;aUO[@nSS-K"J#n [IUU 5(dx[IU[J5(a(S.URL;a[OUSS/[US0S5S19nO[IU[N5(a%S.URL;a[KU[US0S5S29nOJ[;US35(aSUl([;US45(aS5Ul)[;US65(a S7S 0Ul*[IUU 5(dx[;US85(dg[;US)5(aV[IU[J5(a [KUR@[US0S5S29nO![OUR@SS/[US0S5S19n/n!SS9K+J,n" U""S:U!5 [US;S5[ZR\:Xa(UR^S:a[US. U D6 S=[95;a![;US?5(aURg5 [;US@5(a-URhRk5 [;US@5(aU?4[USAS5bURlUR5l6[;USB5(aVURnRpn#Un$[;U$S=5(a&U#U$l9U$Rtn$[;U$S=5(aM&U#U$l9[;USC5(a.[w[yURzR|5U5Ul>g)ENrFrUNSLOTH_ENABLE_FULL_FINETUNINGrrUNSLOTH_FORCE_FLOAT32zKUnsloth: Switching to float32 training since model cannot work with float16TUNSLOTH_MIXED_PRECISIONrXr~ torch_dtyper) _get_dtypezuUnsloth: Model is in float16 precision but you want to use bfloat16 precision. Set fp16 to `True` and bf16 to `False`zuUnsloth: Model is in bfloat16 precision but you want to use float16 precision. Set fp16 to `False` and bf16 to `True`rACCELERATE_MIXED_PRECISIONrprrrrNrrR) __version__z4.45.2z**** Unsloth: Please use our fixed gradient_accumulation_steps by updating transformers, TRL and Unsloth! `pip install --upgrade --no-cache-dir --force-reinstall --no-deps unsloth transformers trl unsloth_zoo`rrrrbfloat16rrruUNSLOTH_RETURN_LOGITSrr6ry padding_siderightrq)UnslothVisionDataCollatorrrUr/)mlmmlm_probabilityr/)r/rr&rTr'rzpad)PatchRLStatistics gkd_trainer parallel_mode_n_gpur5) r5rmr:rnrorprqrrrsrurvrwr7neftune_hook_handlerrtrainr=)?rrtypeboolr$rBrCr>rget_input_embeddingsr~unsloth_zoo.utilsrr(float16 TypeErrorrrrr transformersrr-rrrrrlocalsr8rr6ryrunsloth_zoo.vision_utilsrrr/ column_names+TransformersDataCollatorForLanguageModelingrr&r'unsloth_zoo.logging_utilsr#r1NOT_DISTRIBUTEDn_gpur&rFrGr7r'removerrscaleraccelerator_scalerr5r2rDrHr()&r9r5rmr:rnrorprqrrrsrurvrwr;use_bf16use_fp16 force_float32full_finetuningmixed_precision_dtyper~rr-ga_stepstransformers_versioneval_bszrr_output_logitsmodel_max_seq_lengthargs_max_seq_lengthr_UnslothGKDTrainer__tokenizerr other_metricsr#r8 current_modelrHs& r>rGUnslothGKDTrainer.__init__s < 0 24/ > %%x4/ > %%x **..)I3OSVVBJJNN3JC$PTW$W _ ` M " /H) T gt4b m]a8b =%"<"<">"D"D%05!5==('hyJA@B:Bg(9NEDF>F DIDI7;BJJ3 481F)1SDI# DIAHvfBJJ3 4 4 . :wt_^b?cgk?k!(D t\408C$/4!>E  HqL H+,0AA@A 4$ /4 7t%A1EH1}!A!AH!LptqQqQdNmt6=E(J^@H`d`| '7?  t +e^ '7?  t +e^ 99u)<\`dFY 99t)<[`TEX "'D "'D  ZZ^^5y AZ O"&D "'D "&))D "&))D  8<<)4 0 J]aN 25BJJ. / 68 +GDBR4S4S #*52BD#I #*42BD#I "*/C/O!&!5!54!122.D4G  !?!?    &( "wy.'I'Idk9Ka  )'88Za:J:W'55'BRB\B\^l:m:mZaopJpJpW*<*H&i F-)BCC-)?@@XUbUoUoEo K&))07KT)R ! M+VWW\dhuiCiC]C 6)07KT)R! t455TYt7Qt122bD4Kt-..G]_cFd0C-)BCC;..7; 3T3Tm-CDD$:#---4T;OQU-V%M %P#--#*--4T;OQU-V %M  ?-7 4$ /<3O3O OTXT^T^abTbtXq)Q. fh 75.#A#A      8)))'/-!,I%- 817 8 fh 75/#B#B    ! 4. / /  $ $ + + -t2339Q 4. 5 A?C?W?WE & & ( < 4 ' '%%,,F!M-1139 0 - 3 3 -1106M , 4 ! !#$=dnn>R>R$SUYZDJ rA)r() NNNNNNNNNNNN)r_r`rarbrcrGrfrgrhs@r>rrs;'T(,g g rAr)Hrcr(rtorch.nnr#rrtypingrrrrr r r r trl.trainer.gkd_trainerr rrrrrrrrrrrrrrrrrr r!r"r$r%r&r'r) dataclassesr+r,packaging.versionr-numpynp contextlibr.r/r/r0r3transformers.training_argsr1rBtypesr2rDtorch_compile_optionscompilerhrersrrrrjrr=rAr>rUs0 $IIIOOOOOOOOO (% "$3  4;PRS"|| \\ &,, %    \\ 6ellCELL ~-y~- ~-~ pIpIb P *P drA