Y:iS ~SrSSKJr SSKrSSKJr SSKJr SSKJrJ r J r J r J r J r JrJr SSKJrJrJrJrJrJrJrJrJrJrJrJrJrJ r JrJrJrJ r J!r!J"r"J#r#J$r$J r J%r%J&r&J'r'J(r(J)r)J*r*J+r+JrJ,r,J r JrJ r J#r#J'r'J)r)J+r+Jr SSK+r+SSK7 SSK-J.r.J/r/ SS K0J1r1 SSKrSSK2r3SS K4J5r5 SSKJr SS K6JrJr7 SS K8J9r9 SSK:r:SS K;J\R~"SS\>S9S5r@S\RS\AS\AS\R4SjrBS\RS\RS\AS\AS\R4 SjrCS\RS\AS\R4SjrD\."S S!\55rE"S"S#\#5rF"S$S%\F5rG\H"\)S&5(a3SSK*r*"S'S(\*R5rJ\)R"\J"S)55 gg)*z; 2025.10.10 2025.10.9 4.56.2 0.23.0 __UNSLOTH_VERSIONING__ )TensorN) functional)AnyListOptionalTupleUnionDictSetCallable)(AutoModelForCausalLM AutoTokenizerBaseImageProcessorr DataCollatorDataCollatorForLanguageModelingDataCollatorForSeq2Seq DataLoaderDatasetEvalLoopOutputFeatureExtractionMixinIterativeSFTConfigIterativeSFTTrainerr PPODecoratorsPath PeftModelPreTrainedModelPreTrainedTokenizerBaseProcessorMixinTrainerTrainingArgumentsr generate_model_cardget_comet_experiment_urlis_peft_availableis_wandb_availableloggerloggingostorchwarningsrrrrr#r%r'r()*) dataclassfield)Version) nullcontext)rr) ParallelMode) MethodTypecF^[R"T5U4Sj5nU$)Nc8>[US5(a5[URS5(aURR5 T"U/UQ70UD6n[US5(a5[URS5(aURR5 U$)Nmodel for_training for_inference)hasattrr3r4r5)selfargskwargsoutputfs Q/home/james-whalen/llama.cpp/unsloth_compiled_cache/UnslothIterativeSFTTrainer.pywrapper*prepare_for_training_mode..wrapper0sx 4 ! !gdjj.&I&I JJ # # %4)$)&) 4 ! !gdjj/&J&J JJ $ $ & ) functoolswraps)r;r=s` r<prepare_for_training_moderB/s%__Q Nr?TF)epilogue_fusion max_autotune shape_paddingz trace.enabledztriton.cudagraphs)dynamic fullgraphoptionscd[R"URSURS5SSS9n[R"URS5SSS9n/n[ X#5HupVUR [R 5n[R"USURS5S9RS5n[R"USS9nXx- n URU 5 M [R"U5nURURSURS45nU$)Nr)chunksdim)rMindexrM) r(chunkreshapeshapeziptofloat32gather unsqueezesqueeze logsumexpappendconcat) logitsrNchunked_logits chunked_indexall_per_token_logps chunk_logits chunk_indexselected_logitslogsumexp_valuesper_token_logpss r<chunked_selective_log_softmaxrfEs[[FLL4D!EPQYZ[N[[r!2QaHM%(%G! #u}}5 ,,|2{G\G\]_G`aiijlm ??9CCAFG88;;J,<.J r?tensorpad_idclX:gn[R"USSSS9n[R"USU5nU$)zD Moves all padding tokens in each sequence of a batch to the right. rPT)rM descendingstable)r(argsortrW)rrmasksorted_indices packed_tensors r<left_pack_paddingrs8  D]]4Q4MNLLN;M r?c^\rSrSr%Sr\"SSS0S9r\\\ S'\"SSS 0S9r \\ \ S '\"SSS 0S9r \\ \ S 'SU4S jjr SrU=r$)UnslothIterativeSFTConfiga Configuration class for the [`IterativeSFTTrainer`]. The [`IterativeSFTTrainer`] is deprecated and will be removed in version 0.24.0. Please use the [`SFTTrainer`]. This class includes only the parameters that are specific to Iterative SFT training. For a full list of training arguments, please refer to the [`~transformers.TrainingArguments`] documentation. Note that default values in this class may differ from those in [`~transformers.TrainingArguments`]. Using [`~transformers.HfArgumentParser`] we can turn this class into [argparse](https://docs.python.org/3/library/argparse#module-argparse) arguments that can be specified on the command line. Parameters: > Parameters that control the model model_init_kwargs (`dict[str, Any]` or `None`, *optional*, defaults to `None`): Keyword arguments for [`~transformers.AutoModelForCausalLM.from_pretrained`], used when the `model` argument of the [`IterativeSFTTrainer`] is provided as a string. > Parameters that control the data preprocessing max_length (`int` or `None`, *optional*, defaults to `None`): Maximum length of the tokenized sequence. Sequences longer than `max_length` are truncated. truncation_mode (`str`, *optional*, defaults to `"keep_end"`): The truncation mode to use, either `"keep_end"` or `"keep_start"`. optimize_device_cache (`bool`, *optional*, defaults to `False`): Whether to optimize accelerator cache for slightly more memory-efficient training. NhelpzvLLM SamplingParams)defaultmetadatavllm_sampling_paramsrJz8Chunk size to reduce memory usage. -1 is most efficient.unsloth_num_chunksz'Maximum sequence length to truncate to.max_seq_lengthc >US:a[SUS35 US:a[SUS35 UcU#S:Xa U$S:XaSnS n#[TU]"S0S U_S U_S U_S U_SU_SU_SU_SU_SU _SU _SU _SU _SU _SU_SU_SU_SU_SU_SU_SU_SU_SU_S U_S!U_S"U_S#U_S$U_S%U_S&U_S'U_S(U_S)U _S*U!_S+U"_S,U#_S-U$_S.U%_S/U&_S0U'_S1U(_S2U)_S3U*_S4U+_S5U,_S6U-_S7U._S8U/_S9U0_S:U1_S;U2_SU5_S?U6_S@U7_SAU8_SBU9_SCU:_SDU;_SEU<_SFU=_SGU>_SHU?_SIW@_SJWA_SKWB_SLWC_SMWD_SNWE_SOWF_SPWG_SQWH_SRWI_SSWJ_STWK_SUWL_SVWM_SWWN_SXWO_SYWP_SZWQ_S[WR_S\WS_S]WT_S^WU_S_WV_S`WW_SaWX_SbWY_ScWZ_SdW[_SeW\_SfW]_SgW^_ShW__SiW`_SjWa_SkWb_SlWc_SmWd_SnWe_SoWf_SpWg_SqWh_SrWi_SsWj_StWk_SuWl_SvWm_SwWn_SxWo_SyWp_SzWq_S{Wr_S|Ws_S}Wt_S~Wu_SWv_SWw_SWx_SWy_SWz_SW{_SW|_SW}_SW~_SW_SW_SW_SW_SW_SW_SW_WD6 WUlWUlWUlg)NgHz>z Unsloth: Your learning rate of `zi` is too small and less than 1e-7! Consider increasing it, otherwise gradient updates will be close to 0!rPza` is way too larger > 1! Consider decreasing it to 1e-1, otherwise gradient updates will explode!stepsunsloth_training_checkpointsno output_diroverwrite_output_dirdo_traindo_eval do_predict eval_strategyprediction_loss_onlyper_device_train_batch_sizeper_device_eval_batch_sizeper_gpu_train_batch_sizeper_gpu_eval_batch_sizegradient_accumulation_stepseval_accumulation_steps eval_delaytorch_empty_cache_steps learning_rate weight_decay adam_beta1 adam_beta2 adam_epsilon max_grad_normnum_train_epochs max_stepslr_scheduler_type warmup_ratio warmup_steps log_levellog_level_replicalog_on_each_node logging_dirlogging_strategylogging_first_step logging_stepslogging_nan_inf_filter save_strategy save_stepssave_total_limitsave_safetensorssave_on_each_nodesave_only_model'restore_callback_states_from_checkpointno_cudause_cpuuse_mps_deviceseed data_seed jit_mode_evaluse_ipexbf16fp16fp16_opt_levelhalf_precision_backendbf16_full_evalfp16_full_evaltf32 local_rank ddp_backend tpu_num_corestpu_metrics_debugdebugdataloader_drop_last eval_stepsdataloader_num_workersdataloader_prefetch_factor past_indexrun_name disable_tqdmremove_unused_columns label_namesload_best_model_at_endmetric_for_best_modelgreater_is_betterignore_data_skipfsdpfsdp_min_num_params fsdp_config"fsdp_transformer_layer_cls_to_wrapaccelerator_configparallelism_config deepspeedlabel_smoothing_factoroptim optim_args adafactorgroup_by_lengthlength_column_name report_toddp_find_unused_parametersddp_bucket_cap_mbddp_broadcast_buffersdataloader_pin_memorydataloader_persistent_workersskip_memory_metricsuse_legacy_prediction_loop push_to_hubresume_from_checkpoint hub_model_id hub_strategy hub_tokenhub_private_repohub_always_push hub_revisiongradient_checkpointinggradient_checkpointing_kwargsinclude_inputs_for_metricseval_do_concat_batches fp16_backendpush_to_hub_model_idpush_to_hub_organizationpush_to_hub_token mp_parametersauto_find_batch_sizefull_determinism torchdynamo ray_scope ddp_timeout torch_compiletorch_compile_backendtorch_compile_modeinclude_tokens_per_secondinclude_num_input_tokens_seenneftune_noise_alphaoptim_target_modulesbatch_eval_metrics eval_on_startuse_liger_kernelliger_kernel_configeval_use_gather_objectaverage_tokens_across_devicesmodel_init_kwargs max_lengthtruncation_modeoptimize_device_cache)printsuper__init__rrr)r7rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r r r r rrrrrrrrrrrrrrrrrrr9 __class__s r<r "UnslothIterativeSFTConfig.__init__sX 4 )I-YB(C"D 1 e&F}oVw%xy  -7":zS?P7J M E D#E D#7E D E D E D $ E D * E D$8E D+FE D*DE D(@E D'>E D+FE D'>E D$E D'>E D *!E D"(#E D$$%E D&$'E D(()E D**+E D, 0-E D."/E D0!21E D2(3E D4(5E D6"7E D8!29E D: 0;E D<&=E D> 0?E D@"4AE DB*CE DD&g?g@rJlinear皙?rpassivewarningTNrFrPFrrNTFFFFFFO r)FFFFO1autoFFNrJNNFFNrNrJNNTNFNNFr,rNNNNN adamw_8bitNFFlengthNNNNTFTFFNN every_saveNNFNTNFTr+NNNr,FFNlastiFNNFFNNFFFNFTNNkeep_endFNrJN)__name__ __module__ __qualname____firstlineno____doc__r,rrr__annotations__rintrr __static_attributes__ __classcell__r!s@r<rrs"F+012+(3-*/VW*#&+EF&NXc] #$&'%&#'"&&'"#"%$%""!&!27!'!$!"%) $!& $  -1!!!$%%) $ $(-"%*!%#!%(,%*!%##' $  $!$)(-"#" "!&(, $ %#SZ-Z-r?rc^\rSrSrSrSS/rS!S\\\4S\ \\ \ 4S\ \ S \ \\ \\\ 44S \ \\\\\4S \\R*R,\R*R.R04S \ \\R4\R4/\R44S \ \\/\44U4SjjjrS\S\ S\4SjrS\R4S\R4S\R44Sjr\S\ \RBS\ \RBS\ \RBS\ \S\ \4 Sj5r"\#RH"5S"S\ \ \RBS\ \ \RBS\ \ \RBS\ \ \S\ \ \4 Sjj5r%Sr&U4Sjr'S#S\ \S\ \S\\\ \S44Sjjr(S r)U=r*$)$_UnslothIterativeSFTTraineria The IterativeSFTTrainer can be used to finetune models with methods that requires some steps between optimization. The [`IterativeSFTTrainer`] is deprecated and will be removed in version 0.24.0. Please use the [`SFTTrainer`]. Args: model (`Union[str, PreTrainedModel]`): Model to be trained. Can be either: - A string, being the *model id* of a pretrained model hosted inside a model repo on huggingface.co, or a path to a *directory* containing model weights saved using [`~transformers.PreTrainedModel.save_pretrained`], e.g., `'./my_model_directory/'`. The model is loaded using [`~transformers.AutoModelForCausalLM.from_pretrained`] with the keyword arguments in `args.model_init_kwargs`. - A [`~transformers.PreTrainedModel`] object. Only causal language models are supported. args ([`IterativeSFTConfig`], *optional*, defaults to `None`): Configuration for this trainer. If `None`, a default configuration is used. data_collator (`DataCollator`, *optional*): Function to use to form a batch from a list of elements of the processed `train_dataset` or `eval_dataset`. Will default to [`~transformers.default_data_collator`] if no `processing_class` is provided, an instance of [`~transformers.DataCollatorWithPadding`] otherwise if the processing_class is a feature extractor or tokenizer. eval_dataset (`datasets.Dataset`): The dataset to use for evaluation. processing_class ([`~transformers.PreTrainedTokenizerBase`], [`~transformers.BaseImageProcessor`], [`~transformers.FeatureExtractionMixin`] or [`~transformers.ProcessorMixin`], *optional*, defaults to `None`): Processing class used to process the data. If `None`, the processing class is loaded from the model's name with [`~transformers.AutoTokenizer.from_pretrained`]. optimizers (`tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]`): The optimizer and scheduler to use for training. preprocess_logits_for_metrics (`Callable[[torch.Tensor, torch.Tensor], torch.Tensor]`): The function to use to preprocess the logits before computing the metrics. compute_metrics (`Callable[[EvalPrediction], dict]`, *optional*): The function to use to compute the metrics. Must take a `EvalPrediction` and return a dictionary string to metric values. trlz iterative-sftNr3r8 data_collator eval_datasetprocessing_class optimizerspreprocess_logits_for_metricscompute_metricsc ~>[R"S[5 [U[5(aUOUR R n Uc#U RS5Sn [U S35nOe[U[5(aP[U[5(d;UR5n URU S'U RS5 [S0U D6nUc[R"U 5nURb+[U[5(d[ R""S5 [U[5(aUR%X5n['5(a[U[(5(aSUlOS UlXPl[/UR S S 5UlUc<UR0(a[3US S S 9UlO [7UR,S S9UlOX0lUR8UlUR:UlUR<Ul[>T U]UUUR4UUUUUS9 [CURDS5(a%URDRGURH5 URKURLRN5 URPRSURDURTURV5uUl"Ul*Ul+UR:S:XaSOSUR,l,[CUS5(d [[S5eUR<[\lg)NzkThe `IterativeSFTTrainer` is deprecated and will be removed in version 0.24.0. Please use the `SFTTrainer`./rJz -IterativeSFTrrzYou passed model_init_kwargs to the `IterativeSFTConfig`, but your model is already instantiated. The `model_init_kwargs` will be ignored.TFis_encoder_decoder)label_pad_token_idpad_to_multiple_of)mlm)r3r8r@rArBrErCrDadd_model_tagsr2leftright acceleratorzXYour `Trainer` does not have an `accelerator` object. Consider upgrading `transformers`.r)/r)warn FutureWarning isinstancestrconfig _name_or_pathsplitrr to_dictrpoprfrom_pretrainedrr%r(_create_model_from_pathr#r is_peft_modelrBgetattrrHrr@rrrrrr r6r3rN _tag_namescreate_optimizer_and_schedulerr8rrQprepare optimizer lr_schedulertruncation_sideAttributeErrorr) r7r3r8r@rArBrCrDrEmodel_id model_name dict_argsr!s r<r $_UnslothIterativeSFTTrainer.__init__s     'uc225 8R8R <!,R0J% M&BCD / 0 0DJ\9]9] I%)^^Ik " MM- .%2 2D  #,<GHiseooc2iNH  " "++*-Y)O)O ##&L)O b""#  NN. 5_cJx H!59N9N9[9[![ \++KNyKijKixss:Kijb""#  ?? &##|3BLBRBRBTUBT$!a#4T__!55BT U  %%3CMCSCSCUVCU41aDOO#3#5!66CU V!#S:aURA5nURC5nTR8REU5 TR8RF(a_TRRHbHTR8RKTRRM5TRRH5 TRNRQ5 TRNRS5 TRTbTRTRQ5 TR=RS- slT=RU- sl TRW5 SSS5 GM gs snf!,(df  GM=f)aP Run an optimisation step given a list of input_ids, attention_mask, and labels or a list of text and text_labels. Args: input_ids (list[`torch.LongTensor`]): List of tensors containing the input_ids (if not provided, text will be used) attention_mask (list[`torch.LongTensor`], , *optional*): List of tensors containing the attention_mask labels (list[`torch.FloatTensor`], *optional*): List of tensors containing the labels (if set to None, will default to input_ids) texts (list[`str`], *optional*): List of strings containing the text input (if not provided, input_ids will directly be used) texts_labels (list[`str`], *optional*): List of strings containing the text labels (if set to None, will default to text) Returns: `dict[str, Any]`: A summary of the training statistics rr-Nz@Step should include `input_ids` or `texts` as keyword arguments.ztBoth `input_ids` and `texts` argument are provided. `input_ids` will be ignored. Please provide only one of the two.zNo 'labels' or 'text_labels' are provided. When using an encoder-decoder architecture, 'labels' or 'text_labels' must be passed.Tpt)r truncationpaddingreturn_tensorsrgrmc>[5nUSHYnUS;dM [R"UVs/sHo3UPM sn5RTRR 5X'M[ U$s snf)Nrrp)dictr(stackrUr3rv)data return_dictkeydr7s r<collator2_UnslothIterativeSFTTrainer.step..collatorsf&KAwCC',{{D3IDqcFD3I'J'M'MdjjN_N_'`K$ 4JsA0 r()rxshuffle collate_fnrP),r3trainstate global_stepr(rrUr8rvtr_loss_globalstep_last_loggedrlr%r(rHrrBrr{rkeysupdater from_dict set_formatrr enumeraterQ accumulate compute_lossn_gpumeandetachbackwardsync_gradientsrclip_grad_norm_ parametersrbstep zero_gradrc_maybe_log_save_evaluate)r7rgrmrnr}r~ model_inputsmodel_inputs_names batch_dictr batch_datastep_dataloader_batchryloss tr_loss_steps` r<r _UnslothIterativeSFTTrainer.steps8  :: ! !Q & <<,// 0@0@ADL+/::+A+AD (  _` `  "u'8 NN6  >l2t7N7NS  %.$9IaLt .<.H*d$0d!-a4*6*B|A AEAZAZ vlB > 6,  00$//dDae1L)5[(AF00FS !,"3"3"56 ,' &&z2 g&$ yy<<  "/2HA!!,,TZZ85GH5G58 5G H((\B99??Q&99;D#{{}   ))$/##22tyy7N7N7Z$$44 --/ // ##%((*$$0%%**, &&!+& , --/7983H98s9O+>O& F O+&O++ O; c\URRbfURRURR-S:Xa5URRS:waUR UR 5 URR GbURRURR -S:XaURRS:wa0nURUR5R5R5nU=RUR-sl[X RRUR- - S5US'UR5US'URRUl URU5 gggg)NrrKrr)r8rrrevaluaterAr_nested_gatherrritemroundr_get_learning_ratelog)r7logstr_loss_scalars r<r4_UnslothIterativeSFTTrainer._maybe_log_save_evaluateEsD 99   +zz%% (<(<<AdjjF\F\`aFa d//0 99 " " .zz%% (?(??1DI_I_cdId)+!%!4!4T\\!B!G!G!I!N!N!P  , $^zz7M7MPTPlPl7l%mopqV (,(?(?(A_%/3zz/E/E,JeD /r?c>URRc*[URR5RnO(URRR S5SnUR US9 [TU]!X5 g)NrGrJ)rg) r8rrrrrXcreate_model_cardr_save_checkpoint)r7r3trialrgr!s r<r,_UnslothIterativeSFTTrainer._save_checkpoint]sj 99 ! ! )dii22388J//55c:2>J *5  .r?rg dataset_nametagsc UR5(dg[URRS5(ac[R R URRR5(d!URRRnOSnUc [5nO$[U[5(aU1nO [U5n[URRS5(aURS5 S[R;aURS5 URUR5 [UUUR UU[#5(a+[$R&b[$R&R(OS[+5SS9nUR-[R R/UR0R2S 55 g) a Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *optional*, defaults to `None`): Name of the model. dataset_name (`str` or `None`, *optional*, defaults to `None`): Name of the dataset used for training. tags (`str`, `list[str]` or `None`, *optional*, defaults to `None`): Tags to be associated with the model card. NrWunsloth_versionunslothJOB_IDhf_jobsz Iterative SFT) base_modelrgrrr wandb_url comet_url trainer_namez README.md)is_world_process_zeror6r3rVr'pathisdirrWsetrTrUaddenvironrr_r!rr$wandbrunurlr"savejoinr8r)r7rgrrr model_cards r<r-_UnslothIterativeSFTTrainer.create_model_cardesN"))++  4::$$o 6 6rww}}TZZM^M^MlMl?m?m**88JJ <5D c " "6Dt9D 4::$$&7 8 8 HHY  rzz ! HHY  DOO$(!!**%'9';'; @Ueiimm[_.0(    TYY%9%9;GHr?) rr@rHr]rcrr3rrbrBrr)NNNN)NNNN)NNNNN)NNN)+r3r4r5r6r7r_r rUrrrr rrrrrrrtupler(r OptimizerrcLambdaLRr rrr r\r{ staticmethodr LongTensorrrempty_device_cacherrrrr:r;r<s@r<r>r>s &P)J HL04EI W imFJbIS/)*bIu/1BBCDbI - bI uWd3<.@%@AB bI # )+=?UWee f bI%++//1I1I1R1RRSbI(0%,, 9UW\WcWc9c0d'ebI"(N+;T+A"BCbIbIHU#U=OUTcU ellELLbgbnbn@3F(()3FU--.3FU%%&3FCy 3F 3i 3F3Fj%%'7;;?37%),0 |0D!1!123|0!e&6&6!78|0e../0 |0 S " |0 tCy) |0(|0|0/%)&*,0 4ISM4Ism4ICcD() 4I4Ir?r>c<^\rSrSrSrSU4SjjrSrU=r$)UnslothIterativeSFTTraineria The IterativeSFTTrainer can be used to finetune models with methods that requires some steps between optimization. The [`IterativeSFTTrainer`] is deprecated and will be removed in version 0.24.0. Please use the [`SFTTrainer`]. Args: model (`Union[str, PreTrainedModel]`): Model to be trained. Can be either: - A string, being the *model id* of a pretrained model hosted inside a model repo on huggingface.co, or a path to a *directory* containing model weights saved using [`~transformers.PreTrainedModel.save_pretrained`], e.g., `'./my_model_directory/'`. The model is loaded using [`~transformers.AutoModelForCausalLM.from_pretrained`] with the keyword arguments in `args.model_init_kwargs`. - A [`~transformers.PreTrainedModel`] object. Only causal language models are supported. args ([`IterativeSFTConfig`], *optional*, defaults to `None`): Configuration for this trainer. If `None`, a default configuration is used. data_collator (`DataCollator`, *optional*): Function to use to form a batch from a list of elements of the processed `train_dataset` or `eval_dataset`. Will default to [`~transformers.default_data_collator`] if no `processing_class` is provided, an instance of [`~transformers.DataCollatorWithPadding`] otherwise if the processing_class is a feature extractor or tokenizer. eval_dataset (`datasets.Dataset`): The dataset to use for evaluation. processing_class ([`~transformers.PreTrainedTokenizerBase`], [`~transformers.BaseImageProcessor`], [`~transformers.FeatureExtractionMixin`] or [`~transformers.ProcessorMixin`], *optional*, defaults to `None`): Processing class used to process the data. If `None`, the processing class is loaded from the model's name with [`~transformers.AutoTokenizer.from_pretrained`]. optimizers (`tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]`): The optimizer and scheduler to use for training. preprocess_logits_for_metrics (`Callable[[torch.Tensor, torch.Tensor], torch.Tensor]`): The function to use to preprocess the logits before computing the metrics. compute_metrics (`Callable[[EvalPrediction], dict]`, *optional*): The function to use to compute the metrics. Must take a `EvalPrediction` and return a dictionary string to metric values. c 8>Uc [5n[USS5n [U 5[LaSn [USS5n [U 5[LaSn Sn [R R SS5S:Hn U (d1[R R SS5S:Xa [S5 S n [R R S S 5n [URS S5=(d [URS S5nUcUR5RnSSK J n U"U5nU[R:HnU (dU(aU (a [S5eU (dU(dU (a [S5eU (a"SUlSUlS[R S'OCU (d<U (d5U S :Xa/UUlU(+UlU(aSOS[R S'[USS5b-[USS5S:XaSUl[USS5cSUl[USS5nUb/US:a)SSKJn [-U5[-S5::a [S5 [USS5S:waL[USS5nUS:Xa!UR.U:aUR.Ul[US S5c UbUUl[US!S5n[U5[LaSn[US"S5n[U5[LaSnUR (aU(aSUlS UlUR"(aU(aS UlSUlU (aSUlSUlOc[R R S S 5S#:XaS UlSUlO0U(d)U(d"UR"UlUR UlSn[95R S$S5bS n[95R S%S5bS nU(aS[R S&'S'[95;a[;US'5(dOD[US'S5n[US'S5nUc'Ub$UR<n[;US'5(aUUlUb![;US(5(aUR?5 S)[95;a [;[@S*5(a S+[@l!S,[95;aU[;US*5(aS+Ul![;US)5(a,[;UR@S*5(aS+UR@l!/nSS-K"J#n U"S.U5 [US/S5[HRJ:Xa(URLS:a[US0S5S:waSUl'S1[95;a![;US(5(aUR?5 [PTU]"S8UUUUUUUS2.UD6 S1[95;a![;US35(aURU5 [;US45(a-URVRY5 [;US45(aU?+[US5S5bURZUR5l-[;US65(aVUR\R^nUn[;US15(a&UUl0URbn[;US15(aM&UUl0[;US75(a.[e[gURhRj5U5Ul5g)9NrFrUNSLOTH_ENABLE_FULL_FINETUNING01UNSLOTH_FORCE_FLOAT32zKUnsloth: Switching to float32 training since model cannot work with float16TUNSLOTH_MIXED_PRECISIONrVdtype torch_dtyper) _get_dtypezuUnsloth: Model is in float16 precision but you want to use bfloat16 precision. Set fp16 to `True` and bf16 to `False`zuUnsloth: Model is in bfloat16 precision but you want to use float16 precision. Set fp16 to `False` and bf16 to `True`rACCELERATE_MIXED_PRECISIONrArrrr&rrP) __version__z4.45.2z**** Unsloth: Please use our fixed gradient_accumulation_steps by updating transformers, TRL and Unsloth! `pip install --upgrade --no-cache-dir --force-reinstall --no-deps unsloth transformers trl unsloth_zoo`rrJrrrbfloat16rErDUNSLOTH_RETURN_LOGITSrr4 tokenizer padding_siderPrB)PatchRLStatisticsiterative_sft_trainer parallel_mode_n_gpur3)r3r8r@rArBrDrEr5neftune_hook_handlerrQrr)6rr^rboolr'rgetrrVget_input_embeddingsrunsloth_zoo.utilsrr(float16 TypeErrorrrrr transformersrr-rrrrrlocalsr6rr4rrunsloth_zoo.logging_utilsrr/NOT_DISTRIBUTEDrrrr r5rremoverrQscaleraccelerator_scalerr3r0rBr!r)r7r3r8r@rArBrDrEr9use_bf16use_fp16 force_float32full_finetuningmixed_precision_dtyperrrga_stepstransformers_versioneval_bszrr_output_logitsmodel_max_seq_lengthargs_max_seq_lengthr other_metricsrr current_modelr!s r<r #UnslothIterativeSFTTrainer.__init__sB < 9 ;4/ > %%x4/ > %%x **..)I3OSVVBJJNN3JC$PTW$W _ ` M " /H) T gt4b m]a8b =%"<"<">"D"D%05!5==('hyJA@B:Bg(9NEDF>F DIDI7;BJJ3 481F)1SDI# DIAHvfBJJ3 4 4 . :wt_^b?cgk?k!(D t\408C$/4!>E  HqL H+,0AA@A 4$ /4 7t%A1EH1}!A!AH!LptqQqQdNmt6=E(J^@H`d`| '7?  t +e^ '7?  t +e^ 99u)<\`dFY 99t)<[`TEX "'D "'D  ZZ^^5y AZ O"&D "'D "&))D "&))D  8<<)4 0 J]aN 25BJJ. / 68 +GDBR4S4S #*52BD#I #*42BD#I "*/C/O!&!5!54!122.D4G  !?!?    &( "wy.'I'Idk9Ka  )'88Za:J:W'55'BRB\B\^l:m:mZaopJpJpW ?1=A 4$ /<3O3O OTXT^T^abTbtXq)Q. fh 75.#A#A      8)'/,I- 817 8 fh 75/#B#B    ! 4. / /  $ $ + + -t2339Q 4. 5 A?C?W?WE & & ( < 4 ' '%%,,F!M-1139 0 - 3 3 -1106M , 4 ! !#$=dnn>R>R$SUYZDJ r?)r)NNNNNN)r3r4r5r6r7r r:r;r<s@r<rrs)(X(,| | r?r addFilterc \rSrSrSrSrSrg)HideLoggingMessageiGcXlgNtext)r7rs r<r HideLoggingMessage.__init__Hsd)r?c<URUR5;$r)r getMessage)r7xs r<filterHideLoggingMessage.filterIsalln)DEr?rN)r3r4r5r6r rr:rr?r<rrGs2Er?rz`use_cache=True`)Lr7r(rtorch.nnnnrFtypingrrrrr r r r !trl.trainer.iterative_sft_trainerr rrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r) dataclassesr+r,packaging.versionr-numpynp contextlibr.r+TransformersDataCollatorForLanguageModelingtransformers.training_argsr/r@typesr0rBtorch_compile_optionscompilerfr9rqrrrr>rr6Filterrrrr?r<r1s0 $IIIw w w w w w w w w w w  (% "$3  4;PRS"|| \\ &,, %    \\ 6ellCELL J- 2J- J-V }I'}I| f !<f P 6;FW^^F  '(:;<  r?