f:i SrSSKJr SSKrSSKJr SSKJr SSKJrJ r J r J r J r J r JrJr SSKJrJrJrJrJrJrJrJ r JrJrJrJrJrJrJrJrJrJ r J r J!r!J"r"J#r#J$r$J%r%JrJ&r&J'r'J(r(JrJ r JrJrJ&r&Jr SSK&r&SSK7 SSK)J*r*J+r+ SS K,J-r- SSKrSSK.r/SS K0J1r1 SSKJr SS K2J3r3J4r5 SS K6J7r7 SSK8r8SS K9J:r: Sr;SSSSSS.r<\Rz"SS\S\RS\?S\?S\R4Sjr@S\RS\RS\?S\?S\R4 SjrAS\RS\?S\R4SjrB\*"S S!\55rC"S"S#\5rD"S$S%\D5rEg)&z; 2025.10.10 2025.10.9 4.56.2 0.23.0 __UNSLOTH_VERSIONING__ )TensorN) functional)AnyListOptionalTupleUnionDictSetCallable)"BaseImageProcessorr DataCollator"DataCollatorForTokenClassificationDatasetEvalPredictionFeatureExtractionMixinr PRMConfig PRMTrainer PartialStatePathPreTrainedModelPreTrainedTokenizerBaseProcessorMixinTrainerTrainerCallbackr chaincompute_accuracydisable_dropout_in_modelfeaturesgenerate_model_cardis_wandb_availablennosprepare_peft_modeltextwraptorchrrrr#r&)*) dataclassfield)Version) nullcontext)DataCollatorForSeq2SeqDataCollatorForLanguageModeling) ParallelMode) MethodTypecF^[R"T5U4Sj5nU$)Nc8>[US5(a5[URS5(aURR5 T"U/UQ70UD6n[US5(a5[URS5(aURR5 U$)Nmodel for_training for_inference)hasattrr2r3r4)selfargskwargsoutputfs >/home/james-whalen/unsloth_compiled_cache/UnslothPRMTrainer.pywrapper*prepare_for_training_mode..wrapper0sx 4 ! !gdjj.&I&I JJ # # %4)$)&) 4 ! !gdjj/&J&J JJ $ $ & ) functoolswraps)r:r<s` r;prepare_for_training_moderA/s%__Q Nr>TF)epilogue_fusion max_autotune shape_paddingz trace.enabledztriton.cudagraphs)dynamic fullgraphoptionscd[R"URSURS5SSS9n[R"URS5SSS9n/n[ X#5HupVUR [R 5n[R"USURS5S9RS5n[R"USS9nXx- n URU 5 M [R"U5nURURSURS45nU$)Nr)chunksdim)rLindexrL) r&chunkreshapeshapeziptofloat32gather unsqueezesqueeze logsumexpappendconcat) logitsrMchunked_logits chunked_indexall_per_token_logps chunk_logits chunk_indexselected_logitslogsumexp_valuesper_token_logpss r;chunked_selective_log_softmaxreEs[[FLL4D!EPQYZ[N[[r!2QaHM%(%G! #u}}5 ,,|2{G\G\]_G`aiijlm ?? input_idslogits_to_keep pad_token_idreturnc~XRS:a [S5eUSS2SU*24nX2:HnURSS9nU$)zr Given prompt tensor, it returns all the left padded tokens in that sequence. so [pad, pad, pad, cat] = 3 tokens rOz8logits_to_keep must be smaller than the sequence length.NrN)rR ValueErrorsum)rfrgrhprompt_section padding_maskpad_token_countss r;calculate_pad_tokens_in_promptrpWsX++STTq"2N?"223N"2L#''A'. r>completion_input_idsleft_pad_tokens_per_prompt max_left_padcURupEURnX!- n[R"XVS9R S5nXR S5:n X:gn X-n U $)a) Given that we have a sequence, [p,p,p,c,c,c,pad,pad,pad] Where p are extra prompt tokens we got from slicing the torch tensor, c is completion tokens and pad are pad tokens, this function would make a completion mask that would 0 out the pad and p tokens. so in this example [0,0,0,1,1,1,0,0,0] )devicerrO)rRrur&arangerW) rqrrrsrh batch_sizecompletion_lenrunum_tokens_to_maskindices shift_masknon_padding_mask final_masks r; create_completion_attention_maskr~jsi"6!;!;J ! ( (F%Bll>9CCAFG88;;J,<.J r>tensorpad_idclX:gn[R"USSSS9n[R"USU5nU$)zD Moves all padding tokens in each sequence of a batch to the right. rOT)rL descendingstable)r&argsortrV)rrmasksorted_indices packed_tensors r;left_pack_paddingrs8  D]]4Q4MNLLN;M r>c^\rSrSr%Sr\"SSS0S9r\\\ S'\"SSS 0S9r \\ \ S '\"SSS 0S9r \\ \ S 'SU4S jjr SrU=r$)UnslothPRMConfiga Configuration class for the [`PRMTrainer`]. This class includes only the parameters that are specific to PRM training. For a full list of training arguments, please refer to the [`~transformers.TrainingArguments`] documentation. Note that default values in this class may differ from those in [`~transformers.TrainingArguments`]. Using [`~transformers.HfArgumentParser`] we can turn this class into [argparse](https://docs.python.org/3/library/argparse#module-argparse) arguments that can be specified on the command line. Parameters: max_length (`int` or `None`, *optional*, defaults to `1024`): Maximum length of the sequences (prompt + completion) used for truncation. max_prompt_length (`int` or `None`, *optional*, defaults to `512`): Maximum length of the prompt used for truncation. max_completion_length (`int` or `None`, *optional*, defaults to `None`): Maximum length of the completion used for truncation. The completion is the concatenation of the steps. disable_dropout (`bool`, *optional*, defaults to `True`): Whether to disable dropout in the model. step_separator (`str`, *optional*, defaults to `" "`): Separator used to separate each step of the reasoning process. train_on_last_step_only (`bool`, *optional*, defaults to `False`): Whether to train only on the last step. dataset_num_proc (`int`, *optional*, defaults to `None`): Number of processes to use for processing the dataset. NhelpzvLLM SamplingParams)defaultmetadatavllm_sampling_paramsrIz8Chunk size to reduce memory usage. -1 is most efficient.unsloth_num_chunksz'Maximum sequence length to truncate to.max_seq_lengthc H>US:a[SUS35 US:a[SUS35 UcU#S:Xa U$S:XaSnS n#Wc$S S KJn [[ U"5S -S 5S5n[ TU]"S0SU_SU_SU_SU_SU_SU_SU_SU_SU _SU _SU _SU _SU _SU_SU_SU_SU_S U_S!U_S"U_S#U_S$U_S%U_S&U_S'U_S(U_S)U_S*U_S+U_S,U_S-U_S.U _S/U!_S0U"_S1U#_S2U$_S3U%_S4U&_S5U'_S6U(_S7U)_S8U*_S9U+_S:U,_S;U-_SU0_S?U1_S@U2_SAU3_SBU4_SCU5_SDU6_SEU7_SFU8_SGU9_SHU:_SIU;_SJU<_SKU=_SLU>_SMU?_SNW@_SOWA_SPWB_SQWC_SRWD_SSWE_STWF_SUWG_SVWH_SWWI_SXWJ_SYWK_SZWL_S[WM_S\WN_S]WO_S^WP_S_WQ_S`WR_SaWS_SbWT_ScWU_SdWV_SeWW_SfWX_SgWY_ShWZ_SiW[_SjW\_SkW]_SlW^_SmW__SnW`_SoWa_SpWb_SqWc_SrWd_SsWe_StWf_SuWg_SvWh_SwWi_SxWj_SyWk_SzWl_S{Wm_S|Wn_S}Wo_S~Wp_SWq_SWr_SWs_SWt_SWu_SWv_SWw_SWx_SWy_SWz_SW{_SW|_SW}_SW~_SW_SW_SW_SW_SW_SW_SW_SW_SW_SW_WD6 WUlWUlWUl g)NgHz>z Unsloth: Your learning rate of `zi` is too small and less than 1e-7! Consider increasing it, otherwise gradient updates will be close to 0!rOza` is way too larger > 1! Consider decreasing it to 1e-1, otherwise gradient updates will explode!stepsunsloth_training_checkpointsnor) cpu_countrJ@ output_diroverwrite_output_dirdo_traindo_eval do_predict eval_strategyprediction_loss_onlyper_device_train_batch_sizeper_device_eval_batch_sizeper_gpu_train_batch_sizeper_gpu_eval_batch_sizegradient_accumulation_stepseval_accumulation_steps eval_delaytorch_empty_cache_steps learning_rate weight_decay adam_beta1 adam_beta2 adam_epsilon max_grad_normnum_train_epochs max_stepslr_scheduler_type warmup_ratio warmup_steps log_levellog_level_replicalog_on_each_node logging_dirlogging_strategylogging_first_step logging_stepslogging_nan_inf_filter save_strategy save_stepssave_total_limitsave_safetensorssave_on_each_nodesave_only_model'restore_callback_states_from_checkpointno_cudause_cpuuse_mps_deviceseed data_seed jit_mode_evaluse_ipexbf16fp16fp16_opt_levelhalf_precision_backendbf16_full_evalfp16_full_evaltf32 local_rank ddp_backend tpu_num_corestpu_metrics_debugdebugdataloader_drop_last eval_stepsdataloader_num_workersdataloader_prefetch_factor past_indexrun_name disable_tqdmremove_unused_columns label_namesload_best_model_at_endmetric_for_best_modelgreater_is_betterignore_data_skipfsdpfsdp_min_num_params fsdp_config"fsdp_transformer_layer_cls_to_wrapaccelerator_configparallelism_config deepspeedlabel_smoothing_factoroptim optim_args adafactorgroup_by_lengthlength_column_name report_toddp_find_unused_parametersddp_bucket_cap_mbddp_broadcast_buffersdataloader_pin_memorydataloader_persistent_workersskip_memory_metricsuse_legacy_prediction_loop push_to_hubresume_from_checkpoint hub_model_id hub_strategy hub_tokenhub_private_repohub_always_push hub_revisiongradient_checkpointinggradient_checkpointing_kwargsinclude_inputs_for_metricseval_do_concat_batches fp16_backendpush_to_hub_model_idpush_to_hub_organizationpush_to_hub_token mp_parametersauto_find_batch_sizefull_determinism torchdynamo ray_scope ddp_timeout torch_compiletorch_compile_backendtorch_compile_modeinclude_tokens_per_secondinclude_num_input_tokens_seenneftune_noise_alphaoptim_target_modulesbatch_eval_metrics eval_on_startuse_liger_kernelliger_kernel_configeval_use_gather_objectaverage_tokens_across_devices max_lengthmax_prompt_lengthmax_completion_lengthdisable_dropoutstep_separatortrain_on_last_step_onlydataset_num_proc) printmultiprocessingrminmaxsuper__init__rrr)r6rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr r r r r rrrrrrrrrrrrrrrrrrr r!rrrr8r __class__s r;r(UnslothPRMConfig.__init__sp` 4 )I-YB(C"D 1 e&F}oVw%xy  -7":zS?P7J M  # 1"3y{1}a#8"=  H :#H :#7H : H : H : $ H : * H :$8H :+FH :*DH :(@H :'>H :+FH :'>H :$H :'>H : *!H :"(#H :$$%H :&$'H :(()H :**+H :, 0-H :."/H :0!21H :2(3H :4(5H :6"7H :8!29H :: 0;H :<&=H :> 0?H :@"4AH :B*CH :D&OH :P 0&QH :R%9!"4,r>)rrr)NNFFFrFrJrJNNrrrg-C6 ?g{Gz?g?g+?g:0yE>g?g@rIlinear皙?rpassivewarningTNrFrOFrrNTFFFFFFO r0FFFFO1autoFFNrINNFFNrNrINNTNFNNFr3rNNNNN adamw_8bitNFFlengthNNNNTFTFFNN every_saveNNFNTNFTr2NNNr3FFNlastiFNNFFNNFFFNFTiiNTr3FNNrIN)__name__ __module__ __qualname____firstlineno____doc__r)rrr__annotations__rintrr(__static_attributes__ __classcell__r)s@r;rrs%:+012+(3-*/VW*#&+EF&NXc] #$&'%&#'"&&'"#"%$%""!&!27!'!$!"%) $!& $  -1!!!$%%) $ $(-"%*!%#!%(,%*!%##' $  $!$)(-"#" "!&(, $"'#[d-d-r>rc4^\rSrSrSrSS/rSS\\\\ R4S\\ S\\ S \\ S \\\ \\\ 44S \\\\\\4S \\/\4S \\\/\4S\\\S\\R4R6\R4R8R:4S\\\R<\R</\R<4S\\4U4Sjjjr\ S5r!U4Sjr"SS\\S\\S\\\\S44Sjjr#Sr$U=r%$)_UnslothPRMTraineria Initialize PRMTrainer. Args: model (`transformers.PreTrainedModel`): The model to train, preferably an `AutoModelForTokenClassification`. args (`PRMConfig`): The arguments to use for training. data_collator (`transformers.DataCollator`): The data collator to use for training. If None is specified, the default data collator (`DataCollatorForTokenClassification`) will be used which will pad the sequences to the maximum length of the sequences in the batch, given a dataset of paired sequences. train_dataset (`datasets.Dataset`): The dataset to use for training. eval_dataset (`datasets.Dataset`): The dataset to use for evaluation. processing_class ([`~transformers.PreTrainedTokenizerBase`], [`~transformers.BaseImageProcessor`], [`~transformers.FeatureExtractionMixin`] or [`~transformers.ProcessorMixin`], *optional*, defaults to `None`): Processing class used to process the data. If provided, will be used to automatically process the inputs for the model, and it will be saved along the model to make it easier to rerun an interrupted training or reuse the fine-tuned model. model_init (`Callable[[], transformers.PreTrainedModel]`): The model initializer to use for training. If None is specified, the default model initializer will be used. compute_metrics (`Callable[[transformers.EvalPrediction], dict]`, *optional* defaults to `compute_accuracy`): The metrics to use for evaluation. If no metrics are specified, the default metric (`compute_accuracy`) will be used. callbacks (`list[transformers.TrainerCallback]`): The callbacks to use for training. optimizers (`tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]`): The optimizer and scheduler to use for training. preprocess_logits_for_metrics (`Callable[[torch.Tensor, torch.Tensor], torch.Tensor]`): The function to use to preprocess the logits before computing the metrics. peft_config (`dict`, defaults to `None`): The PEFT configuration to use for training. If you pass a PEFT configuration, the model will be wrapped in a PEFT model. trlprmNr2r7 data_collator train_dataset eval_datasetprocessing_class model_initcompute_metrics callbacks optimizerspreprocess_logits_for_metrics peft_configc >UR(a [U5 Uc[nUc!Uc [ S5e[ XbR S9nSUR;Ga[5R5 UURUR URURURS.n 0U ESS0EnURURUUR UR"S["R$"["R&"["R("S55["R&"["R("S55S .5S 9n0U ESS 0EnUbURURUUR UR"S ["R$"["R&"["R("S55["R&"["R("S55S .5S 9nSSS5 [*TU]YUUUUUUUUU U U S 9 [/UR0S5(a&UR0R3UR45 gg!,(df  Ng=f)NFz^A processing_class must be specified when using the default DataCollatorForTokenClassification)rrf) tokenizerrrrrr is_evalzTokenizing train datasetint64)labelsrf) fn_kwargsnum_procremove_columnsdescrTzTokenizing eval dataset) r2r7rGrHrIrJrKrLrMrNrOadd_model_tags)r$rrrrkrr column_namesrmain_process_firstrrrr map tokenize_rowr!rFeaturesSequenceValuer'r(r5r2rZ _tag_names)r6r2r7rGrHrIrJrKrLrMrNrOrPrVtrain_fn_kwargseval_fn_kwargsr)s r;r(_UnslothPRMTrainer.__init__s6(     $U +  ".O  ' t??O\k\klM m88 8224!1&*&9&9"&//)-)?)?-1-G-G/3/K/K  #BY"A 5"A - 1 1%%-!22#0#9#93%..&.&7&7w8O&P)1):):8>>';R)S !2 ! "@I!?y$!?+#/#3#3))"0!%!6!6'3'<'<6!)!2!2*2*;*;HNN7->x~~g?V-W" $4 $L55P ''%-!+!*G   4::/ 0 0 JJ % %doo 6 1o54s 5FI I%cU"USSS9SnUSV s/sH o"U SS9SPM n n U(a0U(d)S/[US5S- -[USS 5/-n OUSV s/sHn [U 5PM n n URUSS9n U V s/sHoU -PM n n [X5V V s/sHupS/[U 5S- -U /-PM n n n [ [ U 65n[ [ U 65n UR bUR /U-nUbX*S nUb US UnU S Un X-nS/[U5-U -n Ub US UnU S Un XS .$s sn fs sn fs sn fs sn n f) a Tokenize a row of the dataset. Args: features (`dict[str, str]`): Row of the dataset, should contain the keys `"prompt"`, `"completions"`, and `"labels"`. tokenizer (`PreTrainedTokenizerBase`): Tokenizer used to process the data. step_separator (`str`): Separator between steps in the completion. max_length (`int` or `None`): Maximum length of the sequences (prompt + completion). If `None`, the sequences are not truncated. max_prompt_length (`int` or `None`): Maximum length of the prompt. If `None`, the prompt is not truncated. max_completion_length (`int` or `None`): Maximum length of the completion sequences. If `None`, the completion sequences are not truncated. train_on_last_step_only (`bool`): Whether to train only on the last step. If `True`, the labels are `-100` for all tokens except the last token of the completion. is_eval (`bool`): Whether the function is used to tokenize samples from a training or an evaluation dataset. Used only if `train_on_last_step_only` is set to `True`. Returns: `dict[str, list[int]]`: Tokenized sequences with the keys `"input_ids"`, and `"labels". Example: ```python >>> from transformers import AutoTokenizer >>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B") >>> features = { ... "prompt": "Which number is larger, 9.8 or 9.11?", ... "completions": ["11 is greater than 8.", "Hence, 9.11 > 9.8."], ... "labels": [True, False], ... } >>> PRMTrainer.tokenize_row( ... features, tokenizer, "\n", max_completion_length=None, train_on_last_step_only=False, is_eval=False ... ) {'input_ids': [23085, 1372, 374, 8131, 11, 220, 24, 13, 23, 476, 220, 24, 13, 16, 16, 30, 16, 16, 374, 7046, 1091, 220, 23, 13, 198, 39, 763, 11, 220, 24, 13, 16, 16, 861, 220, 24, 13, 23, 13, 198], 'labels': [-100, -100, -100, -100, -100, -100, -100, -100, 1, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, -100, 0]} ``` promptF)add_special_tokensrf completionsirUrOrIN)rfrU)lenr?encoderSlistr bos_token_id)rrRrrrrr rS prompt_ids completioncompletions_idsrUlabel separator_idscompletion_idsrfs r;r^_UnslothPRMTrainer.tokenize_rowhspx1eL[Y [cdq[r [rZIjU ;K H[r   #7Vs8H#56:;s8HCUVXCY?Z>[[F.6x.@A.@Uc%j.@FA"((E(R HWX* 5XUXXgTpqTp?Pz4&C Oa/0E7:Tpqe_56eVn%  ! ! -#001J>J  (#$6$78J ,+,B-BCN223F/ #j/)F2  !!+:.IKZ(F&99I BYrsE(EE1"E c>URRc*[URR5RnO(URRR S5SnUR US9 [TU]!X5 g)N/rI) model_name) r7rrrnamesplitcreate_model_cardr'_save_checkpoint)r6r2trialrwr)s r;r{#_UnslothPRMTrainer._save_checkpointsj 99 ! ! )dii22388J//55c:2>J *5  .r>rw dataset_nametagsc UR5(dg[URRS5(ac[R R URRR5(d!URRRnOSnUc [5nO$[U[5(aU1nO [U5n[URRS5(aURS5 S[R;aURS5 URUR5 [R "S5n[#UUUR$UU['5(a+[(R*b[(R*R,OSSUS S 9 nUR/[R R1UR2R4S 55 g) a Creates a draft of a model card using the information available to the `Trainer`. Args: model_name (`str` or `None`, *optional*, defaults to `None`): Name of the model. dataset_name (`str` or `None`, *optional*, defaults to `None`): Name of the dataset used for training. tags (`str`, `list[str]` or `None`, *optional*, defaults to `None`): Tags to be associated with the model card. N _name_or_pathunsloth_versionunslothJOB_IDhf_jobsa @article{uesato2022solving, title = {{Solving Math Word Problems With Process- and Outcome-Based Feedback}}, author = {Uesato, Jonathan and Kushman, Nate and Kumar, Ramana and Song, Francis and Siegel, Noah and Wang, Lisa and Creswell, Antonia and Irving, Geoffrey and Higgins, Irina}, year = 2022, journal = {arXiv preprint arXiv:2211.14275} }PRMzBSolving math word problems with process-and outcome-based feedback) base_modelrwrr~r wandb_url trainer_nametrainer_citation paper_titlez README.md)is_world_process_zeror5r2configr#pathisdirrset isinstancestraddenvironupdaterbr%dedentr rr!wandbrunurlsavejoinr7r)r6rwr~rrcitation model_cards r;rz$_UnslothPRMTrainer.create_model_cardse"))++  4::$$o 6 6rww}}TZZM^M^MlMl?m?m**88JJ <5D c " "6Dt9D 4::$$&7 8 8 HHY  rzz ! HHY  DOO$??$ )!!**%'9';'; @Ueiimm[_%\    TYY%9%9;GHr>r") NNNNNNNNN)NNNN)NNN)&r9r:r;r<r=rbrr rr"Modulerrrdictrrr rrr rrlrtupler&r Optimizer lr_schedulerLambdaLRrr( staticmethodr^r{rzr@rArBs@r;rDrDs#JJ>B$(04+/EI >BFJ59W im&*%^7oryy89:^7y!^7 - ^7  ( ^7 uWd3<.@%@AB ^7# )+=?UWee f ^7Xb/&9:;^7"(N+;T+A"BC^7D12^7%++//1I1I1R1RRS^7"(0%,, 9UW\WcWc9c0d'e#^7$d^%^7^7@\:\:~/%)&*,0 >ISM>Ism>ICcD() >I>Ir>rDcF^\rSrSrSrSU4SjjrSrU=r$)UnslothPRMTraineria Initialize PRMTrainer. Args: model (`transformers.PreTrainedModel`): The model to train, preferably an `AutoModelForTokenClassification`. args (`PRMConfig`): The arguments to use for training. data_collator (`transformers.DataCollator`): The data collator to use for training. If None is specified, the default data collator (`DataCollatorForTokenClassification`) will be used which will pad the sequences to the maximum length of the sequences in the batch, given a dataset of paired sequences. train_dataset (`datasets.Dataset`): The dataset to use for training. eval_dataset (`datasets.Dataset`): The dataset to use for evaluation. processing_class ([`~transformers.PreTrainedTokenizerBase`], [`~transformers.BaseImageProcessor`], [`~transformers.FeatureExtractionMixin`] or [`~transformers.ProcessorMixin`], *optional*, defaults to `None`): Processing class used to process the data. If provided, will be used to automatically process the inputs for the model, and it will be saved along the model to make it easier to rerun an interrupted training or reuse the fine-tuned model. model_init (`Callable[[], transformers.PreTrainedModel]`): The model initializer to use for training. If None is specified, the default model initializer will be used. compute_metrics (`Callable[[transformers.EvalPrediction], dict]`, *optional* defaults to `compute_accuracy`): The metrics to use for evaluation. If no metrics are specified, the default metric (`compute_accuracy`) will be used. callbacks (`list[transformers.TrainerCallback]`): The callbacks to use for training. optimizers (`tuple[torch.optim.Optimizer, torch.optim.lr_scheduler.LambdaLR]`): The optimizer and scheduler to use for training. preprocess_logits_for_metrics (`Callable[[torch.Tensor, torch.Tensor], torch.Tensor]`): The function to use to preprocess the logits before computing the metrics. peft_config (`dict`, defaults to `None`): The PEFT configuration to use for training. If you pass a PEFT configuration, the model will be wrapped in a PEFT model. c  0>Uc [5n[USS5n [U 5[LaSn [USS5n[U5[LaSnSn[R R SS5S:HnU(d1[R R SS5S:Xa [S5 S n[R R S S 5n[URS S5=(d [URS S5nUcUR5RnSSK J n U"U5nU[R:HnU(dU(aU (a [S5eU(dU(dU(a [S5eU(a"SUlSUlS[R S'OCU (d<U(d5US :Xa/UUlU(+UlU(aSOS[R S'[USS5b-[USS5S:XaSUl[USS5cSUl[USS5nUb/US:a)SSKJn [-U5[-S5::a [S5 [USS5S:waL[USS5nUS:Xa!UR.U:aUR.Ul[US S5c UbUUl[US!S5n[U5[LaSn[US"S5n[U5[LaSnUR (aU(aSUlS UlUR"(aU(aS UlSUlU(aSUlSUlOc[R R S S 5S#:XaS UlSUlO0U(d)U(d"UR"UlUR UlSn[95R S$S5bS n[95R S%S5bS nU(aS[R S&'S'[95;a[;US'5(dOD[US'S5n[US'S5nUc'Ub$UR<n[;US'5(aUUlUb![;US(5(aUR?5 S)[95;a [;[@S*5(a S+[@l!S,[95;aU[;US*5(aS+Ul![;US)5(a,[;UR@S*5(aS+UR@l!S,[95;aUO[@nSS-K"J#n [IUU5(dx[IU[J5(a(S.URL;a[OUSS/[US0S5S19nO[IU[N5(a%S.URL;a[KU[US0S5S29nOJ[;US35(aSUl([;US45(aS5Ul)[;US65(a S7S 0Ul*[IUU5(dx[;US85(dg[;US)5(aV[IU[J5(a [KUR@[US0S5S29nO![OUR@SS/[US0S5S19n/n SS9K+J,n! U!"S:U 5 [US;S5[ZR\:Xa(UR^S:a[US. U D6 S=[95;a![;US?5(aURg5 [;US@5(a-URhRk5 [;US@5(aU?4[USAS5bURlUR5l6[;USB5(aVURnRpn"Un#[;U#S=5(a&U"U#l9U#Rtn#[;U#S=5(aM&U"U#l9[;USC5(a.[w[yURzR|5U5Ul>g)ENrFrUNSLOTH_ENABLE_FULL_FINETUNING01UNSLOTH_FORCE_FLOAT32zKUnsloth: Switching to float32 training since model cannot work with float16TUNSLOTH_MIXED_PRECISIONrUdtype torch_dtyper) _get_dtypezuUnsloth: Model is in float16 precision but you want to use bfloat16 precision. Set fp16 to `True` and bf16 to `False`zuUnsloth: Model is in bfloat16 precision but you want to use float16 precision. Set fp16 to `False` and bf16 to `True`rACCELERATE_MIXED_PRECISIONrIrrrr-rrO) __version__z4.45.2z**** Unsloth: Please use our fixed gradient_accumulation_steps by updating transformers, TRL and Unsloth! `pip install --upgrade --no-cache-dir --force-reinstall --no-deps unsloth transformers trl unsloth_zoo`rrrrbfloat16rLrOUNSLOTH_RETURN_LOGITSrr3rR padding_siderightrJ)UnslothVisionDataCollatorrUr4pad_to_multiple_of)mlmmlm_probabilityr)rrdataset_text_fieldr3dataset_kwargsskip_prepare_datasetpad)PatchRLStatistics prm_trainer parallel_mode_n_gpur2) r2r7rGrHrIrJrKrLrMrOrPr4neftune_hook_handler acceleratortrainr")?rgetattrtypeboolr#rgetr#rget_input_embeddingsrunsloth_zoo.utilsrr&float16 TypeErrorrrrr transformersrr*rrrrrlocalsr5rr3rRrunsloth_zoo.vision_utilsrrr,r[+TransformersDataCollatorForLanguageModelingrrrunsloth_zoo.logging_utilsrr.NOT_DISTRIBUTEDn_gpurr'r(r4rremoverrscaleraccelerator_scalerr2r/rAr)r)%r6r2r7rGrHrIrJrKrLrMrOrPr8use_bf16use_fp16 force_float32full_finetuningmixed_precision_dtyperrrga_stepstransformers_versioneval_bszrr_output_logitsmodel_max_seq_lengthargs_max_seq_lengthr_UnslothPRMTrainer__tokenizerr other_metricsrr current_modelr)s% r;r(UnslothPRMTrainer.__init__6s < 0 24/ > %%x4/ > %%x **..)I3OSVVBJJNN3JC$PTW$W _ ` M " /H) T gt4b m]a8b =%"<"<">"D"D%05!5==('hyJA@B:Bg(9NEDF>F DIDI7;BJJ3 481F)1SDI# DIAHvfBJJ3 4 4 . :wt_^b?cgk?k!(D t\408C$/4!>E  HqL H+,0AA@A 4$ /4 7t%A1EH1}!A!AH!LptqQqQdNmt6=E(J^@H`d`| '7?  t +e^ '7?  t +e^ 99u)<\`dFY 99t)<[`TEX "'D "'D  ZZ^^5y AZ O"&D "'D "&))D "&))D  8<<)4 0 J]aN 25BJJ. / 68 +GDBR4S4S #*52BD#I #*42BD#I "*/C/O!&!5!54!122.D4G  !?!?    &( "wy.'I'Idk9Ka  )'88Za:J:W'55'BRB\B\^l:m:mZaopJpJpW*<*H&i F-)BCC-)?@@XUbUoUoEo K&))07KT)R ! M+VWW\dhuiCiC]C 6)07KT)R! t455TYt7Qt122bD4Kt-..G]_cFd0C-)BCC;..7; 3T3Tm-CDD$:#---4T;OQU-V%M %P#--#*--4T;OQU-V %M  ?-7 4$ /<3O3O OTXT^T^abTbtXq)Q. fh 75.#A#A      0))'/#-!,I% 0)/ 0 fh 75/#B#B    ! 4. / /  $ $ + + -t2339Q 4. 5 A?C?W?WE & & ( < 4 ' '%%,,F!M-1139 0 - 3 3 -1106M , 4 ! !#$=dnn>R>R$SUYZDJ r>)r) NNNNNNNNNNN)r9r:r;r<r=r(r@rArBs@r;rrs8%P(,e e r>r)Fr=r&rtorch.nnr"rFtypingrrrrr r r r trl.trainer.prm_trainerr rrrrrrrrrrrrrrrrrrr r!r#r$r% dataclassesr(r)packaging.versionr*numpynp contextlibr+rr,r-rtransformers.training_argsr.r?typesr/rAtorch_compile_optionscompilerer?rpr~rrrDrr"r>r;rs0 $IIIxxxxxxxxx (% "$3  4;PRS"|| \\ &,, %    \\ 6ellCELL N-yN- N-^ nInI^ L *L \r>