oi / S Qr SSKrSSKJr SSKJr SSKJr SSKJ r J
r
SSK J r SS K
JrJr S
rSSKrSSKr SS KJr SS KJr SSK r SSK r SS
K J r SSK J!r!J"r"J#r# SSK$r$SSK%J&r&J'r' S r( SPS jr) SSKJ*r*J+r+J,r, SSK-J.r. SSK/J0r0 SSK J1r1 SSK2J2r3 SSK r SSK4r4SSK5r5SSK6r6S r7 S\Rp Rr S\:4S jr; S r< SS K=J>r> SSK r SSK?r@SSKArASSK2J2r3 SS KBJCrC \C " S S 5 5 rD S rE \R SQS! j5 rG SSKrSSKHrHSSKIrISSK-r-SSK r SSKJrJSSKKrK \-R& R rM \R SRS$ j5 r] \R SPS% j5 r^ SS&KJ_r_J`r`Jara S' rb S( rc SSS) jrd S*\ 4S+ jre S, rf S-\g\h S\:4S. jri\R STS/ j5 rj S0\hS1\"\h S2\hS3\#\ S4\#\h S\:4 S5 jrk S6\!\h\ 4 S2\hS7\h4S8 jrl S\#\ 4S9 jrm S:rn SUS; jro SVS< jrp S=\h4S> jrq S? rr S@ rs SWSA jrt SB ru SXSC jrv SD rw SXSE jrx SXSF jry SG rz SH r{ SI r| SJ r} SK r~ SYSL jr SM r SYSN jr SO rg! \\4 a SrSr GN]f = f! SS KJr O
! SS KJr O= f GNi= f! \
R " S"5 \R \R \R \R \R \R \R \R \R \Rt \R \R \R \R \R S#.rM GN"= f)Z)create_huggingface_repomerge_and_dequantize_loramerge_and_overwrite_lora N )get_lora_layer_modules)
_get_dtype)dtype_from_config)UNSLOTH_ENABLE_LOGGINGlogger) defaultdict)convert_moe_packed_tensors convert_moe_packed_tensors_cpuaG ---
base_model: {base_model}
tags:
- text-generation-inference
- transformers
- unsloth
- {model_type}
- {extra}
license: apache-2.0
language:
- en
---
# Uploaded finetuned {method} model
- **Developed by:** {username}
- **License:** apache-2.0
- **Finetuned from model :** {base_model}
This {model_type} model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
[
](https://github.com/unslothai/unsloth)
) get_token)PushToHubMixin)Path)UnionListOptional)PeftModelForCausalLM PeftModelc / n/ nU R 5 H u p4[ U[ R R 5 ( aX [ UR S5 ( a* UR R b UR U5 Ms UR U5 M [ U[ R R 5 ( d M UR U5 M X4$ )N quant_state)
named_modules
isinstancebnbnn
Linear4bithasattrweightr appendtorchLinear)modelskipped_modulesquantized_modulesnamemodules R/home/james-whalen/.local/lib/python3.13/site-packages/unsloth_zoo/saving_utils.py find_skipped_quantized_modulesr) S s O ++- fcff / / 0 0v}}m44 9R9R9^!((. &&t,
0
0 " "4 ( . -- c [ U5 [ L d eUR S5 S:w a [ SU S35 eSSKJnJn Uc
[ 5 nU" US9nUR UUSS 9nUR S5 S nU R R n [ R R U 5 ( a[ [ R R U 5 ( a7 [! U 5 n
U
b' [ R R U
5 ( d U
n OUn [" R% UU U R R& S
S S 9n U" U 5 n U R) XS
S9 U" US9n
XU
4$ )N/r z.Unsloth: You are pushing to Hugging Face, but z is not a valid repo.r ) ModelCardHfApitokenT)repo_idprivateexist_ok unsloth)username
base_model
model_typemethodextrazUnsloth Model Card)r0 commit_message)typestrcount TypeErrorhuggingface_hubr- r. r create_reposplitconfig
_name_or_pathospathexistsisdirget_original_model_id
MODEL_CARDformatr8 push_to_hub)r# r1 r2 r0 r- r. apirepo_urlr6 r7 original_model_idcontentcardhf_apis r( r r a sQ =C }}SQ H Qfghh0 }ike
C H
}}S!!$H ++J ww~~j!!bggmmJ&?&? 1* = ( @Q1R1R*J !J \\,, G W DW>RS
5
!F f $$r* )snapshot_downloadhf_hub_download HfFileSystem) safe_open) save_file) OrderedDict)tqdmc UR b
UR c U $ U R S[ R SS9n U R UR R S[ R SS9UR R S[ R SS9UR S9n [ R " [ R " U 5 5 R 5 ( d [ SU- 5 eU $ )NcudaT)dtype non_blocking)alphaz8Unsloth: Merge failed as there are infinite elements in ) lora_Alora_Btor! float32addmm_r^ isfiniteamaxitem
ValueError)W
lora_statsr& s r( _merge_lorarj s J$5$5$=ax VU]]4@A VU]]4 P VU]]4 P A
>>%**Q- ( - - / /SVZZ[[ Hr* r' returnc & [ U S5 ( d g[ U S5 ( a g[ U R 5 R R S5 ( a gU R n[ U[ R R 5 ( d! [ U[ R 5 ( a ggUR R nUS; a g[ U SS 5 nUR nUR [ R " S5 R :H ng) Nr FW_qTztorchao.)
Params4bit
Int8Paramsstatecpu)r r< r
__module__
startswithr r! r ParameterTensor __class____name__getattrdevice)r' r cls_namer ry is_cpus r( check_if_quantizedr| s 68 $ $Uvu
fmm ' ' 2 2: > >
]]F fehh 0 0 1 1 fell + + ((H33&'40K
]]F
[[ELL / 4 4
4F r* c UR 5 R 5 nU H oBR U S- U- 5 M U$ )N.)
state_dictkeysadd)r& r'
original_keysr keys r( expand_module_keysr s: # # %D&&tczC'78t r* )dequantize_module_weight) dataclassc \ rS rSr% \R
R \S' \R \S' \R \S' \ \S' Sr
g) LoraStats r' r_ r` r^ N) rw rr __qualname____firstlineno__r! r Module__annotations__ru float__static_attributes__r r* r( r r s)
XX__
\\
\\
Nr* r c l [ U S5 ( a U R R OU nUR 5 R 5 n[ 5 nU H nUR
S5 nUR
S5 nUS:w a US U[ S5 - nOUS:w a US U[ S5 - nO [ UR
S5 UR
S5 5 nUS:w a US U XUR
S5 S - nUR U5 M U[ U5 - n [ U 5 S:w a [ S U S
35 eg ) Nr7 .weight.bias.lora_ .base_layerr~ r zUnsloth: Extracted keys = z do not match!) r r7 r# r r setrfindlenmaxr RuntimeError)
r# new_state_dict inner_modelr all_original_keysx where_weight
where_biasj
differences
r( assert_same_keysr s, ,3E<,H,H%""((eK **,113M
wwy) www'
2 1%ClS^&C#Dq
2
1%AjS\&A#Dq
!177=#9 : 7"1''#,- 00A a "S %88J
:! 7
|>RSSr* c [ 5 n[ S U 5 5 n[ R " S 5 nSu pVpx[ 5 n [ 5 n
[ U 5 n U R
5 GHB u pU S:X a M U R S5 ( a3 U
R XLS [ S5 * l
US-
n[ XU 5 GOU R S5 ( a3 U
R XLS [ S5 * l US-
n[ XU 5 GO[ X5 ( aU [ U
S5 ( a U
R S O U
R nU
R" U XL l US-
n[ XU 5 GO=U R S
5 ( aK XU S [ S
5 * l US-
nU R) U 5 U R) U S [ S
5 * 5 OU( dm [+ U
5 ( a] XU l U
R) U S - 5 [- U
S S 5 b U
R) U S
- 5 [ XU 5 U R) U 5 OhSU ; a GM [ X[ 5 5 nU H4 nUR S5 ( a M SU; a U R) U5 M4 M6 U R) U 5 GME XVs=:X a Us=:X a U:X d e eU( a U R/ 5 n[ R0 " 5 nUR3 5 H u n nU R S5 ( a U S [ S5 * n X; aD XL UU S - ' [- XL R& S S 5 b XL R& R4 UU S
- ' Mt X; a U S [ S 5 * nUU; a UU nOOX; a M UUU ' M OS n U( a [7 U U5 UU4$ )Nc 3 * # U H oS v M g7f)r Nr .0r s r( )create_lora_statistics..
s @-?t-?s c [ S S S S5 $ )Nr )r r r* r( (create_lora_statistics.. s 9T4q3Qr* )r r r r r4 z.lora_A.defaultr z.lora_B.defaultactive_adaptersr r r biasr r )r r z.weight.z.base_layer.weight) r tuple collectionsr r find_lora_base_modelr endswithr r r_ r r` r r r active_adapterscalingr^ r' r r| rx r rX itemsr r )r# merge_into_originalreturn_state_dictLinear_LoRA_Layers lora_weights module_count lora_A_count lora_B_count
scaling_count remove_keys keep_keysr r& r' r new_keysr old_state_dictr param lora_names r( create_lora_statisticsr s
01 @-? @@ **+QRL>H;L %K%I&u-K#113 2:x
]],
-
-BH--L 6$5 6 67 8 ? A L t[ 9
]],
-
-BH--L 6$5 6 67 8 ? A L t[ 9
3
3 122 $ 3 3A 68>8M8M
'-~~n'EL $ Q M t[ 9
]]=
)
)>D 2M 2 23 4 ; A L OOD ! OOD!53}#5"5 6 7%+=f+E+E(. % MM$* +vvt,8)--w:W t[ 9 OOD !
x *$>H ||$899!S(#,
OOD ! ] 4^ H< H= HI HI H $//1$002
)//1KD%}} 122 7c"6 7 78#1=1C
4)+,<-44fdCO1=1C1J1J1O1OJtg~." !13y>/ 2 , ( 3E $h $Jt ) 2*
*5*= ##r* zSUnsloth: `safetensors.torch._TYPES` does not exist. Will set to our default version)F64F32F16BF16I64I32I16I8U8BOOLF8_E4M3F8_E5M2U64U32U16c
N U( a; US:X a5 US:w a/ [ ( a [ R " S5 [ XX#XEU5 $ [ R
R
X5 nSn [ U/ US9n
S n S n S n
Sn [ US5 n [ R " U R 5 S[ R S9n [ R U R S5 S5 n[ R " U R U5 5 n
U R# S5 [% US S
S 9 n[' UR) 5 5 n[ UUUS9n
[+ 5 nU GH nUU; a M [, R. R1 5 ( a< [, R. R3 5 [, R. R5 5 U=( a US:H =( a US:H nU( aR UR7 S 5 ( d UR7 S
5 ( a& [ ( a [ R " SU 35 M UnSnUR9 U5 nUR: nUc GM UR7 S5 ( a US [= S5 * OUnU
R? US 5 nUb0 [A US5 ( a URB b [E UUU5 nU S-
n [G XUUUU5 nU( d [I S5 eA[, R. R3 5 GM S S S 5 U RK 5 U RM 5 U RM 5 [, R. R1 5 ( a [, R. R3 5 U U b U RM 5 U b U RM 5 $ $ ! , ( d f N= f! N+= f! $ = f! [N a n[I SU 35 eS nAff = f! U b U RM 5 O! O= fU b U RM 5 f ! f = ff = f)Nmxfp4z\mxfp4 quantized model detected. Using safe rewrite strategy (requires temporary disk space).r model_class_namezr+b)lengthaccess littleptrq frameworkry _blocks_scalesz![DEBUG] Preserving MXFP4 tensor: Fr r_ r z%Failed to write tensor to model file.z Model merge failed with error: )(r
r info _merge_and_overwrite_lora_mxfp4rE rF join'_convert_lora_keys_to_safetensor_formatopenmmapfileno ACCESS_WRITEint
from_bytesreadjsonloadsseekrV listr r r! r[ is_available empty_cache synchronizer
get_tensorr\ r getr r_ rj _write_tensor_direct_torchr flushclose Exception) save_directoryfilenamer output_dtyper base_model_is_quantized
quant_type save_methodfilename_originalr> converted_lora_weights raw_pointermmheader_metadatalength_of_headerfilesafetensor_keysprocessed_mxfp4_keysr
is_save_mxfp4
output_key
action_loggedrh W_original_dtypelora_keyri successes r( _merge_and_overwrite_lorar s :#8[G=S ! ! KKv w / l z
^>
E E
+ K
BOZ,e4
YY{))+a$BSBS
T >>"''!*h? **RWW-=%>?
(dU
Kt "499; /O &M #3& " $'5 &.. ::**,,JJ**,JJ**, 7 lJ'AQJE 5RJZ\fhik{| &)NPP
&&(W 'X
q Lt
:: " " $ $ JJ " " $ >
"
!!# #Y L
KT
B
"
!!#
#s 9B$O
G(N+A;O
N<O+
N95O
<O O
O'O""O''O* *P$/P ?P$ PP$ P P$ P P$c
0
[ R R X5 n[ 5 nSn SS Kn
SS Kn Sn [
U/ US9n
[ USSS9 n[ UR 5 5 n[
UUUS9n
[ 5 nU GH nUU; a M [ R R 5 ( a< [ R R 5 [ R R 5 S nUnSnUR! S5 ( GaD [" c [% S 5 eUS [' S5 * nUS
- nUnUU; a [( R* " S U S U S
35 M UR- U5 UR- U5 nn[ R R 5 ( a< [ R R 5 [ R R 5 [/ UU5 u nnnUS:X aW SSKJn U " UUUS9R5 SS5 R7 5 n[8 ( a [: R< " SU SUS S35 OR[# UUUS9R5 SS5 R7 5 n[8 ( a [: R< " SU SUS S35 UR? U5 UR? U5 U
RA US 5 n U ( aU [C U S5 ( aD U RD b7 [8 ( a [: R< " SU 35 U S-
n [G UU U5 nO$[8 ( a [: R< " SU 35 SnO*UR! S
5 ( a GM UR- U5 nUR! S5 ( a US [' S5 * OUn U
RA U S 5 n Ub< U b9 [C U S5 ( a( U RD b U( d U S-
n [G UU U5 nSnUc GMt [H RJ " S SS 9 n U RL n [ RN " URQ U5 U XRR S 9 A[ RT " U SSSS 9nS S S 5 [ RV " W 5 WUU' [ R R 5 GM" S S S 5 [ RL S :X a+ [Z R\ " 5 [^ R` " S!5 [H RJ " S"U SS#9 n!U!RL n"S S S 5 [c UW"S$S0S%9 [ Rd " U"U5 U $ ! [$ a- [# UUUS9R5 SS5 R7 5 n GNf = f! , ( d f GN!= f! [X a GNf = f! , ( d f N= f! , ( d f N= f! [X a n#[g S&U# S'35 SS K4n$[ RL S :X aj [k S(5 HZ n% U$Rl " U"U5 OY! [n a6 U%S:X a e [^ R` " S)5 [Z R\ " 5 MX f = f OU$Rl " U"U5 [ RV " U"5 S n#A#U $ ! S n#A#U $ = fS n#A#ff = f)*Nr i +r r rq r Fr zYMXFP4 dequantization is required, but `convert_moe_packed_tensors` could not be imported.r zFound mxfp4 tensor z but missing its scales tensor z . Skipping.)r )rows_per_chunkr z%[DEBUG] Using CPU dequantization for with ,z rows per chunkz%[DEBUG] Using GPU dequantization for r_ z:[DEBUG] DEQUANTIZING MXFP4 & MERGING LoRA into Key Group: z&[DEBUG] DEQUANTIZING MXFP4 Key Group: Tr z.pt)suffixdelete)
pickle_modulepickle_protocol) map_locationr weights_onlyntg? .safetensors)r dirr rK metadataz Error renaming temporary file: z . Attempting copy and replace. g ?)8rE rF r rX psutilpickler rV r r r r! r[ r r r r r
ImportErrorr warningswarnr !_choose_mxfp4_processing_strategy transformers.integrations.mxfp4r transpose
contiguousr
r r r r r r_ rj tempfileNamedTemporaryFiler& savera HIGHEST_PROTOCOLloadremoveOSErrorgccollecttimesleeprW replaceprintshutilrangecopy2PermissionError)&r r r r r r r r tensorsr> r" r# limitr r r r r rh r
r base_name
scales_key
blocks_tensor
scales_tensor device_type device_idr r ri r
temp_file
temp_filenametmpfiletemp_filename_safetensorsr r8 attempts& r( r r s ^>mG
E E E
+
$$ G4 tyy{+ "I /"
#u "C** zz&&((
&&(
&&(A J !M ||I&&-5% 'B C C 0#i. 1 & 2
&
_4 MM$7u<[\f[ggr"st /3s/CT__U_E`}
::**,,JJ**,JJ**, :[!=:6 Y
%' 7b : )= #)Aq/**, 21 "KK*OPY{Z`aopq`r sB )C D 3%}^ i1ojjl .- &KI;V\]klm\nn}$~$((-/C/G/G
/S 3 7 7 4 H
'*h"?"?JDUDUDa-- &`aj`k$lmQJEK:z$J-- &LYK$XY $
i((
OOC( 8B7J7J97U7Uz"2C N?3[eH/33HdCJ}!7GJPXA$(My,,E%HI )
144 -}Fd{d{|JJ}5tZ_`
I
-(
#$GJ
JJ " " $a #
H@
ww$
3
$ $NW\ ]ah$+LL!
^ g0Hd;KL
,.?@6 LW ' 7 6 )= #)Aq/**, 7^ IH
m
H GJ
^ ]
/s2PQR 77d? 8!LL!:
V(!V9 :3U1-V0U11V4
V >V
VVVV
V%(
V69
Z5Z9X Z"\11%a((r* c D [ U [ 5 ( a g [ U 5 $ rM )r r rK r s r( get_torch_storage_id_newrW s !Y #A&&r* c ^ SSK Jn Un [ U 5 [ La U " U5 n S n
Su pnU( aj Uc
[ 5 n[ U UUUS9u pn[ R R U5 ( d U
( a" [ R " SS9n
U
R nSn
Uc [ [ U R 5 5 nU[ R [ R" [ R$ [ R& 4; d e[ [ R& 5 [ R( L d e[ R* " / US9R- 5 m[/ U USS9u nn[1 U4S jUR3 5 5 5 n[ R R5 U5 ( d [ R6 " USS
9 [> R@ " U5 u nnn[ US
- 5 nS n UU: a U
( a= U( a UU : a SnO-U" 5 O%US: a U( a SnUn OU" 5 O [; S5 e [ R " SS9nUR n[> R@ " U5 u nnn[ US
- 5 nU( d UU: a U" 5 O@U( a UU: a [; S5 eU( a [C SU S35 Un
UnSn
O [; S5 e XUUUTUUUUXU
X4$ ! [8 a n[; S U S [= U5 35 eS nAff = f)Nr )parse_size_to_int)NNN)r# r1 r2 r0 T)ignore_cleanup_errors)r\ )r r c 3 < ># U H n[ UT5 v M g 7frM )rT )r r rR s r( r !prepare_saving.. s ]I\A.q,??I\s r3 z"Unsloth: Error creating directory z with error = gffffff?c [ S5 e)NzjUnsloth: Failed saving locally - no disk space left. Uploading can work luckily! Use .push_to_hub instead.)r r r* r( raise_upload_works*prepare_saving..raise_upload_works& s
D
r* z#Failed saving - no disk space left!z/Unsloth: Failed uploading - no disk space left.zUnsloth: Saving to zU will fail, but using a temp folder works! Switching to a temp folder then uploading!)"#huggingface_hub.serialization._baserY r< r r r rE rF rH r+ TemporaryDirectoryr& r r rC r! rb float16float64bfloat16r\ tensorrR r sumvaluesrG makedirsr r r= r8
disk_usager7 ) r# r rL max_shard_sizer2 r0 r r low_disk_space_usagemin_size_in_bytes
use_temp_filerY max_shard_size_in_bytesrD r6 r1 rR r r save_sizeerrortotalusedfreer_
try_temp_filetry_save_directoryrR s @r( prepare_savingrx s F , #$C/"3N"C I 0 Hv =)+%$;$ %
!6 77== ( (M 33DQI &^^N M J7H 7V,W\ EMM5==%-- X XY X 5;; ./ .<<L9FFHL 6
1 L* ]IZIZI\]]I 77>>. ) ) p KK4 8 )).9E4 tDy>D
i "+++/((* # +/(.2+(*"#HII !33DQ
* / / " - -.@ AtT49~ti /1C1E
TI - PQ Q
%n%5 6= =
&I /N M DE E 65lj)T= s p !CNCSSabefkblam no o ps (K" "
L ,LL config_pathc < U R 5 ( d S5 e[ U SSS9 n[ R " U5 nS S S 5 SW; a US Og [ U SSS9 n[ R " X!SS9 S S S 5 g ! , ( d f NC= f! , ( d f g = f) NzGiven config does not existrutf-8encodingquantization_configw indent)rG r r r/ dump)ry frC s r( _remove_quantization_configr a s >!>>
k37 3q1
4 & ( )
k37 3q &a(
4
4 3
4 3s A< B
<
B
Bc [ R R US5 n[ R R U5 ( a U b [ U SS 5 nUbp [ USSS9 n[ R " U5 nS S S 5 SW; d US c U R US' [ USSS9 n[ R " XESSS 9 S S S 5 [ USSS9 n[ R " U5 nS S S 5 SW; a US S
:X d US c US [ USSS9 n[ R " XESSS 9 S S S 5 [ R R US 5 n[ R R U5 ( ao [ USSS9 nUR 5 nS S S 5 WR S S
5 nUR SS5 n[ USSS9 nUR U5 S S S 5 g g ! , ( d f GNw= f! , ( d f GND= f! GNL= f! , ( d f GN2= f! , ( d f GN= f! GN
= f! , ( d f N= f! , ( d f g = f! g = f)Nztokenizer_config.json
chat_templater{ r| r} r r F)r ensure_asciir4 config.jsonz"dtype"z
"torch_dtype"z'dtype'z
'torch_dtype')
rE rF r rG rx r r r/ r r r r6 write) tokenizer saved_foldertokenizer_config_pathold_chat_templater new_fconfig_file_pathdatas r( fix_tokenizer_config_jsonr p s GGLL 7N O ww~~+,,1F #I E (
/wG1 ! A H"!+q /A/I)2)@)@Ao&/wG5IIa5I H
+S7CqIIaL D !#?);r)AQEWE_o&+S7Cu !QuE D ww||L-@ ww~~&'' &g>!vvx ?<< ?;D<< ?;D&g>!
? C HG HG
DC DC ?> ?>
s H $G+;.H )G= H
H; H,+H; H).H; < I% I7I% I!I% +
G:5H =
H H H H
H&!H; )
H83H; 8H; ;I
I
I%
I" I% "I% %I) filenamesc [ R " S5 nU Vs/ s H o!R U5 PM nn[ U5 ( d gU Vs/ s H4 oDR S5 UR S5 UR S5 4PM6 nn[ U6 u pgn[
[ U5 5 S:H =( a [
[ U5 5 S:H $ s snf s snf )zMCheck if filenames follow HF sharded naming: model-00001-of-00005.safetensorsz#^(.+?)-(\d+)-of-(\d+)\.safetensors$Fr r r! )recompilematchallgroupzipr r ) r patternr matchesmparsedprefixes_totalss r( is_hf_sharded_safetensorsr s jj?@G)23A}}Q G3 w<< =D
DGqwwqz1771:qwwqz2GF
D v,H s8} " " U [ R R' TS5 5 [ S5 O [A US9RC US
S9n!U! Hi n#U#S R# S5 ( d M U
R% [ R RI U#S 5 S 5 [+ UU#S 5 nUU#S -
nMk U
( d [K S U 35 eUS:w a US:w d e[M UTUS!UUUU S
UU
S"9 u n$mJmInnn%n&n'n(n)n*mn+n n,U
=( d U+n
[O T5 n-S#R' U&RQ 5 5 n.U.RS S$5 U.RS S#5 S%- :H n/U/( aK US&:X aE UR# S'5 ( a/ U " URU S'5 SS9n[ S(U S)35 / n
SnSnGM O Sn0SdUIUJU4S* jjn1 Ub URW TS+9 [Y UT5 US,:X d US-:X Ga [ U[ 5 ( a UR OUn2[ S.5 [ U[Z 5 ( d [ U[ 5 ( d [] S/5 e[_ UR
S0S 5 ( d [G S15 e U2Ra 5 n3[ S25 [c U35 u n4n5[e U45 S: aT [ S4U4 S535 [ U3R
S05 ( d 0 U3R
l3 U4U3R
Rf S6' [ S7T S835 U3RW TS+9 [ S95 [Y UT5 U( a U1" 5 U ( a W*b [ S;5 U*Ri 5 [ S=5 T$ US :X a' URW T5 [k [O T5 S>- S?9 O.US&:X a( SS@K6J7n6 U6Rq WS SSA9n7U7RW T5 [Y UT5 U( a U1" 5 SS K9n8SB[u [v Rx R{ S5 5 ; n9US :X a
[} U9UU
5 OSn:[ 5 n;Sn[e U
5 S%: d U>( a S/O/ n?W( a WS&:X a U( a US&:X a US&:X a U:( d W( a [ R: " TS
S9 U?( a{ [ R R' WS5 n@[ R R U@5 ( a7 [< R> " W@[ R R' TS5 5 O%O$SU? V s/ s H n U PM sn ; a
[ WTS/SSD9 U( a U?( a U1" S5 W( d U;b [ WU
TU;USE9n<[ US/TU;USE9n=U<( d, U ( d% U( d [ SFW S835 [ UTU?U
- SSD9 U=( d* U ( d# U( d [ SGW S835 [ UTS/SSD9 / nA[ U
SHSI9 GH' nB[ R R' TUB5 nU( a [ R R U5 ( dk [ R R' WWB5 nC[ R R UC5 ( a& [< R> " WCU5 [ SJWB SK35 OEU<( d> U ( a7 [ R R U5 ( d U( d [ WWBSLTSM9 U:( a [ WBTSNSO9nDOWB/nDWAR WD5 GM* U ( ak U( dd U<( d] [ R R' TS5 n [ R R U 5 ( d [ WSSLTUSP9 [ SQ5 U:( a [ WAT5 nA[ WA5 nEU=( a WS&:H =( d U:=( a# [e WA5 S%: =( d WE=( a US&:g nF0 nG[ WASSUS&:X a S&OST 3SI9 H nBU0[ TUBW&U[ U5 R R UWUSU9-
n0[v Rx R 5 [ R R' TUB5 nWF( a0 [ USVSWSX9 n U RQ 5 H nHWBWGUH' M
S S S 5 U ( aC U( a< U1" WB5 [ R " [ R R' TUB5 5 M WF( ai [ SY5 0 WGSZ.n [ R R' TS5 n[- US[SS9 n [. R " U U S\S]9 S S S 5 U( a U1" S5 U ( d U( a U
H nBU1" UB5 M
U1" 5 [e W&5 U0:w a [K S^[e U&5 S_U0 S`35 e W*b U*Ri 5 U( a; [ R R T5 ( a [< R " T5 [ Sc[ R R T5 S)35 T$ ! UR
R n G
NK= f! , ( d f G
N= f! [8 a n [ SU 35 S n A G NS n A ff = f! [E U5 n"U"nU"c [G SU S 35 e[A US9RC US
S9n! G
N= f! G N;= f! [8 a n [K S3U 35 eS n A ff = f! [8 a n [K S:U 35 eS n A ff = f! [8 a n [ S TSSS.nU c TR " SST0UD6 g TR " S[ R R TU 5 U S.UD6 g )Nr# z(Trained with Unsloth))r1 repo_typer; folder_path)path_or_fileobj path_in_repor )
upload_folder upload_filerE rF r )r extrasrR r1 r s r( upload_items.merge_and_overwrite_lora..upload_itemsK sj %WQil I~ I I
"$'',,~x"H '
r* r merged_4bitforced_merged_4bitz0Unsloth: Merging LoRA weights into 4bit model...zIModel must be a PeftModelForCausalLM or PeftModel for 'merged_4bit' save.r z@Model does not appear to be quantized. Cannot use 'merged_4bit'.zUnsloth: Merging finished.z-Failed to merge LoRA weights for 4-bit save: z Unsloth: Found skipped modules: z. Updating config.llm_int8_skip_modulesz%Unsloth: Saving merged 4bit model to z...z!Unsloth: Merged 4bit model saved.z#Failed to save merged 4-bit model: z&Unsloth: Cleaning up temporary file...z&Warning: Failed to cleanup temp file: z-Unsloth: Merged 4bit model process completed.r )ry )
AutoConfig)r0 trust_remote_codezTesla T4z,Error copying model.safetensors.index.json: )r1 local_dirallow_patternslocal_dir_use_symlinks)r1 filenames_to_checktarget_dir_str hf_cache_dirr0 z Downloading safetensors for z+Attempting to download tokenizer.model for z)Unsloth: Preparing safetensor model filesdesczCopied z from local model directoryr# )r1 r r r g ?)max_shard_size_gb)r1 r r r r0 zDownloaded tokenizer.modelzINote: tokenizer.model not found (this is OK for non-SentencePiece models)z Unsloth: Merging weights into 16bit)r r r r r r r r r rq r zFUnsloth: Regenerating safetensors index for dequantized MXFP4 model...)r r r r r z8Unsloth: Saving LoRA finetune failed since # of LoRAs = z% does not match # of saved modules = z. Please file a bug report!z.Warning: Failed to remove temporary directory : z+Unsloth: Merge process complete. Saved to `rM )Sr r r7 r# r rC r9 r% r& rD determine_base_model_sourcerE rF rG rH r7 listdirr r r getsizer r r r/ r ri r r rj r8 r: rU lsrI rg rB r rx r r r> removesuffixsave_pretrainedr r r? rx merge_and_unloadr) r r cleanupr transformersr from_pretrained
subprocessr= r! r[ get_device_nameshould_split_shards_get_hf_cache_dirr
SameFileErrorrS _try_copy_all_from_cache ProgressBarrT split_safetensor_fileextendrenumber_safetensor_filesr r rv rw r rV r0 r rmtreeabspath)Kget_model_namer# r r rL r2 r0 r r rm ro cleanup_temp_filer safetensors_listmax_size_in_bytestotal_size_in_bytesrC loop_iteration
model_namefinal_model_name
is_local_path source_infor r r file_path file_size
index_pathr
index_data
indexed_filesr tokenizer_model_path file_listrO r r6 rR r r rq ru rD new_use_temp_filerp _save_dir_path
all_lora_keysonly_attention_lorasn_saved_modulesr r7 merged_modelr$ r r model_configr is_t4needs_splitting
_hf_cache_dircopied_all_from_cache!copied_tokenizer_model_from_cache
is_hf_shardedsafe_tensor_index_fileslocal_index_pathfinal_safetensors_listr local_file_pathresulting_files!is_final_safetensors_list_shardedregenerate_indexr r rR r1 sK ` @@r( r r s " -7ui,H,H%""((eK,3E<,H,H+((kK
\\F (%++ MM Y
Z Q
8 +ELL,F,FW\ ]
\w yC EJ ]KY-6Mz "
e(;zU?RXcguXu MM T
U # MMF:,.RS T%
77>>* % %"''--
*C*C M 4ZLA B
:.== 00$++D1 " Z >I " :I(+,=y(I%'94'
/ j2PQJww~~j))I j#'Ba%)YYq\
':5,/
<0H0O0O0Q,RM#337
3F 0,!B!)^< m#{6J'J)3E9)E)EU%%5
@B%!566z%QZ?[?[ hi
iu||%:DAA `a
a T%668L . 0
n$ ~.#$~2F 2VW +!11 % 2
$$^4
i 8 #ejj 8 8 ;< ?MCFGWCX[\C\`m = >tv %w)>D[`jnu`u {F JQ {Q \k KK4 8&#%77<<
>"233 %5rww||NTr7st .=T1U=T!=T1UU! ( .&D%E-2 2 7 8 ] 6 8 !1 +( !
-E 1 2)& -
) !)=m
,ZL<= & 47G G%*
-5IR_
;JM N4 + '' 3E : D D M M&=#%
GGLL : 9$G1 668C&.JsO $ H K " IIbggll>8< = 3 N4
VW"$J?
WW\\.2PQ
*cg
6! IIj!a 0 7 7 8 K )H " ) <O+FsBRRTUVTWX Y Y Zs! %w0 x (Bx>4x2x y +zz 3z, <{ 5{2 |*%|/ 6 },}$.}6 +}> 0x
x x x
y*x>>y8zz
z)z$$z),
{ 6{{
{/{**{/2|' |'|""|'/
}9 }
}
}! $
}36};>
~$~ ~$r1 r r r r0 c & SSK Jn U( a U( d [ S5 g[ U5 n[ S5 0 nSnU H n [ X SS9n
[ U
5 Xy' M U( d [ S5 g [ R " USS9 [ R " U[ R [ R - 5 ( d [ SU 35 e Sn
[ UR 5 S[ U5 SU S3S9 H u p [! XU 5 M U
( a [ S[ U5 SU S35 g[ S5 g! U a [ SU S 35 Sn M [ a! n [ S
U S U S 35 Sn S
n A GM# S
n A ff = f! [ a n [ SU S U 35 S
n A gS
n A ff = f! [" [ [$ 4 a% n[ SU SU SU S U 35 Sn
S
nA M S
nAf[ a n[ SU S U 35 Sn
S
nA M S
nAff = f) z
Checks if ALL specified files exist in the HF cache. If yes, creates the
target_dir_str and copies ALL files into it using os functions.
Returns True if successful, False otherwise.
r )LocalEntryNotFoundErrorz?Skipping cache check: No cache directory or no files specified.Fz.Checking cache directory for required files...T)r1 r local_files_onlyzCache check failed: z not found in local cache.z'Cache check failed: Error checking for r r~ NzENot all required files found in cache. Will proceed with downloading.r] z2No write/execute permission for target directory: z?Cache copy failed: Could not create or access target directory zUnsloth: Copying z files from cache to `r r z!Cache copy failed: Error copying z from to z8Cache copy failed: An unexpected error occurred copying zSuccessfully copied all zKFailed to copy one or more files from cache. Will proceed with downloading.)huggingface_hub.errorsr r7 r= rT r r rE rj r W_OKX_OKr; r r r _copy_file_from_sourceIOErrorFileNotFoundError)r1 r r r r0 r hf_cache_dir_strcached_paths_map all_foundr cached_path_str check_errdir_err
all_copied cached_pathcopy_errr s r( r r o sW ? 1
OP<( :<I&
-im nO)-o)> & '
UV
Nt4yy 277):;;"%WXfWg#hi
i < J!,-=-C-C-EPabefxbyaz {Q R` Qa ab Oc "d & "; I "d
(-?)@(AAWXfWgghij
[\Q ' (
2LM N I ;H:R {RST U I
OP^O__abiajkl *;< '
6xj{mSWXfWggijrist
u Z & LXJVXYZX[\ ] J &s[ D?0A F ? F/?F F#E??F
F,F''F,/HG$$
H1H Hsrc_pathr c [ U 5 n [ R R X5 nU R 5 ( d [ SU S35 e[ R " U [ R 5 ( d [ SU 35 e [ R " [ U 5 U5 g! [ a n[ SU SU SU 35 UeSnAff = f)zECopies a file from src_path to target_dir_str/filename using os.path.zSource z is not a valid file.z$No read permission for source file: zFailed to copy r r N)r rE rF r is_filer r R_OKr; r8 r: r= r r )r r r dst_pathr s r( r r s H~Hww||N5H '(3H IJJ
99Xrww ' ' !EhZ PQ QN S]H- NzhZr!EFAMNs B) )
C
3CC
c n / n S[ R ; a+ U R [ [ R S 5 5 S[ R ; a. U R [ [ R S 5 S- 5 U R [ R " 5 S- S- S- 5 U H n UR 5 ( a [ R " U[ R [ R - [ R - 5 ( a. [ SUR 5 35 UR 5 s $ [ SU S35 g UR 5 ( a [ S
U S 35 g M [ S5 g ! [ a n[ S U S
U S35 S nAM S nAff = f)z0Determines the Hugging Face Hub cache directory. HF_HUB_CACHEHF_HOMEhubz.cache huggingfacez'Found HuggingFace hub cache directory: z Warning: Found cache directory z/, but lack R/W/X permissions. Cannot use cache.NzWarning: Path z1 exists but is not a directory. Cannot use cache.z.Warning: Error accessing potential cache path r z. Checking next option.z>No existing and accessible Hugging Face cache directory found.)rE environr r homeis_dirr r r
r r7 resolverG r )potential_paths cache_dirr s r( r r sk O# tBJJ~$> ?@BJJ tBJJy$9 :U BC499;1MAEIJ$ !! 99Y"''(9BGG(CDD CIDUDUDWCXYZ$,,.. ;I;Fuvw !!## {2cde $ %2
JK B9+RPQsRij k s% 7B F
F
$F
F4F//F4a
PushToHubMixin._upload_modified_files(
PushToHubMixin,
working_dir = save_directory,
repo_id = '{repo_id}',
files_timestamps = files_timestamps,
commit_message = "Upload Unsloth finetuned model",
token = token,
create_pr = False,
revision = {revision},
commit_description = "Upload Unsloth finetuned model",
)
if {use_temp_file} and temp_file is not None: temp_file.cleanup()
else:
shutil.rmtree(save_directory)
os.makedirs(save_directory, exist_ok = True)
if {use_temp_file}:
temp_file = tempfile.TemporaryDirectory(ignore_cleanup_errors = True)
save_directory = temp_file.name
files_timestamps = PushToHubMixin._get_files_timestamps(PushToHubMixin, save_directory)
c h [ R " SU 5 nUc eUR S5 nU S US S-
S- XS S - n SU ; a [ S5 e[ R " SU [ R [ R
- S9nUc eUR S5 nU [
US US S -
5 US S-
n[ R " S
US S S 2 5 R S5 S nXS US U-
S-
n[ [ R " S U5 S 5 n UR S
5 S- n
UR 5 nU( aa US U
XzS - SU - - [ R " SS[ R UUUS9R 5 5 R S
S
SU - - 5 - n OUn U S
- SU - - S- SU S- - - S- n U( a4 U S
- SU S-
- - SU S3- SU - - S- SU - - S- SU - - SU S3- n U R X{5 n U( d U R SSS5 n U $ ) Nz!os\.makedirs\(save_directory.+?\nr r zS; files_timestamps = self._get_files_timestamps(save_directory); temp_file = None;
z.for shard_file, tensors in filename_to_tensorszHUnsloth: Failed to find `for shard_file, tensors in filename_to_tensors`zPfor shard_file, tensors in filename_to_tensors\:.*?[\n]{1,}[ ]{4}[a-zA-Z0-9\_\#])flagsr z[\n]{1,}r z \n([ ]{4,})r z[ ]{8,}r4 )r1 revisionro zfor tensor in shard:
r z4if tensor in DEQUANTIZED_KEYS: shard[tensor] = None
zif z:
zFtemp_file = tempfile.TemporaryDirectory(ignore_cleanup_errors = True)
z save_directory = temp_file.name
z repo_id = 'z'
zfor shard_file, tensors in ProgressBar(filename_to_tensors, desc = 'Unsloth: Saving ' + str(len(filename_to_tensors)) + ' safetensor(s)'))r searchspanr DOTALL MULTILINEr r findallfindrstripsub
_PUSHING_CODErK r6 ) r rm ro r1 r. makedirr0 for_loopwherespaces
first_newline new_for_loops r( incremental_save_pretrainedr> s ii 2 7 7 :1 =E Awa q 9H
NH5a8
9F MM$'!+M H / ^ $% J
FF:r ''!("*'4 ( fh
gdD3v: -. /
F
!! VAX
@ @L # N - $%
L
V V
L
00
L '# & ' %--hEO )11 <
X
r* c H ^! [ U [ 5 ( a U R R OU n
[ U S5 ( a U
R OU
n
[ U UUUUUU U
SSU S9 u pnnn nnnnnnp+n
nSS Kn[ R " UR R R 5 nUR S5 m!UR S5 nSR U!4S jU 5 5 nU( a
[ UU
U UUS 9n [! UR 5 n[# S
S R U5 S 3[% 5 [' 5 5 S
[) U 5 SU SU S3nUR S5 n U S:X a [+ S5 eUR S U 5 S- n US U U- UU S - nSU; a [+ S5 eUR- SSS5 nSU; a [+ S5 eUR- SSS5 n0 n[# U[' 5 U5 US n [. R0 " U 5 n [2 R4 " [2 U5 n U " U
4USUUUUUS .U D6 Ub UR US 9 U( a0 [2 R6 " [2 UUU S USUS S 9 n [9 S U 35 U $ Ub UR; 5 g g ! g = f)!Nr7 Fra r r defr c 3 , ># U H oTS v M g 7frM r )r r r; s r( r ,merge_and_dequantize_lora.. s DOq&'
Os )r rm ro r1 r. z)from transformers.modeling_utils import (z, )a:
DEQUANTIZED_KEYS = []
def merge_lora_weights(state_dict, name):
x = state_dict[name]
if type(x) is LoraStats:
DEQUANTIZED_KEYS.append(name)
W = dequantize_module_weight(x.module)
W = _merge_lora(W, x, name)
x = W.to(device = 'cpu', dtype = z, non_blocking = True)
# Remove memory leak
state_dict[name] = None
return x
pass
state_dict_split = split_state_dict_into_shards_factory(
state_dict,
max_shard_size = z,
filename_pattern = filename_pattern,
get_storage_size = functools.partial(get_torch_storage_size_new, element_size = zB),
get_storage_id = get_torch_storage_id_new,
)
z5state_dict_split = split_torch_state_dict_into_shardsr z*Unsloth: Failed to find `state_dict_split`r z state_dict[tensor].contiguous()z9Unsloth: Failed to find `state_dict[tensor].contiguous()`z3merge_lora_weights(state_dict, tensor).contiguous()zdef save_pretrainedz-Unsloth: Failed to find `def save_pretrained`z def save_pretrained_dequantizedsave_pretrained_dequantized)r rL rl safe_serializationr0 r2 r r z Upload Unsloth finetuned model) working_dirr1 files_timestampsr; r0 create_prr. commit_descriptionz2Unsloth: Uploaded model to https://huggingface.co/) r r r7 r# r rx transformers.modeling_utilsinspect getsourcemodeling_utilsPreTrainedModelr r4 rB r r> r execlocalsglobalsr= r r6 r! inference_moder _get_files_timestamps_upload_modified_filesr7 r )"r# r r rL rl rE r0 r2 r. r rm ro kwargsr r6 r1 rR rR r r rq ru rD rp r r functionsreplace_state_dictleftrightrD rG commitr; s" @r( r r W s1 " -7u>R,S,S%""((Y^K,3E<,H,H+((kK '!'# 3 # '%
65lj)T> 5 ' '' (C(C(S(S(c(cdO
!
!%
(F%++D1Oii DO DDO 5 -#7 )
L//0I 4TYYy5I4J! LfhX_Xab . /2,.?-@ A 44 5YYeXf g%,
!X YD rz &RSS d +a /E%et,/AAOTYTZD[[O(?VWW%--)= O O3JKK%-- ) O I ')Y/"+,I"J "'"6"67R"S %;;
+ " 4 / " $ '
i 7 7 7 Y 66 ( / = !A
B7)LM
s J J!
local_pathc SS K nSS KnUR R U S5 nUR R U5 ( a1 [ USSS9 nUR " U5 nS S S 5 SW; a US $ UR R U S5 nUR R U5 ( a1 [ USSS9 nUR " U5 nS S S 5 SW; a US $ g ! , ( d f N= f! , ( d f N+= f) Nr r r{ r| r} rD zadapter_config.jsonbase_model_name_or_path)r rE rF r rG r r/ )r[ r rE ry r rC s r( rI rI s
'',,z=9K ww~~k""
+sw
71YYq\F 8
f $/* *'',,z+@AK ww~~k""
+sw
71YYq\F 8 % .34 4 ! 8
7 8
7s C$=C5$
C25
Dc z [ SU /S9n[ X5 n[ US0 5 $ ! [ [ 4 a 0 s $ f = f)z@Get the checkpoint conversion mapping for a specific model classr )fromlist _checkpoint_conversion_mapping)
__import__rx r$ AttributeError)r r' model_classs r( "_get_checkpoint_conversion_mappingrd sJ N6F5GH f7 {$DbII ( s "% ::c H ^
U( d gSnSnUR 5 Vs/ s H n[ R " U5 PM nnUR 5 Vs/ s H0 n[ R " S[ R " U5 - 5 PM2 nnU H` m
[ T
[ 5 ( d M [ U
4S jU 5 5 n[ U
4S jU 5 5 n U( a US-
nMR U ( d M[ US-
nMb US: a US:X a gUS: a US:X a gX2: a gX#: a ggs snf s snf )Nnewr ^c 3 D ># U H oR T5 v M g 7frM r r r{ r s r( r %detect_keys_format..$ C0B1''#,,0B c 3 D ># U H oR T5 v M g 7frM ri rj s r( r rk % rl rm r old)r r r ri escaper r= any)
keys_to_checkforward_mappingcount_matches_old_patterncount_matches_new_patternpold_regex_compiledvalnew_regex_compiled matched_new matched_oldr s @r( detect_keys_formatr| s ! ! 2A1E1E1GH1GA"**Q-1GHGVG]G]G_`G_"**TBIIcN%:;G_` #s##X C0BCC C0BCC % * %
[ % * % !1$)Ba)GPU 1$)Ba)GPU !,>,@( .44BL*> ? #7 */' "e +0@E0I )8(=(=(?$ ff%5uk>P>PQT>U V &(gggD\&]#) q=/7,6:3
)@ %
-2Be2K(7(=(=(?$&(gggD\&]#) q=/7,6:3 )@ + $ ) $BL&>?? -A@ )(r* c ~ U n[ US5 ( a UR n[ US5 ( a UR nU$ )Nr7 r# )r r7 r# )model_to_inspectcurrents r( r r r s: Gw %%$$w -- Nr* c d [ US9R U SS9n[ S U 5 5 $ ! g= f)z$Check if model exists on HuggingFacer/ Tr c 3 H # U H oS R S5 v M g7f)r& r N)r r s r( r (check_hf_model_exists.. s Iy!V9%%n55ys "F)rU r rq )r r0 r s r( check_hf_model_existsr { s< u-00D0I IyIIIs (+ /c S nS n/ nUR U 5 U" U 5 nU( a UR U5 SU ; ay U R S5 S nUR U5 [ R " S5 H8 nUR 5 UR 5 :X d M' UR U5 O [ 5 n/ nU H5 n U ( d M X; d M UR U 5 UR U 5 M7 U H1 n U" U 5 ( d M [ R R U 5 s $ g! [
[ 4 a Nf = f)zx
Check if model exists locally with case insensitive naming patterns.
Returns the actual path if found, None otherwise.
c . [ R R U 5 ( a$ [ R R U 5 ( d g [ R " U 5 H nUR S5 ( d M g g! [ [ 4 a gf = f)z-Check if directory contains safetensors filesFr T)rE rF rG rH r r r1 r; ) directoryr s r( has_safetensors1check_local_model_exists..has_safetensors sp ww~~i((
i0H0H
9-== 00 . ) s -B <B >B BBc [ R R U 5 ( a U $ U R [ R 5 nSn[ U5 GHO u p4US:X a$ US:X a [ R nM$ US:X a SnO"UnO [ R R
X$5 n[ R R U5 ( a Mu US: a [ R R U5 OSn[ R R U5 R 5 n[ R R U5 ( d g Sn[ R " U5 H: nUR 5 U:X d M [ R R
XX5 nSn O U( d gGMR [ R R U5 ( a U$ S$ ! [ [ 4 a gf = f)z+Find a path that matches case-insensitivelyr4 r r~ NFT)
rE rF rG rB sep enumerater dirnamebasenamelowerr r1 r; ) target_pathparts current_pathipart parent_path target_name found_matchrf s r( find_case_insensitive_path.find_case_insensitive_path sw
77>>+ & & !!"&&) 'GAAv2:#%66L S[#&L#'L !ww||L ? ww~~l++ <=q5"''//,7cK ''**<8>>@K77>>+..
# JJ{ 3Dzz|{2')ww||K'F &* 4 # #A (J "ww~~l ; ;|EE _-
s -G4,GG#"G#r, r r~ N) r rB rE r r r1 r; r r rF r )
model_pathr r paths_to_checkcase_insensitive_fullr rf seen unique_pathsrF s
r( check_local_model_existsr s5
.Fb N *% 7z B 34 j %%c*2.
j)
3::<:#3#3#55"))$/ ( 5DL 4D$ HHTN % 4 77??4( ( ! ) s &9D8 #D8 6D8 8E
E c d Sn[ R R U 5 ( a [ R R U 5 ( aq [ R R U S5 n[ R R U5 ( a, [ USSS9 n[ R " U5 nSSS5 O;O: SSKJ n U" U SSUS9n[ USSS9 n[ R " U5 nSSS5 U( a S U; a~ US n[ U[ 5 ( a UR S
5 S :X a g [ U[ 5 ( a9 UR S
S5 nUR SS5 nU( a SU( a U4$ S4$ g! , ( d f N= f! N= f! , ( d f N= f! N= f)z;Check if a model is quantized (works for both HF and local)Nr r{ r| r} r )rT )r1 r r* r0 r quant_methodr )Tr r Fbnb_4bit_quant_typeT bitsandbytesFN)
rE rF rG rH r r r r/ r@ rT r dictr ) model_name_or_pathr0 rC ry r rT quant_config is_quantizedr s r( check_model_quantization_statusr sv
F ww~~())bggmm>+ & &
+sw?1 !YYq\F @? ' 7),( K k39Q 1 :
'61 34 lD ) )l.>.>~.NRY.Y" d
+
+'++NEBL%))*?FJ JjKKNKK G @?
:9 sT F F2F =F+ F.F+
F F F F
F($F+ (F+ +F/c z [ X5 n[ U 5 nSu pESu pgU( a
[ X5 u pEU( a
[ U5 u pgU( a U( d USSSS4$ U( a U( a
US:X a USSSS4$ U( a U( d U SSSS4$ U( a U( a
U SS U 3SU4$ U( a U( a
USS
U 3SU4$ g ) z
Determine the best source for base model using branched logic
Returns: (final_model_name, is_local_path, source_info, is_quantized, quant_type)
)NNTlocal_unquantizedFNr local_mxfp4HF_unquantizedHF_local_)NFr4 FN)r r r )r r0 hf_existsr[ hf_is_quantized
hf_quant_typelocal_is_quantizedlocal_quant_types r( r r s &j8I)*5J &0"O+5()H)[&/Nz/Z, ,D"5udCC (-=-HD-w?? E#3UDAA _ES #8$
NN (DF+;*<"=tEUVV *r* c
> 0 n SSK nUR 5 nUR UR UR UR
UR S.U S' / U S' [ R R 5 ( a [ [ R R 5 5 Hv n[ R R U5 nUS nUS nU S R U[ R R U5 UUXV-
US: a
XV-
U- S- OSS.5 Mx U $ ) z-Get current memory statistics for CPU and GPUr N)rs availablert percentru rq gpusr d )rC r& rs ru rt r )r" virtual_memoryrs r rt r r! r[ r r9 device_count mem_get_infor r )statsr" cpu_memr gpu_memrs ru s r( get_memory_statsr K s E ##%G && ??!! E%L E&M zz uzz..01Ajj--a0GAJE1:D &M
2215 =BQYU\U 2c9A
"
2 Lr* c J S H nU S: a
U S SU 3s $ U S- n M U S S3$ )z&Convert bytes to human readable format)BKBMBGBTBg @.2fr- z PBr ) bytes_valueunits r( format_bytesr m sD - !#&av. .v . #
c ""r* c < [ SUS- 5 nU S- US- S- - $ )N ?i @g333333?
g?)min) speed_score
chunk_size chunk_factors r( calculate_combined_scorer v s. sJ*; <=L |b 03 6 66r* c ^^ ^ SSK nU R Gt p4nUR U5 U- m U( a US- OSmU R 5 UR 5 S- - nT U- S- S- nXg- m SnSn S n
UU U 4S jn [ 5 n / n
U S
Hk nU
" US US 9nU " U5 nU( d M [ UT 5 T- nT U- n[
S
U5 nU
R SUS UUS U- US UUS
UUUU- US. 5 Mm U
" U S S U S 9nU " U5 nU( aQ [ UT 5 T- nT U- n[
SU5 nU
R SSUU S S U - U S S UUSUU SUU- US.
5 U
( a U
R S SS9 U
S n[ ( a^ [ R " SUS SUS =( d S S US S S US! S" S#US$ S% S&[ US' 5 S([ US) 5 35 US US US 4$ / nUR SSU S S U - U S S S*.5 U S
H% nUR SUS US U- US S*.5 M' UR S+ SS9 US nUS T -
nUS: a$ [ S,[ S-[ UT- 5 T 5 5 nO [ S,T 5 n[ R " S.US SUS =( d S S/US S0[ US1 5 S2[ T 5 S33 5 US US U4$ )4zZ
Choose optimal device and chunk size for mxfp4 dequantization based on available memory.
r N# i r r g ?c X- nU$ rM r ) free_memory
safety_factorusable_from_frees r( calculate_safe_usable_memoryG_choose_mxfp4_processing_strategy..calculate_safe_usable_memory s &6 r* c b > U T-
nUS:: a g[ UT- 5 n[ TU5 nUS: a gU$ )zDCalculate the largest chunk size that fits in the safe usable memoryr N )r r )safe_usable_memorytemp_memory_budgetmax_chunk_from_memory
optimal_chunkbase_memory_per_rowpersistent_memory
rows_totals r( calculate_optimal_chunk_sizeG_choose_mxfp4_processing_strategy..calculate_optimal_chunk_size sJ /2C C " #$6:M$M N J(=>
4 r* r ru )r r g @r[ rC rs ) rB rC r available_memory total_memoryr
needed_memoryr efficiency_scorer memory_utilizationcombined_scorerq r r )
rB rC r r r r r r r r fragmentation_factorr r c U S $ )Nr r rV s r( r 3_choose_mxfp4_processing_strategy.. s
q1A/Br* T)r reversez[MXFP4] Selected rB :r4 r r r z rows per chunk (safety factor: r z.0%z, safe memory utilization: r z.1%z
) - Need: r z
, Available: r )rB rC r total_availablec U S $ )Nr r rV s r( r r
s +r* r i zH[MXFP4] Insufficient memory for optimal processing on any device. Using z with minimal chunks (z). Available: r z , Required: z. Processing will be slow.)mathrS rQ numelr r r r sortr
r r r r r r% r& ) r@ rA r prefix_shapeGr
input_size output_sizeGPU_SAFETY_FACTORCPU_SAFETY_FACTORr r r suitable_strategiesgpur r temp_memorytotal_memory_neededr cpu_safe_usable_memorycpu_chunk_sizebestfallback_options
best_fallbackremaining_memoryfallback_chunk_sizer r r s @@@r( r' r' } sT (--\a<(1,J %&!b&8 $$& )<)<)>)BBJ q.1$q(K"0
E V} 9F +
22DE
: j*58KKK"3k"A 5c: FN & & % -",$'K2C$C #G &8!4 "$.!2&9,,01E,Fs+K L'_(=>? @ *40B+C D E G
]#T+%6=M8NOO 5\+.1BB < 4 V} ! [) V'88 "6{ !
6E$Q'M %[14EE! !$D#6FJ]6]2^`j(k l !$
3 MM }-.a
k0J0Pb/Q R 3A 6 7"=1B#CDE F!"345 6# $
- (- *DFY ZZr* c t U ( d g[ US5 ( a UR R 5 S:X a gg)zHDetermine if we need to split shards based on T4 and GPT-OSS conditions.Fr8 gpt_ossT)r r8 r )r r r s r( r r s2 |\** " " ( ( *i 7 r* c [ R R X5 n[ R R U5 ( d U /$ [ R R U5 nUS- S- S- nXE:: a U /$ [ SU SUS- S S35 [
X25 nSSKn[ UR 5 5 SS n/ n [ U5 H u pS
U S U
S S
3n [ R R X 5 n
[ XSS0S9 U R U 5 [ S U R 5 5 5 n[ ( d Mx [ R" " SU SUS- S S35 M [ R$ " U5 [ ( a [ R" " SU 35 U $ ! [& a n[ SU SU 35 U /s SnA$ SnAff = f)zGSplit a file if needed, using temporary names to avoid messy numbering.r z
Splitting z (size: i @r z GB)...r Nr temp_split_r 03dr rK r r c 3 b # U H% oR 5 UR 5 - v M' g 7frM )r rR )r rg s r( r (split_safetensor_file..J s$ aR`\\^f.A.A.C CR`s -/zCreated temp chunk: z GB)zRemoved original file: zError splitting r )rE rF r rG r r7 split_safetensors_to_shardsuuidr= uuid4r rW r rh ri r
r r r0 r )r r r r r max_shard_size_bytesshardsr temp_basetemp_filenamesr shardrE temp_file_path
shard_sizer s r( r r , s ^6I
77>>) $ $z *I,t3d:TA(z Jxjg)>s(C7
KL ,YJ
%bq) !&)HA ))AaWL IM WW\\. HN ex6F G ! !- 0 aRWR^R^R` aaJ%% 2=/*X_J`adIeei jk * ) ! ! KK1(< =
"QC01zs& B,F! A F! !
G +G>G G c Z [ U 5 S:: a [ U 5 S:X a U S S:w a [ R R XS 5 n[ R R US5 n[ R R U5 ( a> [ R
" X#5 [ ( a [ R " SU S S35 S/$ U $ [ U 5 n[ U5 Vs/ s H nSUS- S SUS S 3PM nn[ ( a [ R " S
5 [ [ X5 5 nU H u p[ R R X5 n[ R R X5 n[ R R U5 ( d Mi X:w d Mp [ R R US U 35 n
[ R
" X*5 [ R
" X5 [ ( d M [ R " SU S U 35 M U$ s snf )
z/Renumber all files with clean sequential names.r r zmodel.safetensorszRenamed z -> model.safetensorszmodel-05dz-of-r zBUnsloth: Renumbering safetensor files with sequential numbering... renaming_z -> ) r rE rF r rG renamer
r r r9 r r ) r r old_pathnew_path total_filesr clean_names rename_pairsold_namenew_name temp_paths r( r r Z s
9~ y>Q 9Q<3F#Fww||NaLAHww||N4GHHww~~h'' (-))KK(9Q<.8M NO'( ( i.KPUVaPbcPb1VAaC9DS(9 FPbKc XY I34L +77<<977<<9
77>>( # #(< ^y
5KLI IIh * IIi *%% hxjXJ ?@ + + ds )H(c US- S- S- n[ U SSS9 nUR 5 Vs0 s H oDUR U5 _M nnSSS5 / n[ 5 nSnWR 5 HZ u pIU R 5 U R
5 - n
X- U: a$ U( a UR U5 [ 5 nSnXU' X-
nM\ U( a UR U5 U$ s snf ! , ( d f N= f)z-Split a safetensors file into smaller shards.r r rq r Nr )rV r r rX r r rR r ) r r rl r r all_tensorsr
current_shard current_sizerg tensor_sizes r( r r s &-4t;N 9U ;q9:B#ALL -- B
< F MML"((* llnv':':'<< % 6= MM- ( 'MM L #c# +
m$ M) C
< ;s C*C%C*%C**
C8c X1; a gX nUS u pxUSU- -
nUSU- -
nX-
n UR U5 R 5 R 5 n
U
R 5 R 5 n X:w a+ [
( a [ R " SU SU SU 35 gU
R [ R 5 n SSK n
U R 5 nU
R U R 5 - R U5 n[! U5 XU& AA A
Ag ! [" a1 n[
( a [ R$ " S
U S U 35 SnAgSnAff = f) zK
Write tensor directly to memory-mapped file using pure PyTorch operations
F data_offsetsr zSize mismatch for z : expected z, got r NTz Direct tensor write failed for r )ra r* rq untyped_storagenbytesr
r warningviewr! uint8ctypesdata_ptrc_ubyter from_addressbytesr r )r r r r
rg r key_metadataindex_Lindex_R
expected_sizetensor_formatted tensor_bytes tensor_viewr> r? byte_datar s r( r r s] / ,&2 '7 1'''1''' )
"99\2==?CCE (779@@B (%%!3J<{=/Y_`l_m no '++EKK8 '')^^k&7&7&99GGQ $I .7
! ! KK9* RsK Ls$ D BD "A0D
E 'E Er )FT)FNr ) Fr TNNFFra F)
Nunsloth_finetuned_mergeFFNr NFFT)TTr4 N) NrK Fr TNFNNFF)rN rM )r )__all__r%
peft_utilsr utilsr hf_utilsr temporary_patches.commonr
r r r r( r
r r$ ModuleNotFoundErrorrJ r! r r r@ r huggingface_hub.utils huggingface_hub.utils._tokenrJ r r rE pathlibr typingr r r r+ peftr r r) r rS rT rU safetensorsrV safetensors.torchrW rX rY r r8 r functoolsrj r r boolr| r peft.utils.integrationsr numpyrP rK dataclassesr r r rR r r2 r4 r r> _TYPESSAFETENSORS_DTYPESr re rb rd rf int64int32int16int8r=
float8_e4m3fn float8_e5m2uint64uint32uint16r r rI rJ rK rT rW rx r r r r= r r r r r r7 r> r rI rd r| r r r r r r r r r r' r r r r r r r* r( ri s "
. ' D #*j 2
) 6 ( ( 0 .
.%^
" ' # $
uxx 4 L
= $ !
, _$ _$@
$**11( $ C CH x xr ) '
#yt T ( R =c =t =( 4 ) | |z
C
C S C C 4. C
C= C
CH
NU39%5
Ns
NVY
N #8D> #H 2 Pb
4 R f c . #H
'8)r jV +X )*T @ # 7
`[B +X %L 4 3h gM () * &* %)" *H ;3;:X
KKef}}}}}}{{{{{{jjkk
&&$$|||||| sA J J 7J0 JJJ-J J- J'%J-0B