+h8 >SrSSKrSSKrSSKJr SSKJr \"5(aSSKr\"\5r "SS\R5r SS S S S S SSSSSS. r SSSSSSSSSSSSS. r SSSSSSSSSSS. r0SS _SS!_SS"_SS#_SS$_SS%_SS&_SS'_S(S)_S*S+_S,S-_S.S/_S0S1_S2S3_S4S5_S6S7_rS S!S"S#S$S%S&S'S8S9S:S;S<. rS=S>S?.r\ R$\\ R&\ 0r\ R$\\ R*\0r\ R*\0rS@SA0rSBrSKSCjrSKSDjrSErSFrSKSGjrSKSHjrSI\ 4SJjr!g)LzI State dict utilities: utility methods for converting state dicts easily N)is_torch_available) get_loggerc(\rSrSrSrSrSrSrSrSr g) StateDictType z. The mode to use when converting state dicts. diffusers_oldkohya_sspeft diffusersN) __name__ __module__ __qualname____firstlineno____doc__ DIFFUSERS_OLDKOHYA_SSPEFT DIFFUSERS__static_attributes__r Z/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/utils/state_dict_utils.pyrr s$MH DIrrz.to_out.0.lora_Bz.to_out.0.lora_Az .to_q.lora_Az .to_q.lora_Bz .to_k.lora_Az .to_k.lora_Bz .to_v.lora_Az .to_v.lora_Bz.lora_B.lora_Az.to_out.0.lora_magnitude_vector) .to_out_lora.up.to_out_lora.down.to_q_lora.down .to_q_lora.up.to_k_lora.down .to_k_lora.up.to_v_lora.down .to_v_lora.upz.lora.upz .lora.down.to_out.lora_magnitude_vectorz.q_proj.lora_Bz.q_proj.lora_Az.k_proj.lora_Bz.k_proj.lora_Az.v_proj.lora_Bz.v_proj.lora_Az.out_proj.lora_Bz.out_proj.lora_Aztext_projection.lora_A.weightztext_projection.lora_B.weight) .q_proj.lora_linear_layer.up.q_proj.lora_linear_layer.down.k_proj.lora_linear_layer.up.k_proj.lora_linear_layer.down.v_proj.lora_linear_layer.up.v_proj.lora_linear_layer.down.out_proj.lora_linear_layer.up .out_proj.lora_linear_layer.down.lora_linear_layer.up.lora_linear_layer.downz text_projection.lora.down.weightztext_projection.lora.up.weight) rrr rr"r!rrr,r-r$r%r&r'r(r)r*r+z to_k.lora_Azto_k.lora.downz to_k.lora_Bz to_k.lora.upz to_q.lora_Azto_q.lora.downz to_q.lora_Bz to_q.lora.upz to_v.lora_Azto_v.lora.downz to_v.lora_Bz to_v.lora.upzto_out.0.lora_Azto_out.0.lora.downzto_out.0.lora_Bzto_out.0.lora.upz.k_proj.lora_magnitude_vectorz.v_proj.lora_magnitude_vectorz.q_proj.lora_magnitude_vectorz.out_proj.lora_magnitude_vector) rrr rr"r!rrz.to_k.lora_magnitude_vectorz.to_v.lora_magnitude_vectorz.to_q.lora_magnitude_vectorr# lora_downlora_up)lora_Alora_Bz .processor..c"0nUR5Hxup4[R5H$nXS;dM [UnURXV5nM& UR5HnXS;dM XnURXV5n O XBU'Mz U$)a Simply iterates over the state dict and replaces the patterns in `mapping` with the corresponding values. Args: state_dict (`dict[str, torch.Tensor]`): The state dict to convert. mapping (`dict[str, str]`): The mapping to use for conversion, the mapping should be a dictionary with the following structure: - key: the pattern to replace - value: the pattern to replace with Returns: converted_state_dict (`dict`) The converted state dict. )itemsKEYS_TO_ALWAYS_REPLACEkeysreplace) state_dictmappingconverted_state_dictkvpattern new_patterns rconvert_state_dictr?s   "-224G|4W= IIg35 ||~G|%. IIg3 & #$Q# rc lUcw[SUR555(a[RnOA[SUR555(a[RnO [ S5eU[ R5;a[ SUS35e[ Un[X5$)a Converts a state dict to the PEFT format The state dict can be from previous diffusers format (`OLD_DIFFUSERS`), or new diffusers format (`DIFFUSERS`). The method only supports the conversion from diffusers old/new to PEFT for now. Args: state_dict (`dict[str, torch.Tensor]`): The state dict to convert. original_type (`StateDictType`, *optional*): The original type of the state dict, if not provided, the method will try to infer it automatically. c3,# UH nSU;v M g7f to_out_loraNr .0r;s r -convert_state_dict_to_peft..=+-8IJKK&}5G j 22rc ^URSS5mTbST-mOSmUc[SUR555(a[RnOk[U4SjUR555(a[R nO2[SUR555(aU$[ S5eU[R5;a[ S US 35e[Un[X5$) a^ Converts a state dict to new diffusers format. The state dict can be from previous diffusers format (`OLD_DIFFUSERS`), or PEFT format (`PEFT`) or new diffusers format (`DIFFUSERS`). In the last case the method will return the state dict as is. The method only supports the conversion from diffusers old, PEFT to diffusers new for now. Args: state_dict (`dict[str, torch.Tensor]`): The state dict to convert. original_type (`StateDictType`, *optional*): The original type of the state dict, if not provided, the method will try to infer it automatically. kwargs (`dict`, *args*): Additional arguments to pass to the method. - **adapter_name**: For example, in case of PEFT, some keys will be prepended with the adapter name, therefore needs a special handling. By default PEFT also takes care of that in `get_peft_model_state_dict` method: https://github.com/huggingface/peft/blob/ba0477f2985b1ba311b83459d29895c809404e99/src/peft/utils/save_and_load.py#L92 but we add it here in case we don't want to rely on that method. adapter_nameNr2c3,# UH nSU;v M g7frBr rDs rrF2convert_state_dict_to_diffusers..rHrIc36># UHnSTS3U;v M g7frz.weightNr rEr;peft_adapter_names rrFr[s#VDUq7,-W5:DUc3,# UH nSU;v M g7frKr rDs rrFr[rMrIrNrOrP) poprQr6rrrrRDIFFUSERS_STATE_DICT_MAPPINGSr?)r8rTrUr9r_s @rconvert_state_dict_to_diffusersrds, >48$"33 =:??+<= = =)77M VJOODUV V V)..M E:??3DE E E LM M9>>@@>-8IJKK+M:G j 22rc$[n[X5$)zY Converts a state dict from UNet format to diffusers format - i.e. by removing some keys )UNET_TO_DIFFUSERSr?)r8r9s rconvert_unet_state_dict_to_peftrgs G j 22rc[U5n[ SUR 555(d [ S5eU$![a%n[U5S:Xa[U5nSnAN[eSnAff=f)z Attempts to first `convert_state_dict_to_peft`, and if it doesn't detect `lora_linear_layer` for a valid `DIFFUSERS` LoRA for example, attempts to exclusively convert the Unet `convert_unet_state_dict_to_peft` rNNc3D# UHnSU;=(d SU;v M g7f)r0r1Nr )rEkeys rrF1convert_all_state_dict_to_peft..s"N=Mcx31(c/1=Ms z#Your LoRA was not converted to PEFT)rV ExceptionstrrgrQr6rR)r8 peft_dictes rconvert_all_state_dict_to_peftrpsi .z:  NY^^=MN N N>??   q6D D7 CI  s ? A. A)(A))A.c ^ SSKnUR SS5m T bST -m OSm Uc8[ U 4SjUR 555(a[RnU[R 5;a[SUS 35e[U[[R5n0nUR5HupgS U;aURS S 5nOJS U;aURS S 5nO1SU;aURSS5nOSU;aURSS5nURSSURS5S- 5nURT S5nXuU'SU;dMURS5SS3nUR "[#U55XX'M U$![a [RS5 ef=f)a Converts a `PEFT` state dict to `Kohya` format that can be used in AUTOMATIC1111, ComfyUI, SD.Next, InvokeAI, etc. The method only supports the conversion from PEFT to Kohya for now. Args: state_dict (`dict[str, torch.Tensor]`): The state dict to convert. original_type (`StateDictType`, *optional*): The original type of the state dict, if not provided, the method will try to infer it automatically. kwargs (`dict`, *args*): Additional arguments to pass to the method. - **adapter_name**: For example, in case of PEFT, some keys will be prepended with the adapter name, therefore needs a special handling. By default PEFT also takes care of that in `get_peft_model_state_dict` method: https://github.com/huggingface/peft/blob/ba0477f2985b1ba311b83459d29895c809404e99/src/peft/utils/save_and_load.py#L92 but we add it here in case we don't want to rely on that method. rNzDConverting PEFT state dicts to Kohya requires torch to be installed.rXr2rYc36># UHnSTS3U;v M g7fr]r r^s rrF.convert_state_dict_to_kohya..9s#TBSQ*+73q8BSr`rOrPztext_encoder_2.z lora_te2.z text_encoder.z lora_te1.unet lora_unetlora_magnitude_vector dora_scale_r.z.alpha)torch ImportErrorloggererrorrbrQr6rrKOHYA_STATE_DICT_MAPPINGSrRr?r4r7countsplittensorlen) r8rTrUrzkohya_ss_partial_state_dictkohya_ss_state_dict kohya_keyweight alpha_keyr_s @rconvert_state_dict_to_kohyars&  >48$"33 T*//BST T T)..M5::<<>-8IJKK#5ZAZ[h[m[mAn"o9>>@  )!))*;[II  )!))/;GI y !))&+>I $ 1!))*A<PI%%c3 0Dq0HI %%&7< )/I& ) #$??3/236:I-2\\#f+-F  *A" O  [\ s F!F?c ^Ub[[U[5(aU/nUR5V^Vs0sH%umn[U4SjU55(dM"TU_M' nnn[ SUR 555$s snnf)Nc3,># UH oT;v M g7fNr )rEfr;s rrF&state_dict_all_zero..\s@\Q[AaQ[sc3p# UH,n[R"US:H5R5v M. g7f)rN)rzallitem)rEparams rrFr^s+M9Luyy!$))++9Ls46) isinstancermr4rQrvalues)r8 filter_strr;r<s ` rstate_dict_all_zerorXst j# & &$J'1'7'7'9]'9tq!S@\Q[@\=\dad'9 ] M9J9J9LM MM^s !BB model_filecFSSKnSSKJn URR USSS9nUR 5=(d 0nSSS5 WR SS5 U(a0URU5nU(a[R"U5$S$g!,(df  NX=f)Nrry)LORA_ADAPTER_METADATA_KEYptcpu) frameworkdeviceformat) safetensors.torchloaders.lora_baserrz safe_openmetadatarbgetjsonloads)r safetensorsrrrraws r_load_sft_state_dict_metadataras=    $ $Z4 $ NRS::<%2 O LL4 ll45"%tzz#/4/ O Ns B B r)"renumr import_utilsrloggingrrzrr|EnumrrfDIFFUSERS_TO_PEFTDIFFUSERS_OLD_TO_PEFTPEFT_TO_DIFFUSERSDIFFUSERS_OLD_TO_DIFFUSERSPEFT_TO_KOHYA_SSrrrSrrcr~r5r?rVrdrgrprrrmrr rrrsl , H DII*+%#%#%#%F  %5&6$4&6$4&6&8(:&((G&E  &'%'%')+&( 4646  4  6 8:#>#>#>+ )!(4737377;#B#B#B%F  !6. !;)! +//1AB3  B36,3^3&<~N c r