+hISSKJrJrJr SSKrSSKJs Jr SSKJr SSK J r J r SSK J r \ "5(aSSKJrJr \ R""\5rS\R(4S jrS\R(4S jrSS \4S jjr"S S\R R(5r"SS\R(5r"SS\R(5r"SS\R85r"SS\R<5rg))OptionalTupleUnionN)nn) deprecatelogging)is_transformers_available) CLIPTextModelCLIPTextModelWithProjection text_encoderc0/n[U[[45(aX[URR R 5H)up#SUS3nURnURXE45 M+ U$[SURR35e)Ntext_model.encoder.layers.z .self_attnz.do not know how to get attention modules for: ) isinstancer r enumerate text_modelencoderlayers self_attnappend ValueError __class____name__)r attn_modulesilayernamemods O/home/james-whalen/.local/lib/python3.13/site-packages/diffusers/models/lora.pytext_encoder_attn_modulesr )sL,0K LMM!,"9"9"A"A"H"HIHA/s*=D//C    ,J I,J`J`JiJiIjkllc0/n[U[[45(aX[URR R 5H)up#URnSUS3nURXT45 M+ U$[SURR35e)Nrz.mlpz(do not know how to get mlp modules for: ) rr r rrrrmlprrrr)r mlp_modulesrrmlp_modrs rtext_encoder_mlp_modulesr&7sK,0K LMM!,"9"9"A"A"H"HIHAiiG/s$7D    /J CLDZDZDcDcCdeffr! lora_scalec[U5Hfup#[UR[5(dM&XRlXR lXR lXRlMh [U5HFup$[UR[5(dM&XRlXRlMH gN) r rq_projPatchedLoraProjectionr'k_projv_projout_projr&fc1fc2)r r'_ attn_module mlp_modules radjust_lora_scale_text_encoderr4Es3LA k((*? @ @,6   ),6   ),6   ).8 + B2,?  jnn&; < <(2NN %(2NN %@r!cZ^\rSrSrS U4SjjrSSSS.U4SjjrS SjrS rS rS r U=r $)r+SNc Z>Sn[SSU5 [T U] 5 SSKJn XlUR R RnUc UR R RnU"UR RUR RUUUUS9Ul X l g)NzsUse of `PatchedLoraProjection` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.r+1.0.0r)LoRALinearLayer) network_alphadevicedtyperank) rsuper__init__ models.lorar9regular_linear_layerweightr;r< in_features out_featureslora_linear_layerr') selfrAr'r:r=r<deprecation_messager9r;rs rr?PatchedLoraProjection.__init__TsT)74GH 1$8!**1188 =--44::E!0  % % 1 1  % % 2 2' " %r!F destinationprefix keep_varscv>URcURR"XAX#S.6$[TU]"XAX#S.6$)NrJ)rErA state_dictr>)rFrKrLrMargsrs rrO PatchedLoraProjection.state_dictosA  ! ! ),,77v w!4eer!c @URcgURRRRURRRR pCURRRR 5nURRRRR 5nURRRRR 5nURRb/X`RR-URR- nXQ[R"USSS24USSS245S--nU(aF[R"U5R5R5(a[!SUS35eUR#XCS9URRlSUlUR%5UlUR%5UlXlgNraThis LoRA weight seems to be broken. Encountered NaN values when trying to fuse LoRA weights for .LoRA weights will not be fused.r;r<)rErArBdatar<r;floatupdownr:r=torchbmmisnananyitemrtocpuw_upw_downr' rFr' safe_fusingr<r;w_origrbrc fused_weights r _fuse_lora PatchedLoraProjection._fuse_loraws  ! ! ) 1188==CCTE^E^EeEeEjEjEqEqv**1166<<>%%((//44::<'',,3388>>@  ! ! / / ;00>>>AWAWA\A\\DeiiT1W vdTUg.WXY.Z!Z[ 5;;|488:??AAOOSfU22  1=v0[!!((-"&HHJ jjl $r!c D[USS5b[USS5cgURRRnURUR p2UR RUS9R5nURRU5R5nUR5UR[R"USSS24USSS245S-- nURX2S9URRlSUlSUl gNrbrcr;rrV) getattrrArBrWr<r;rbr`rXrcr'r[r\rFrgr<r;rbrcunfused_weights r _unfuse_lora"PatchedLoraProjection._unfuse_lorasfd+7GD(TX0A0A0A0]!!((-  r!cURcSUlURcURU5$URU5URURU5--$)N?)r'rErA)rFinputs rforwardPatchedLoraProjection.forwards[ ?? "!DO  ! ! ),,U3 3((/4??TE[E[\aEb3bccr!)rEr'rArcrb)NNrsF) r __module__ __qualname____firstlineno__r?rOrhrpru__static_attributes__ __classcell__rs@rr+r+Ss4%6-1uff%@ ddr!r+c^\rSrSrSrSS\S\S\S\\S\\\ R\ 4S\\ R4 U4S jjjr S \ RS \ R4S jrS rU=r$)r9aM A linear layer that is used with LoRA. Parameters: in_features (`int`): Number of input features. out_features (`int`): Number of output features. rank (`int`, `optional`, defaults to 4): The rank of the LoRA layer. network_alpha (`float`, `optional`, defaults to `None`): The value of the network alpha used for stable learning and preventing underflow. This value has the same meaning as the `--network_alpha` option in the kohya-ss trainer script. See https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_network_README-en.md#execute-learning device (`torch.device`, `optional`, defaults to `None`): The device to use for the layer's weights. dtype (`torch.dtype`, `optional`, defaults to `None`): The dtype to use for the layer's weights. rCrDr=r:r;r<c>[TU]5 Sn[SSU5 [R"XSXVS9Ul[R"X2SXVS9UlX@lX0lX l Xl [RRUR RSU- S9 [RRUR R5 g)NzmUse of `LoRALinearLayer` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.r9r8F)biasr;r<rwstd)r>r?rrLinearrZrYr:r=rDrCinitnormal_rBzeros_) rFrCrDr=r:r;r<rGrs rr?LoRALinearLayer.__init__s N#W.ABIIkeFX ))DU6W+ (&  ((a$h7 tww~~&r! hidden_statesreturnc.URnURRRnURURU55nUR U5nUR bXPR UR - -nURU5$r)r<rZrBr`rYr:r=rFr orig_dtyper<down_hidden_statesup_hidden_statess rruLoRALinearLayer.forward"((    &&!YY}'7'7'>?77#56    )  2 2TYY > > "":..r!)rZrCr:rDr=rY)rxNNN)rrzr{r|__doc__intrrXrr[r;strr<r?Tensorrur}r~rs@rr9r9s0)-59'+''' '   ' u||S012 ' $''4 /U\\ /ell / /r!r9c^\rSrSrSrSS\S\S\S\\\\\44S\\\\\44S\\\\\4\4S \ \ 4U4S jjjr S \ RS \ R4S jrSrU=r$)LoRAConv2dLayera A convolutional layer that is used with LoRA. Parameters: in_features (`int`): Number of input features. out_features (`int`): Number of output features. rank (`int`, `optional`, defaults to 4): The rank of the LoRA layer. kernel_size (`int` or `tuple` of two `int`, `optional`, defaults to 1): The kernel size of the convolution. stride (`int` or `tuple` of two `int`, `optional`, defaults to 1): The stride of the convolution. padding (`int` or `tuple` of two `int` or `str`, `optional`, defaults to 0): The padding of the convolution. network_alpha (`float`, `optional`, defaults to `None`): The value of the network alpha used for stable learning and preventing underflow. This value has the same meaning as the `--network_alpha` option in the kohya-ss trainer script. See https://github.com/darkstorm2150/sd-scripts/blob/main/docs/train_network_README-en.md#execute-learning rCrDr= kernel_sizestridepaddingr:c >[T U]5 Sn[SSU5 [R"XXEUSS9Ul[R"X2SSSS9UlXplX0l[RRUR RSU- S 9 [RRUR R5 g) NzmUse of `LoRAConv2dLayer` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.rr8F)rrrrrwrw)rrrrwr) r>r?rrConv2drZrYr:r=rrrBr) rFrCrDr=rrrr:rGrs rr?LoRAConv2dLayer.__init__s N#W.ABIIk[ahotu ))DF6X]^+   ((a$h7 tww~~&r!rrc.URnURRRnURURU55nUR U5nUR bXPR UR - -nURU5$r)rrs rruLoRAConv2dLayer.forwardrr!)rZr:r=rY)rxrrrN)rrzr{r|rrrrrrrXr?r[rrur}r~rs@rrrs439.445)-''' ' 3c3h/0 ' c5c?*+ 'sE#s(OS01' ''8 /U\\ /ell / /r!rc^\rSrSrSrSS.S\\4U4SjjjrS\\4SjrSS\ S \ 4S jjr S r SS \ RS \ S\ R4SjjrSrU=r$)LoRACompatibleConvi+z3 A convolutional layer that can be used with LoRA. N lora_layerrcP>Sn[SSU5 [TU]"U0UD6 Xlg)NzpUse of `LoRACompatibleConv` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.rr8rr>r?rrFrrPkwargsrGrs rr?LoRACompatibleConv.__init__0s1Q&1DE $)&)$r!c.Sn[SSU5 XlgNznUse of `set_lora_layer()` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.set_lora_layerr8rrrFrrGs rr!LoRACompatibleConv.set_lora_layer7sO"G-@A$r!r'rec<URcgURRRURRRpCURRR 5nURR RRR 5nURRRRR 5nURRb/X`RR-URR- n[R"URSS9URSS95nURUR5nXQU--n U(aF[R"U 5R!5R#5(a[%SUS35eU R'XCS9URlSUlUR)5UlUR)5UlXlg)Nrw start_dimrTrUrV)rrBrWr<r;rXrYrZr:r=r[mmflattenreshapeshaper]r^r_rr`rarbrc _lora_scale) rFr'rer<r;rfrbrcfusionrgs rrhLoRACompatibleConv._fuse_lora=s ?? "  ((.. 0@0@0G0Gv!!'')!!((--335%%,,11779 ?? ( ( 4//777$//:N:NND$,,,3V^^a^5PQ/f!45 5;;|488:??AAOOSfU22  (??&?F HHJ jjl %r!c[USS5b[USS5cgURRnURRURRp2UR R US9R5UlURR U5R5Ul[R"UR RSS9URRSS95nURUR5nUR5URU-- nUR X2S9URlSUlSUlg)NrbrcrlrwrrV)rmrBrWr<r;rbr`rXrcr[rrrrr)rFrgr<r;rros rrpLoRACompatibleConv._unfuse_lora_sfd+7GD(TXSn[SSU5 [TU]"U0UD6 Xlg)NzrUse of `LoRACompatibleLinear` is deprecated. Please switch to PEFT backend by installing PEFT: `pip install peft`.rr8rrs rr?LoRACompatibleLinear.__init__s1S('3FG $)&)$r!c.Sn[SSU5 Xlgrrrs rr#LoRACompatibleLinear.set_lora_layersO"G-@A$r!r'rec URcgURRRURRRpCURRR 5nURR RRR 5nURRRRR 5nURRb/X`RR-URR- nXQ[R"USSS24USSS245S--nU(aF[R"U5R5R5(a[SUS35eUR!XCS9URlSUlUR#5UlUR#5UlXlgrS)rrBrWr<r;rXrYrZr:r=r[r\r]r^r_rr`rarbrcrrds rrhLoRACompatibleLinear._fuse_loras ?? "  ((.. 0@0@0G0Gv!!'')!!((--335%%,,11779 ?? ( ( 4//777$//:N:NNDeiiT1W vdTUg.WXY.Z!Z[ 5;;|488:??AAOOSfU22  (??&?F HHJ jjl %r!c [USS5b[USS5cgURRnURURp2UR R US9R5nURR U5R5nUR5UR[R"USSS24USSS245S-- nUR X2S9URlSUlSUlgrk) rmrBrWr<r;rbr`rXrcrr[r\rns rrp!LoRACompatibleLinear._unfuse_lorasfd+7GD(TXURc[TU] U5nU$[TU] U5X RU5--nU$r))rr>ru)rFrroutrs rruLoRACompatibleLinear.forwardsD ?? "'/-0CJ'/-0EOOMrs0*) &:G   H % BII  299  3U 3YdEHHOOYdx9/bii9/x=/bii=/@TOTOnG299Gr!