oi+SSKrSSKJr SSKrSSKJr SSKJs Jr SSK J r SSK J r J r SSKJr "SS\ 5r"SS \R"\5rg) N)Optional)Conv1D)BaseTunerLayercheck_adapters_to_merge) transposec\rSrSrSrS\R 4Sjr\S\ 4Sj5r SS\ S\ S \ S \ S \ S \ S \ S\ 4SjjrSrSrg) VBLoRALayer)vblora_logits_Avblora_logits_Bvblora_vector_bank base_layerc \Xl0Ul0Ul[R"05Ul[R "05Ul[R "05UlSUl /Ul UR5n[U[R5(aURURpCO^[U[ 5(aI[#UR$S5(aUR$R&OUR$R(up4WUlWUlX lg)NFds_shape)rrtopknn ModuleDictvblora_dropout ParameterDictr r _disable_adaptersmerged_adaptersget_base_layer isinstanceLinear in_features out_featuresrhasattrweightrshapekwargs)selfrr!rrs R/home/james-whalen/.local/lib/python3.13/site-packages/peft/tuners/vblora/layer.py__init__VBLoRALayer.__init__s$  mmB/ "//3!//3"'!((* j")) , ,(2(>(> @W@W  F + +.5j6G6G.T.T !!**ZdZkZkZqZq &K'( returnc,[UR5$N)boolr)r"s r#mergedVBLoRALayer.merged9sD(())r& adapter_namerr num_vectors vector_lengthrinit_logits_stdinference_modec  US::a[SUS35eUS::a[SUS35eURU-S:wa[SURSU35eURU-S:wa[SURSU35eX0RU'X@RU'US:a[ R "US 9n O[ R"5n URR[ R"X055 [ R"[R"X0RU-U5S S 9URU'[ R"[R"URU-X55S S 9URU'X lUR#X5 UR%U5 UR'UR(U S 9 g) Nrz`r` z# should be a positive integer valuez`topk` z`in_features` z& must be divisible by `vector_length` z`out_features` )pT) requires_grad)r1) ValueErrorrrrrrDropoutIdentityrupdater Parametertorchzerosr r r reset_vblora_logits%_move_adapter_to_device_of_base_layer set_adapteractive_adapters) r"r-r rrr.r/rr0r1r!vblora_dropout_layers r# update_layerVBLoRALayer.update_layer=s 6tA3&IJK K 19wtf,OPQ Q   m +q 0~d.>.>-??efsetuv v   } , 1!$"3"3!44Z[hZij  !|"& , C #%::#? #%;;=  ""2==,1U#VW-/\\ KK++}[[RU]5 [R"X40U D6 XlX0lURX2XGXVX5 Xlgr)) superrrr$r rY_active_adapterrBrZ)r"rr r-rr.r/rrr0rYrZr! __class__s r#r$Linear.__init__qsU bii')T88,+  a{Sa (?$r& safe_merge adapter_namesc>[X5nU(dgUGHnX0RR5;dM#UR5nU(aURR R 5nXPRU5- n[R"U5R5(d[SUS35eXTRlO.UR=R URU5- slURRU5 GM g)a Merge the active adapter weights into the base weights Args: safe_merge (`bool`, *optional*): If True, the merge operation will be performed in a copy of the original weights and check for NaNs before merging the weights. This is useful if you want to check if the merge operation will produce NaNs. Defaults to `False`. adapter_names (`List[str]`, *optional*): The list of adapter names that should be merged. If None, all active adapters will be merged. Defaults to `None`. Nz1NaNs detected in the merged weights. The adapter z seems to be broken)rr rErrdatacloneget_delta_weightr;isfiniteallr6rappend)r"r`raactive_adapterr orig_weightss r#merge Linear.merges0D  +N!5!5!:!:!<<!002 $.#4#4#9#9#?#?#AL $9$9.$IIL >>,7;;==(OP^O__rs.:%%*%%**d.C.CN.SS*$$++N;,r&cUR(d[R"S5 g[UR5S:aURR 5nXR R5;a<UR5R=RURU5-sl [UR5S:aMgg)Nz Already unmerged. Nothing to do.r) r+warningswarnlenrpopr rErrrcre)r"ris r#unmergeLinear.unmerges{{ MM< = $&&'!+!11557N!5!5!:!:!<<##%,,11T5J5J>5ZZ1$&&'!+r&logitscURUSS9upE[R"USS9nURS5X%-R S5$)N)dim)rFsoftmax unsqueezesum)r"rtr r top_k_logitsindices topk_weightss r#_get_low_rank_matrixLinear._get_low_rank_matrixsL & Db 9 yy26 &&r*-?-HHMMbQQr&cURUnURUnUR(a1USR5R 5(a [ S5eUR URUR5nURUnU(a0UR5nUR5nUR5nURX5U5RURSS5nURXEU5RSS5RSURS5nXx4$)N)rrzoFound infinity values in VB-LoRA logits. Ensure training was not resumed from a `save_only_topk_weights` model.rrv)r r trainingisinfany RuntimeErrorr todevicerrTrreshaper r) r"adapter cast_to_fp32r r r rABs r#_get_lora_matricesLinear._get_lora_matricess-..w7..w7 ==_T288:>>@@B "44W=@@AWAWXyy! -335O-335O!3!9!9!;   % %o4 P X XYhYnYnopYqsu v  % %o4 P Yq!_ WR..q1 2 t r&cURURnURURnURS:H=(a U[R :HnUR X5upV[Xe-UR5nU$)z Compute the delta weight for the given adapter. Args: adapter (str): The name of the adapter for which the delta weight should be computed. cpu) r rdtypetyper;float16rrrY)r"rrrrrr output_tensors r#reLinear.get_delta_weightsx%%g.55$$W-33{{e+F0F &&w=!!%)<)<= r&xc URnUR(a8UR(aUR5 UR"U/UQ70UD6nOUR(aUR"U/UQ70UD6nOUR"U/UQ70UD6nUR HnX`R R5;aM"URU5upxURURUR5nURUn U[R"[R"U "U5U5U5-nM URU5nU$r))rdisable_adaptersr+rrrr@r rErrr rrylinear) r"rargsr!previous_dtyperesultrirrdropouts r#forwardLinear.forwards  {{ __Q888F [[__Q888F__Q888F"&"6"6!)=)=)B)B)DD..~>DD00@FFG--n=!((188GAJ+BA"FF #7>* r&)r]rYrZ)rr3rJFF)FN)r'N)F)rKrLrMrNrRrSrTr*r$rlistrkrrr;tensorTensorrtuplerrerrU __classcell__)r^s@r#rros #!%$(-? ?  ?  ??????"&? ??2!<!rsG -L&QX.QXhLRYY Lr&