oi$SSKrSSKrSSKJr SSKrSSKJr SSKJs Jr SSK J r J r "SS\ 5r "SS\R\ 5rg)N)Optional)BaseTunerLayercheck_adapters_to_mergecd\rSrSrSrSrS\R4SjrS S\ S\ 4Sjjr S r S r S r g ) ShiraLayer) shira_weight)rscaling shira_indices base_layerc Xl0Ul0Ul[R"05Ul0UlURRUl SUl /Ul UR5n[U[R5(aURUR pCO [#S5eX0lX@lX lg)NFz)Only nn.Linear layers supported currently)r r r nn ParameterDictr r weightshape weight_shape_disable_adaptersmerged_adaptersget_base_layer isinstanceLinear in_features out_featuresNotImplementedErrorkwargs)selfr rrrs Q/home/james-whalen/.local/lib/python3.13/site-packages/peft/tuners/shira/layer.py__init__ShiraLayer.__init__ s$ ,,R0&--33"'!((* j")) , ,(2(>(> @W@W%&QR R&( init_weightsinference_modec US::a[SU35eX0RU'SURU'X0RUR--nXpRUR-:a)[SUSURUR-S35eU(a[ R "U5O[ R"U5n[R"URURRR5RURRR5SS9URU'UGb[ R "US:H5n [ R""U SR%S5U S R%S5/S5R[ R&5UR(U'UR(URURRR5UR(U'UR(UR*S URUR*S:wa[S US UR35eUR-U5 UR/UR0US 9 g) Nrz?`r` should be a positive integer value but the value passed is g?z The set rank zP results in more shira params than the total number of params in the base layer z and this is not allowed.T) requires_gradzFThe SHiRA indices and weights are not the same dimensions for adapter z in layer )r#) ValueErrorr r rrtorchzerosrandnr Parametertor rdtypedevicer wherecat unsqueezeintr r%_move_adapter_to_device_of_base_layer set_adapteractive_adapters) r adapter_namemaskr r"r#rnum_shira_weightshira_init_weight mask_indicess r update_layerShiraLayer.update_layer6sy 6^_`^abc c |  \" 0 043D3D DE ..1B1BB Bs"rswtDtDGKGXGXtXsYYrs >JEKK(89u{{[kOl*,,,  !7!7!=!= > A A$//BXBXB_B_ `+ ,'   ;;ts{3L/4yya**1-|A/H/H/KLa0bm   | ,04/A/A,/O/R/RSWSbSbSiSiSpSp/qD  | ,!!,/55a8D)r )radapterscales r set_scaleShiraLayer.set_scaleis ,, &  % Wr!) rr rrrrr r r r rN)TF)__name__ __module__ __qualname____firstlineno__adapter_layer_namesother_param_namesrModulerboolr;rArF__static_attributes__r!rrrsM+92996"$ .N  .N  .N`8&r!rc ^\rSrSrSS\S\S\S\SS4 U4SjjjrSS \S \\ \SS4S jjr SS jr S\ R4S jrS\ RS\ R4SjrS\4U4SjjrSrU=r$)rpr6r fan_in_fan_outr"returnNc >[TU]5 [R"X40UD6 XPlURUR 5La [ S5eX0lURX2XFS9 g)Nz)SHiRA does not support nested base layers)r") superrrrTr rr'_active_adapterr;) rr r7r6r rTr"r __class__s rrLinear.__init__rsc D77, ??$"5"5"7 7HI I+ ,aKr! safe_merge adapter_namesc>[X5nU(dgUGHnX0RR5;dM#UR5nU(aURR R 5nXPRU5- n[R"U5R5(d[SUS35eXTRlO.UR=R URU5- slURRU5 GM g)a Merge the active adapter weights into the base weights Args: safe_merge (`bool`, *optional*): If True, the merge operation will be performed in a copy of the original weights and check for NaNs before merging the weights. This is useful if you want to check if the merge operation will produce NaNs. Defaults to `False`. adapter_names (`List[str]`, *optional*): The list of adapter names that should be merged. If None, all active adapters will be merged. Defaults to `None`. Nz1NaNs detected in the merged weights. The adapter z seems to be broken)rr keysrrdatacloneget_delta_weightr(isfiniteallr'rappend)rr[r\active_adapterr orig_weightss rmerge Linear.merges0D  +N!2!2!7!7!99!002 $.#4#4#9#9#?#?#AL $9$9.$IIL >>,7;;==(OP^O__rs.:%%*%%**d.C.CN.SS*$$++N;%,r!cUR(d[R"S5 g[UR5S:aURR 5nXR R5;a<UR5R=RURU5-sl [UR5S:aMgg)Nz Already unmerged. Nothing to do.r) mergedwarningswarnlenrpopr r^rrr_ra)rres runmergeLinear.unmerges{{ MM< = $&&'!+!11557N!2!2!7!7!99##%,,11T5J5J>5ZZ1$&&'!+r!cURURURUR5URU'[R "URUURUUR U-UR5$)z Compute the delta weight for the given adapter. Args: adapter (str): The name of the adapter for which the delta weight should be computed. )r r,r r.r(sparse_coo_tensorr r)rrDs rraLinear.get_delta_weights'+&8&8&A&D&DTEVEVW^E_EfEf&g7#&&   w '):):7)CdllSZF[)[]a]n]n  r!xc0UR(a9UR(aUR5 UR"U/UQ70UD6nU$UR(aUR"U/UQ70UD6nU$[R "URR R5nURH5nX`RR5;aM"XPRU5- nM7 [R"XURRS9nU$)N)bias)disable_adaptersrjror copydeepcopyrr_r5r r^raFlinearrv)rrtargsrresult new_weightres rforwardLinear.forwards  {{ __Q888F [[__Q888F t'='='B'BCJ"&"6"6!):):)?)?)AA33NCC #7 XXa$//2F2FGF r!c*>[TU]5nSU-$)Nzshira.)rW__repr__)rreprYs rrLinear.__repr__sg #~r!)rXrT)rFT)FN)rUN)rHrIrJrKstrr2rOrrlistrgror(TensorrarrrP __classcell__)rYs@rrrps$!L L  L  LL LL&%<%rsB  LS&S&liRYY ir!