oi:SSKrSSKrSSKJrJrJr SSKrSSKJr SSK Js J r SSK J r Jr "SS\ 5r"SS\R \5rg)N)AnyOptionalUnion)BaseTunerLayercheck_adapters_to_mergec \rSrSrSrSrS\RSS4SjrSS\ S \ S \ S \ SS4 S jjr S\ 4S jr S\ 4SjrS\ 4SjrS\SS4SjrSSSjjrSrg) BoneLayer) bone_block)bone_r base_layerreturnNc \Xl0Ul[R"05UlSUl/UlSUlX lUR5n[U[R5(a$URURsUl Ul g[S[U535e)NFTzUnsupported layer type )r r nn ParameterDictr _disable_adaptersmerged_adapterscast_input_dtype_enabledkwargsget_base_layer isinstanceLinear in_features out_features ValueErrortype)selfr rs P/home/james-whalen/.local/lib/python3.13/site-packages/peft/tuners/bone/layer.py__init__BoneLayer.__init__s$ **2.!&!(,% ((* j")) , ,2<2H2H*JaJa /D d/6tJ7G6HIJ J adapter_namer init_weightsinference_modec US::a[SU35eX RU'UR5n[U[R 5(aA[R "[R"X R5SS9URU'O![S[U5R35eUS:XaCURU-S:wdURU-S:wa [S5eURX5 O*U(aUR!X5 OUR#U5 UR%U5 UR'UR(US9 g ) zInternal function to create bone adapter Args: adapter_name (`str`): Name for the adapter to add. r (`int`): Rank for the added adapter. init_weights (`bool`): Whether to initialize weights. rz?`r` should be a positive integer value but the value passed is T requires_gradz0Bone is not implemented for base layers of type batz=The weight matrix must be fully divisible into [r, r] blocks.)r%N)rr rrrr Parametertorchzerosrr TypeErrorr__name__rreset_bat_parametersreset_bone_parametersreset_bone_parameters_random%_move_adapter_to_device_of_base_layer set_adapteractive_adapters)rr"r#r$r%rr s r update_layerBoneLayer.update_layer0s' 6^_`^abc c$% L!((* j")) , ,,.LLQHYHY9Zjn,oDOOL )NtT^O_OhOhNijk k 5 !#q(D,=,=,AQ,F !`aa  % %l 6   & &| 7  - -l ; 22<@ --nMr!c[R"[R"X R5SS9UR U'gNTr'rr*r+r,rr rr"r#s rr0BoneLayer.reset_bone_parametersYs+(* U[[DUDU5Vfj(k %r!c[R"[R"URU-X"5SS9UR U'gr8r9r:s rr/BoneLayer.reset_bat_parameters\s4(* U[[ARARVWAWYZ5^nr(s %r!c[RRURU[R "S5S9 g)N)a)rinitkaiming_uniform_r mathsqrt)rr"s rr1&BoneLayer.reset_bone_parameters_random_s*   !>$))A, Or!scalecUS:XagURH8nX RR5;aM"[R"S5 M: g)NzGScaling operation for Bone not supported! Automatically set scale to 1.r4r keyswarningswarnrrFactive_adapters r scale_layerBoneLayer.scale_layerbs? A: "22N__%9%9%;; MMc d 3r!cURH8nX RR5;aM"[R"S5 M: g)Nz?Unscaling operation for Bone not supported! Keeping scale at 1.rIrMs r unscale_layerBoneLayer.unscale_layerls5"22N__%9%9%;; MM[ \ 3r!) rr r r rrrrrFNrN)r. __module__ __qualname____firstlineno__adapter_layer_namesother_param_namesrModulerstrintboolr5r0r/r1floatrOrR__static_attributes__r!rr r s)#K299K4K, % 'N'N 'N 'N  'N 'NRl#lttPPee4e]]r!r c ,^\rSrSrSrSS\S\S\\\4SS4U4Sjjjr SS \S \ \ \SS4S jjr SS jr SS \S\R4SjjrSS \S\R4SjjrS\RS\S\S\R4SjrS\4U4SjjrSrU=r$) BoneLineartz$ Bone implemented in a dense layer. r"r#r$rNc >[TU]5 [R"X40UD6 X lUR"X#U40UD6 X@lgrU)superrr _active_adapterr5bone_fn)rr r"r#r$r __class__s rrBoneLinear.__init__ysC 46v6+ ,<B6B# r! safe_merge adapter_namesc0[X5nU(dgUGHnX0RR5;dM#UR5nURR nU(aURR R5nURS:XaURX65nXg- nO1URX0RRR 5nUn[R"U5R5(d[SUS35eUR!U5URlOURS:Xa^URX0RRR 5nUR=R UR!U5- slOOURX0RRR 5nUR!U5URlUR"R%U5 GM g)a Merge the active adapter weights into the base weights Args: safe_merge (`bool`, *optional*): If `True`, the merge operation will be performed in a copy of the original weights and check for NaNs before merging the weights. This is useful if you want to check if the merge operation will produce NaNs. Defaults to `False`. adapter_names (`List[str]`, *optional*): The list of adapter names that should be merged. If `None`, all active adapters will be merged. Defaults to `None`. Nr)z1NaNs detected in the merged weights. The adapter z seems to be broken)rr rJrweightdtypedatacloneriget_delta_weightget_delta_weight_boner r+isfiniteallrtorappend)rrlrmrNr orig_dtype orig_weight delta_weights rmergeBoneLinear.merges0D  +N!5!5!77!002 '..44 #-"3"3"8"8">">"@K||u,'+'<'<^'Y #3 '+'A'A.RaRaRhRhRmRm'n &2  >>+6::<<(OP^O__rs.9^^J-GJ%%*||u,'+'<'<^__McMcMhMh'i "))..,//*2MM.'+'A'A.RaRaRhRhRmRm'n 1=1L )).$$++N;9,r!c~UR(d[R"S5 g[UR5S:aURR 5nUR 5nURRnXRR5;aUR 5RRR5nURS:XaURXSS9nOURXSS9nUR!U5URl [UR5S:aMgg)zG This method unmerges all merged adapter layers from the base weights. z Already unmerged. Nothing to do.Nrr)T)re)mergedrKrLlenrpoprrorpr rJrqrrrirsrtrw)rrNr ryrzr{s runmergeBoneLinear.unmerges{{ MM< = $&&'!+!11557N,,.J#**00J!5!5!77"113::??EEG <<5(#'#8#8Y]#8#^L#'#=#=n^b#=#cL)5)D !!&$&&'!+r!rcURURnURURnURS:H=(a- U[R :H=(d U[R :HnURUnU(aUR5nURUR5nURS5nU(aURURS5U-XRS5U-U5RSSSS5n [R"URS55RUR5n [R"X-5n U RUR5n X- U -n U RSSSS5R"UR6n OzURURS5U-XRS5U-U5RSSSS5U-U-n U RSSSS5R"UR6n U(a2U RUS9n URU5URUlU $) Compute the delta weight for the given adapter. Args: adapter (str): The name of the adapter for which the delta weight should be computed. cpurrHrp)r devicerprr+float16bfloat16r`rwsizereshapepermuteeyeinverseshaperq)radapterrzrrrp cast_to_fp32 weight_boner#oone inv_I_plus_bw output_tensors rrsBoneLinear.get_delta_weights-)00(..{{e+c%--1G1b5TYTbTbKb oog. %++-K!nn[%6%67   R  ##K$4$4Q$71$:D ! AvvbzA~*()FF2JN(:a'? EE!a%67..q**=A#eii 0\1773B<0\PRWX0\Z[0\bd&ehl&llF';>* r!c*>[TU]5nSU-$)Nzbone.)rg__repr__)rreprjs rrBoneLinear.__repr__^sg }r!)rhri)rT)FNrVrT)r.rWrXrY__doc__r]r^rr_rrlistr|rr+Tensorrsrtrrrra __classcell__)rjs@rrdrdts)- $ $  $ D#I& $  $ $.<.rsG '' LX]X]vlIlr!