oi3-SSKJr SSKrSSKrSSKJr SSKrSSKJr SSK J r SSK J r SSK JrJr SSKJrJr SSKJr S S KJr S S KJr S S KJr S SKJrJr SSjr"SS\5rg)) annotationsN)Union)_calculate_correct_fan)Conv1D)is_bnb_4bit_availableis_bnb_available) BaseTunerBaseTunerLayer)2TRANSFORMERS_MODELS_TO_VERA_TARGET_MODULES_MAPPING) BufferDict) _maybe_include_all_linear_layers) VeraConfig)Linear VeraLayerc[U[5(a[R"U5nOUn[ US5n[ R "S5nU[ R "U5- n[ R "S5U-n[R"5 URU*XaS9sSSS5 $!,(df  g=f)a Kaiming Uniform Initialisation adapted to accept a `torch.Generator` object for PRNG. Args: tensor_or_shape (`Union[torch.Tensor, tuple[int, ...]]`): Tensor to initialise, or shape of new tensor to create and then initialise. generator: (`torch.Generator`): Generator object that manages the state of the PRNG algorithm in use. Returns: `torch.Tensor`: The initialised tensor. fan_inr g@ generatorN) isinstancetupletorchemptyrmathsqrtno_graduniform_)tensor_or_shapertensorfangainstdbounds P/home/james-whalen/.local/lib/python3.13/site-packages/peft/tuners/vera/model.py _kaiming_initr&&s /5))_-   2C 99Q>> from transformers import AutoModelForCausalLM >>> from peft import VeraConfig, get_peft_model >>> base_model = AutoModelForCausalLM.from_pretrained("facebook/opt-125m") >>> config = VeraConfig(r=128) >>> model = get_peft_model(base_model, config) ``` **Attributes**: - **model** ([`~transformers.PreTrainedModel`]) -- The model to be adapted. - **peft_config** ([`VeraConfig`]): The configuration of the Vera model. vera_lambda_strprefixcURUR5nURX5n[X0R5nSnURR 5HupVUR X55(dM[ U[R5(aURUR4nOg[ U[5(aP[URS5(aURROURRnUSSS2nOMUcUnMXt:wdM[!S[#XG555nM Uc Sn[%U5eU$)z Finds the largest input and output dimensions across linear layers that have been wrapped with VeRA. This will be used for determining the size of the shared vera_A and vera_B matrices. Nds_shapec3<# UHup[X5v M g7fN)max).0abs r% &VeraModel._find_dim..s%]<\DAc!ii<\sz[No layers types compatible with VeRA were found. Please check `peft_config.target_modules`.)get_model_configmodel_prepare_adapter_configr named_modules_check_target_module_existsrnnr out_features in_featuresrhasattrweightr.shaperzip ValueError) selfconfig model_config peft_config largest_shapekeymodule module_shapemsgs r% _find_dimVeraModel._find_dimes" ,,TZZ8 226H 6{JJO  ::335KC33KEE&")),,%22F4F4FF FF++9@PZ9[9[v}}55agananatat +DbD1 $ , , %%]C <\%] ] #6&  oCS/ !cvURU5up4[0URS9Ul[0URS9Ul[ R "SS9RUR5n[URU4US9n[X1R4US9nX`RU'XpRU'g)N) persistentcpu)devicer) rNr save_projectionvera_Avera_Br Generator manual_seedprojection_prng_keyr&r)rErF adapter_namelinear_out_dim linear_in_dimrrVrWs r%_init_vera_A_vera_BVeraModel._init_vera_A_vera_Bs(,v(>%!0F0FG  0F0FG OO51==f>X>XY -8IN9YO$* L!$* L!rPc&URX#5 gr1)r_)rEr9rFr\s r%_pre_injection_hookVeraModel._pre_injection_hooks   6rPc>[TU]U5 URR5HJnX!LaM URUR:wdM%[ SUR<SURS35e [ URR5Vs1sHoRiM sn5n[U5S:a[ SU35egs snf)z A helper method to check the config when a new adapter is being added. Raise a ValueError if there is something wrong with the config or if it conflicts with existing adapters. z_Vera PRNG initialisation key must be the same for all adapters. Got config.projection_prng_key=z but previous config had .rzcVeRA projection weights must be saved for all adapters or none, but got multiple different values: N) super_check_new_adapter_configrHvaluesrZrDsortedrUlen)rErFexisting_configsave_project_unique_values __class__s r%rg#VeraModel._check_new_adapter_configs )&1#//668O(22f6P6PP v[a[u[uZwx++:+N+N*OqR 9&,RVRbRbRiRiRk,lRk-C-CRk,l%m" ) *Q .u-.0  /-msCc Uc [S5eURn[US5=(a URSLn UURUR UR [URSS5[URSS5S.n XS'[U[5(aHURUURURUURUR URS9 gUR"XRURX#40U D6n X R ;aU R#S5 UR%XTX5 g)NzCurrent Key shouldn't be `None`biasis_loaded_in_8bitFis_loaded_in_4bit)r[ vera_dropoutfan_in_fan_out init_weightsloaded_in_8bitloaded_in_4bit) d_initial)rDr[r@rprsrtrugetattrr9rr update_layerrVrWrx_create_new_moduleactive_adapterrequires_grad__replace_module) rE vera_configr\target target_nameparent current_keyoptional_kwargsr[rpkwargs new_modules r%_create_and_replaceVeraModel._create_and_replaces'  >? ? MMvv&B6;;d+B'44)88'44%djj2EuM%djj2EuM  v ff % %     ((((%//  00kk4;;XdwpvwJ#6#66))%0  j IrPc [5(a SSKnSSKJn [ 5(aSSKJn UR SS5n URSS5n URSS5n [U[5(aUR5n OUn U (a[U WRR5(aaUR5n U RU RRU RR U R"S .5 W"XCX40U D6$U (a[U WRR 5(aaUR5nURU R$U R&R(U R&R*S .5 W"XCX40UD6$[U [,RR.5(a-US (a"[0R2"S 5 S=US 'UlOV[U [65(a2S US'US (d"[0R2"S5 S =US 'UlO[9SUS35e[/UUUU4U UR:S.UD6nU$)Nrr) Linear8bitLt) Linear4bitrpFrvrw)has_fp16_weights thresholdindex) compute_dtypecompress_statistics quant_typertzjfan_in_fan_out is set to True but the target module is `torch.nn.Linear`. Setting fan_in_fan_out to False.Tis_target_conv_1d_layerzafan_in_fan_out is set to False but the target module is `Conv1D`. Setting fan_in_fan_out to True.zTarget module z is not supported. Currently, only the following modules are supported: `torch.nn.Linear`, `transformers.pytorch_utils.Conv1D`.)rprx)r bitsandbytesbnbrrrpopgetrr get_base_layerr=copyupdatestaterrrrrArrrrwarningswarnrtrrDrx)rrVrWr\rrrrrrprvrwtarget_base_layereightbit_kwargsfourbit_kwargsrs r%r{VeraModel._create_new_modulesG    & ) " " 'zz&%($4e<$4e< fn - - & 5 5 7  &  j):CFFO>O P P#[[]N  ! !%6%D%D+<+C+C+W+W"3":":"E"E fFUnU U )588?? ; ;&' 7INM'(;+E )6 2 204F, -*+ wIML'(;+E )JJ       !++   rP)rVrW)returnztuple[int, int])rFrr\r+rNone)r9z nn.ModulerFrr\r+rr)rFrrr)__name__ __module__ __qualname____firstlineno____doc__r,__annotations__rtuner_layer_clsr target_module_mappingrNr_rbrgr staticmethodr{__static_attributes__ __classcell__)rms@r%r(r(CsQ:!FC ON#J +76(JTDDrPr()rz$Union[torch.Tensor, tuple[int, ...]]rztorch.Generatorrz torch.Tensor) __future__rrrtypingrrtorch.nnr= torch.nn.initrtransformers.pytorch_utilsrpeft.import_utilsrrpeft.tuners.tuners_utilsr r peft.utilsr _buffer_dictr tuners_utilsrrFrlayerrrr&r(rPr%rsi#  0-E>&;$C9CCC:c crP