oindSSKJr SSKrSSKrSSKrSSKrSSKrSSKrSSKJ r J r SSK J r SSK JrJr SSKJrJrJrJr SSKrSSKJr SSKJrJr SS KJr SS KJr SS KJr SS KJ r SS K!J"r" SSK#J$r$ SSK%J&r& SSK'J(r(J)r)J*r*J+r+J,r, SSK-J.r. SSK/J0r0J1r1J2r2J3r3J4r4 SSK5J6r6J7r7 SSK8J9r9 SSK:J;r; SSKJ?r? \R"\R5\R"S5:rB\B=(a \RR5rE\S5rFS1SjrGS2SjrH"SS\R\ 5rJ"S S!\ 5rKS3S"jrL"S#S$5rMS4S%jrNS5S6S&jjrOS7S'jrPS8S9S(jjrQS:S;S*jjrRSS-jrU\K4S?S.jjrVS@SAS/jjrWS@SBS0jjrXg)C) annotationsN)ABCabstractmethod)Sequence)contextmanager nullcontext)AnyOptionalUnionoverload)AlignDevicesHook)named_module_tensorsoffload_state_dict)version)nn)tqdm)PreTrainedModel)Conv1D)PEFT_TYPE_TO_PREFIX_MAPPING)INCLUDE_LINEAR_LAYERS_SHORTHAND)DUMMY_MODEL_CONFIGDUMMY_TARGET_MODULESEMBEDDING_LAYER_NAMES#MIN_TARGET_MODULES_FOR_OPTIMIZATIONSEQ_CLS_HEAD_NAMES)init_empty_weights)AuxiliaryTrainingWrapper%_get_module_names_tied_with_embedding _set_adaptermatch_target_against_key set_additional_trainable_modules)PeftTypeTaskType) PeftWarning) PeftConfig)_get_submodules) BufferDictz2.5.0c## /nUR5Hup#US;aM [US5(dM [UR[5(dMAURR (dM^URR U5 URU5 M Sn[US5(Ga[URS5(Ga[URR[5(GaURRR (Ga[R"S5URRRR5;Ga+[URRRS5(aURRRRRn[![#URRRR5R%55SnXVSn/n[&R(R+U5H-n S U ;aURU 5 OURU 5 M/ [&R(R,"U6n U S -n URRR UR5 S nS v UH3nURR/U[R0"/55 M5 U(Ga6[3UR5VV s0sHup,X,R5S 5_M sn nURRl [R"S5URRRR5;aY[URRRS5(a*[7W URRR5 URRR/UR[R0"/55 g g s sn nf7f)a A utility for modifying a module containing one or more tuners and a base layer, any of which are offloaded to the CPU or disk. Moves a module's sub-modules to the execution device before some action is performed, after that the base layer state dictionary is re-assigned (if that layer was offloaded to the disk) and finally the parameters are offloaded. If the module has no offloaded sub-modules, this function does nothing. Args: layer ('torch.nn.Module'): layer with tuners to be merged ) base_layer_hf_hookFr,metadatasetrsafetensors_filez--z-mergedTNcpu) named_moduleshasattr isinstancer-r offload pre_forwardappendr,torchdeviceoriginal_devicesvalues weights_mapr/indexlistdictkeysospathsplitjoin post_forwardtensorrtor) layeroffloaded_modulesnamemodulebase_layer_offloadr= module_name file_name base_name_arri base_namesafetensors_filenameparams R/home/james-whalen/.local/lib/python3.13/site-packages/peft/tuners/tuners_utils.py onload_layerrUAs7++-  % %  6: & &:fooGW+X+X]c]l]l]t]t]t OO ' ' /  $ $V , .ul##  *-- u''002B C C    % % - - - << 5#3#3#<#<#M#M#T#T#V V[b    % % 1 19\ \ $$--99AAGGEtE$4$4$=$=$I$I$Q$QRWWYZ[\]K*+=>IMWW]]9-19!((+$$Q' .  m4I#,y#8  !!--e.>.>?! #$$VU\\"-=>$6J%JZJZ5[1 5[kdD((5/ !5[1 !!- << 5#3#3#<#<#M#M#T#T#V V[b    % % 1 19\ \  3U5E5E5N5N5Z5Z [ !!..u/?/?bAQR1 s$/QQQ3J0Q#QDQc 1SknSS1n1SknURU;aj[US5(aX[URSS5U;a<X$;a6[ SURS US URR S US 3 5egggg) X Prevent applying LoRA to incompatible modules in specific architectures (e.g., Mamba). >LORAXLORAADALORARANDLORAout_projconv1d>mambamamba2 falcon_h1 falcon_mambaconfig model_typeNz[PEFT:z ] Module 'z7' is incompatible with Mamba-based models (model_type='z'). Incompatible modules: zG. Please remove it from `target_modules` to avoid compatibility issues.) peft_typer3getattrrb ValueErrorrc) peft_configmodel target_namelora_like_typesincompatible_modulesmamba_model_typess rT _check_lora_target_modules_mambarms ?O&1H 0 E8 $ $ ELL, 59J J  .../z+G % 7 788RSgRhiXX  / K % 1c[U[R5(a[(af[UR[ R RR5(a)URR5RupX!4$URURpX!4$[U[R5(aURURpX!4$[U[R 5(aURURpX!4$[U[R"5(aURURpX!4$[U[R$5(aUR&UR(pX!4$[U[*5(aL[-URS5(aURR.OURRup!X!4$[U[R05(a9UR2(d [5S5eUR6SUR6-pX!4$[-US5(a+[-US5(aUR8UR:pX!4$[-US5(a+[-US5(aUR<UR>pX!4$[-US5(a4UR@RBS :XaURURpX!4$[-US 5(a4UR@RBS :XaURURpX!4$UR@RBS :XaURURpX!4$[-US 5(a4UR@RBS:XaURURpX!4$UR@RBS:XaURURpX!4$[-US5(a)[-US5(aURURpOSup![DRF"S[IU5S3[J5 X!4$)am Get the in_features and out_features of the layer. Returns in_features and out_features as a tuple. If they cannot be determined, return a tuple of None and None. This function covers a broad range of layers, some of which the caller might not support. Therefore, just because this function returns a valid result does not imply that the layer type is supported. ds_shapezPOnly same dim for query/key/value is supported as of now for MultiheadAttention. infeatures outfeatures input_size output_size codebooksQuantizedLinearw_bit WQLinear_GEMM EetqLinearW_q HQQLinear PatchedLinear in_features out_features)NNzUnsupported layer type 'z(' encountered, proceed at your own risk.)&r4rLinear_torch_supports_distributedweightr8 distributedrFDTensorto_localshaper~rConv1d in_channels out_channelsConv2dConv3d Embeddingnum_embeddings embedding_dimrr3rpMultiheadAttention_qkv_same_embed_dimrf embed_dimrrrsrtru __class____name__warningswarntype UserWarning)rKrr~s rT_get_in_out_featuresrs&"))$$ & &:fmmUEVEVE]E]EeEe+f+f(. (>(>(@(F(F %L^  $$[)/(:(:Ffgitu  $$rnc^\rSrSr%SrS\S'S\S'S\S'S-S.U4S jjjr\S/S j5rS0S jr S1S jr S2S jr S3Sjr \ S4Sj5r\S5S6Sjj5rS7SjrS8S9SjjrS:SjrS:SjrS;SjrS8SSjjrSrS?S@SjjrSASBSjjrSCSjrSDSjrS:SjrSESFSjjrSGS jr S:S!jr!SHSIS"jjr"S#r#SJS$jr$SKSJS%jjr%\ SLS&j5r&SMS'jr'S/S(jr(S)r)SNS*jr*SOU4S+jjr+S,r,U=r-$)P BaseTunera A base tuner model that provides the common methods and attributes for all tuners that are injectable into a torch.nn.Module For adding a new Tuner class, one needs to overwrite the following methods: - **_prepare_adapter_config**: A private method to eventually prepare the adapter config, for example in case the field `target_modules` is missing. - **_create_and_replace**: A private method to create and replace the target module with the adapter module. - **_check_target_module_exists**: A private helper method to check if the passed module's key name matches any of the target modules in the adapter_config. The easiest is to check what is done in the `peft.tuners.lora.LoraModel` class. Attributes: model (`torch.nn.Module`): The model to which the adapter tuner layers will be attached. forward (`Callable`): The forward method of the model. peft_config (`Union[`PeftConfig`, dict[str, PeftConfig]]`): The adapter configuration object, it should be a dictionary of `str` to `PeftConfig` objects. One can also pass a PeftConfig object and a new adapter will be created with the default name `adapter` or create a new dictionary with a key `adapter_name` and a value of that peft config. config (`dict[str, Any]`): The model configuration object, it should be a dictionary of `str` to `Any` objects. targeted_module_names (`list[str]`): The list of module names that were actually adapted. Can be useful to inspect if you want to quickly double-check that the `config.target_modules` were specified correctly. targeted_parameter_names (`list[str]`): The list of parameter names that were actually adapted. Can be useful to inspect if you want to quickly double-check that the `config.target_parameters` were specified correctly. prefix (`str`) The PEFT-method specific unique prefix. E.g. `"lora_"` for LoRA. strprefixtype[BaseTunerLayer]tuner_layer_clszdict[str, list[str]]target_module_mappingcv>[TU]5 Xl/Ul/Ul[ US5(d [ U[5(aX20OUUlOU[R"S5 [ U[5(aX RU'OURRU5 X0l URURURUU5 U[R:wdX#[R:waUR!URX4US9 URURlg)NrgzAlready found a `peft_config` attribute in the model. This will lead to having multiple adapters in the model. Make sure to know what you are doing!)low_cpu_mem_usage state_dict)super__init__rhtargeted_module_namestargeted_parameter_namesr3r4r&rgrrupdateactive_adapter_pre_injection_hookr"rYinject_adapter)selfrhrg adapter_namerrrs rTrBaseTuner.__init__ s  02"35%t]++>HV`>a>a :grD  MMG +z221<  .  '' 4/;   T-=-=l-K\Z (.. (K,E,W    Ljt  u"&!1!1 rncr[UR[5(a UR/$UR$Nr4rrrs rTactive_adaptersBaseTuner.active_adapters/0 d))3 / /''( ("""rnc:URR"U0UD6$r)rhforward)rargskwargss rTrBaseTuner.forward6szz!!42622rncg)aH A hook to be called before the adapter is injected into the model. This method can be overridden by child classes to perform any pre-injection operations. Args: model (`nn.Module`): The model to be adapted. config (`PeftConfig`): The adapter config. adapter_name (`str`): The adapter name. N)rrhrbrs rTrBaseTuner._pre_injection_hook9s rncURc<URRUS5nUc [S5e[ U5UlU$)ar A private method to prepare the adapter config. For transformers based models, if `peft_config.target_modules` is None, for some model architectures, we can automatically infer the target modules from the `TRANSFORMERS_MODELS_TO_XXX_TARGET_MODULES_MAPPING`. Args: peft_config (`PeftConfig`): The adapter config. model_config (`dict`): The transformers model config, that config should contain the `model_type` key. Returns: peft_config (`PeftConfig`): The PEFT config with updated `target_modules`. Raises: ValueError: Raises an error if the model type was not recognized. rcz0Please specify `target_modules` in `peft_config`)target_modulesrgetrfset)rrg model_configrs rT_prepare_adapter_config!BaseTuner._prepare_adapter_configHsP*  % % -!77;;L3K"LMM&&w/+rncURHAn[URUSS5nUS:wdM%SUS3n[R"U5 MC UR SS9 g)z| Disable all adapters in-place. When disabling all adapters, the model output corresponds to the output of the base model. rrz>Careful, disabling adapter layers with bias configured to be 'zL' does not produce the same output as the base model would without adaption.FrN)rrergrrr)rrbias_valmsgs rTdisable_adapter_layers BaseTuner.disable_adapter_layerssq#22Nt//?PH6!TU]T^_LL c"3    /rnc"URSS9 g)z Enable all adapters in-place TrN)rrs rTenable_adapter_layersBaseTuner.enable_adapter_layerss    .rncU[URR55;a[SUS35eURU [ UR XR URS9nU=(d /Ulg)z` Deletes an existing adapter. Args: adapter_name (str): Name of the adapter to be deleted. Adapter z does not exist)rhrr layer_clsN) r>rgr@rfdelete_adapterrhrrr)rr new_adapters rTrBaseTuner.delete_adaptersq tD$4$4$9$9$;< <x ~_EF F   \ *$**< W[WkWk */Rrnc,[URXS9 g)" Enable or disable gradients on the given adapter(s). Args: adapter_name (`str` or `Sequence[str]`): The name of the adapter(s) whose gradients should be enabled/disabled. requires_grad (`bool`, *optional*) Whether to enable (`True`, default) or disable (`False`).  adapter_namesrN)set_requires_gradrh)rrrs rTrBaseTuner.set_requires_grads $**M_rnc^[UR5S::ag[U4SjURR555(d [ S5eURR5Vs/sHn[ USS5PM nn[ SU55S:a"[ URRS35egs snf) z A helper method to check the config of a new adapter being added. Raise a ValueError if there is something wrong with the config or if it conflicts with existing adapters. r(Nc3*># UHoTLv M g7frr).0confrbs rT 6BaseTuner._check_new_adapter_config..sH.Gd6>.Gsz_check_new_peft_config was called incorrectly, this should not happen. Please open an issue and report the error: https://github.com/huggingface/peft/issuesrrc3*# UH oS:gv M g7f)rNr)r bias_values rTrr sBk V#kszf supports only 1 adapter with bias. When using multiple adapters, set bias to 'none' for all adapters.) lenrganyr;rfresumrr)rrbr bias_valuess ` rT_check_new_adapter_config#BaseTuner._check_new_adapter_configs t A % Hd.>.>.E.E.GHHHO  BFAQAQAXAXAZ[AZwtVV4AZ [ BkB BQ F>>**+,77  G\s5Cc,[URXS9 g)a2 A helper method to cast the adapter weights to the correct dtype. Currently, this only upcasts float16 and bfloat16 to float32. Args: adapter_name (`str`): The adapter name. autocast_adapter_dtype (`bool`, *optional*): Whether to autocast the adapter dtype. Defaults to `True`. )rautocast_adapter_dtypeN)cast_adapter_dtyperh)rrr s rT_cast_adapter_dtypeBaseTuner._cast_adapter_dtypes 4::Lprnc[R"S5nURUR5nU(a[R "SU<S3U-5 gg)zHelper method to check whether the adapter can be merged. Raise a ValueError if it is not possible to merge the adapter with the given configuration. a ```python from transformers import AutoModelForCausalLM # Load original tied model model = AutoModelForCausalLM.from_pretrained("google/gemma-2-2b-it", tie_word_embeddings=False) # Set the randomly initialized lm_head to the previously tied embeddings model.lm_head.weight.data = model.model.embed_tokens.weight.data.clone() # Save the untied model untied_model_dir = "dir/for/untied/model" model.save_pretrained(untied_model_dir) model.config.save_pretrained(untied_model_dir) # Now use the original model but in untied format model = AutoModelForCausalLM.from_pretrained(untied_model_dir) ``` BModel with `tie_word_embeddings=True` and the tied_target_modules=z are part of the adapter. This can lead to complications. You can opt to merge the adapter after cloning the weights (to untie the embeddings). You can untie the embeddings by loading the model with `tie_word_embeddings=False`. For example:N)textwrapdedent_get_tied_target_modulesrhrr)r example_codetied_target_moduless rT_check_merge_allowedBaseTuner._check_merge_allowed sb    *#;;DJJG  MMUAT@VWss   rncU(aUR5 URR5VVs/sHupVURU;dMUPM nnnSU(aSOS-S-n[ Xr(+US9Hn[ URU5upn [U 5 [U S5(a#U RXUS9n URXX5 OH[U S5(a7U(aU RX4S 9 URXU R5U 5 SSS5 M UR$s snnf![ a Mf=f!,(df  M=f) Nz Unloading z and merging r+rh)disabledesc"unload_and_optionally_merge_module)merge safe_mergerr,)rr) rrhr2rrr'AttributeErrorrUr3r_replace_modulerget_base_layer) rr progressbarrrr_key_listrrrriunloaded_modules rT_unload_and_optionally_merge&BaseTuner._unload_and_optionally_mergeDs9   % % '&*jj&>&>&@[&@FCDKKWZDZC&@[~B?'I/EC .=djj#.N+ f%6#GHH&,&O&O#-'P'O((oVV\22 X((f>S>S>UW]^&% F"zz'\ "  %%s*D5D5<D; A=E ; E E  E c"URXUS9$)a@ This method merges the adapter layers into the base model. This is needed if someone wants to use the base model as a standalone model. The returned model has the same architecture as the original base model. It is important to assign the returned model to a variable and use it, this is not an in-place operation! Args: progressbar (`bool`): whether to show a progressbar indicating the unload and merge process (default: False). safe_merge (`bool`): whether to activate the safe merging check to check if there is any potential Nan in the adapter weights. adapter_names (`List[str]`, *optional*): The list of adapter names that should be merged. If None, all active adapters will be merged. Defaults to `None`. Example: ```py >>> from transformers import AutoModelForCausalLM >>> from peft import PeftModel >>> model_id = ... >>> base_model = AutoModelForCausalLM.from_pretrained(model_id) >>> peft_model_id = ... >>> model = PeftModel.from_pretrained(base_model, peft_model_id) >>> merged_model = model.merge_and_unload() ``` )r#rrr')rr#rrs rTmerge_and_unloadBaseTuner.merge_and_unloadcs"D00#-1  rnc URSS9$)z Return the base model by removing all the PEFT modules. It is important to assign the returned model to a variable and use it, this is not an in-place operation! F)rr*rs rTunloadBaseTuner.unloads 00u0==rnc[XU5 g)rWN)rm)rrgrhris rT!_check_target_module_compatiblity+BaseTuner._check_target_module_compatiblitys )[IrncF[URRS35e)Nz) does not support targeting nn.Parameter.)rrr)rrgrrrirrs rT_create_and_replace_parameter'BaseTuner._create_and_replace_parameters""T^^%<%<$==f"ghhrnc &URUn/n/n/n URU5 URX5 URU5n UR Xj5nUR Xa5 [ US/5(aU(a [S5e[UR55n U V V s/sHupU PM nn n [ USS5[:HnU(a/n /n[Xa5n[UR[[45(a[UR5[ :aUR"[$R&:wa[)SUR55nUVs/sH/nUUR;dMUR+U5(aM-UPM1 nn[-URU5n[U5[UR5:aUUl /nU H1un n[U[.5(dMUR1U S-5 M3 [5nUb;[2UR"nUVs1sHnUR5SU-S5SiM nnU GHun nU (dMUH,nU R7U5(dMUR1U 5 O U(a US U :XaMTUcUR9Xl5n[U[:5(aUR1U 5 MU(dUR1U 5 MUR<R1U 5 [?X5unnnURAXaU5 U(a[BO[DnU"5 URGXbUUUU S 9 SSS5 GMU U;aUR1U 5 OrUR<R1U 5 [?X5unnnURAXaU5 U(a[BO[DnU"5 URGXbUUUU S 9 SSS5 UR9Xl5(dGMU R1U 5 GM [ US/5(aURIXaX$S 9 Ub[U 5n[UR<5n UU - n!U U- n"S n#U!(dU"(aS n#U!(aU#S[KU!5S3- n#U"(aU#S[KU"5S3- n#U#(aRN"U#[P5 UR<(GdURR(GdU(dU(aU(d [S5eU(d#U(aUR(d [S5eU(ddU(a]SURS3n$[ USS5bU$SURTS3- n$[ USS5bU$SURVS3- n$[U$5eSn$[ USS5bU$SURTS3- n$[ USS5bU$SURVS3- n$[U$5e[YUS5(a=URZ(a,U(d%[LRN"SURZS35 OU(dUR(a;UR<(d*[LRN"SURS3[P5 OL[ US/5(a:URR(d)[LRN"SUR\S 3[P5 UR_US!9n%U%(aRN"S"U%<S#35 URaURbURdS$9 URgU5 URURd(a)URi5Hun&n'UU&;dMS%U'l5M [mUU[nRU5UX Rb;S&9 gs sn n fs snfs snf!,(df  GM=f!,(df  GN=f)'a Creates adapter layers and replaces the target modules with the adapter layers. This method is called under the hood by `peft.mapping.get_peft_model` if a non-prompt tuning adapter class is passed. The corresponding PEFT config is directly retrieved from the `peft_config` attribute of the BaseTuner class. Args: model (`nn.Module`): The model to be tuned. adapter_name (`str`): The adapter name. autocast_adapter_dtype (`bool`, *optional*): Whether to autocast the adapter dtype. Defaults to `True`. low_cpu_mem_usage (`bool`, `optional`, defaults to `False`): Create empty adapter weights on meta device. Useful to speed up the loading process. state_dict (`dict`, *optional*, defaults to `None`) If a state_dict is passed here, the adapters will be injected based on the entries of the state_dict. This can be useful when the exact `target_modules` of the PEFT method is unknown, for instance because the checkpoint was created without meta data. Note that the values from the state_dict are not used, only the keys are used to determine the correct layers that should be adapted. target_parameterszTrying to inject a PEFT adapter from a state_dict but the PEFT config uses `target_parameters`. This is not supported -- when using `target_parameters`, please inject the adapter without the state_dict.rNc3,# UH nSU-v M g7f.Nr)rsuffixs rTr+BaseTuner.inject_adapter..sS8RfS6\8Rsr:r(r)r)rgrhrrr+zWhile injecting the PEFT adapters, an inconsistency was discovered between the PEFT config and the provided state_dict. This is not necessarily an issue and can be ignored if this was the intent. z;The PEFT config contained these additional target modules: z. z:The state_dict contained these additional target modules: zAll modules were excluded. This is likely unintended. Check your `target_modules`, `exclude_modules` and `modules_to_save` configuration.znNo `target_modules` passed but also no `target_parameters` found. Please check the values for these arguments.zTarget modules zL not found in the base model. Please check the target modules and try again.layers_to_transformz, Note: You specified 'layers_to_transform': layers_patternz& You also specified 'layers_pattern': a"No modules were targeted for adaptation. This might be caused by a combination of mismatched target modules and excluded modules. Please check your `target_modules` and `exclude_modules` configuration. You may also have only targeted modules that are marked to be saved (`modules_to_save`).exclude_modulesz You have passed exclude_modules=zS but no modules were excluded. Please check that exclude_modules was set correctly.ztarget_modules=z$ were set but no module was matched.ztarget_parameters=z' were set but no parameter was matched.)rhrz are part of the adapter. This can lead to complications, for example when merging the adapter or converting your model to formats other than safetensors. See for example https://github.com/huggingface/peft/issues/2018.inference_modeF)rhrgrractivate_adapter)8rgr _check_tied_modulesget_model_configrrrerfr>r2r _maybe_include_all_linear_layersr4rrrrrdr"IA3tupler_find_minimal_target_modulesrr7rrsplit startswithr_ExcludedModulerr'r1rrr_inject_parameterssortedrrRuntimeWarningrr>r?r3r@r7r set_adapterrrBrrrr!r)(rrhrr rrrgexcluded_modulesunmatched_modules!targeted_modules_from_peft_configrr2rr$r%uses_dummy_target_modulessuffixesrJnames_no_targetnew_target_modulesexisting_adapter_prefixesrK module_namesrk adapter_keyresultrrrictxtargeted_set_from_peft_configtargeted_set_from_state_dictdiff_peft_configdiff_state_dict warning_msg error_msgrrrs( rTrBaseTuner.inject_adaptersD&&|4 79) &&{3   4,,U3 22;M  K/ ; 3R 8 8Zx  U0023 &34mFCCm4$+K9I4$PTh$h! $MH7{J  {11D#; ? ?[//04WW&&(,,6S 8R8RSSH!)!)d+:T:T.T^b^k^klt^u ">k>X>XZi!j %&[-G-G)HH-? * %'!(KC&.11)00s;) "%  !01F1FGFBLM*QAHHS6\15a8*LM(KC 9 >>+..$++C0 9  $4R$8C$?!99+Kfo66$++C0%,,S1..55c:2A%2M/FFK::;{[0A,{C00'v{F`c1 l*%,,S1..55c:2A%2M/FFK::;{[0A,{C00'v{F`c1 33KEE5<:)))$2O2O2OXq(9 j&*;KD^D^ '&*;&k&@&@%ABEF;(=tDP#OP[PoPoOppq!rrI;(8$?K#I+JdJdIeef!ggI ++] ;(=tDP#OP[PoPoOppq!rrI;(8$?K#I+JdJdIeef!ggI ++ ["3 4 49T9T]m MM2;3N3N2OPGG  + ))$2L2L %k&@&@%AAefhv&92>>tGdGd ()F)F(GGno" #;;%;H  MMUAT@VWSS  --k>X>XY --e4   L ) 8 8..011$&+AO1 )#"33D9%)-A-AA  o5:(N8s6-_(_$_$_$<"_)"_./`. _>  ` cR^^^^^^^SmUUUUUU4Sjn[TR5n[U5nTR5Hup[ U S5(aZUHRmTR S5upn X:waM[ XS5cM-U"UTU 5 TRRT5 MT MpT"U5n U RSS9HSupU SU 3mTU;d[U4SjU55(dM.U"UTU 5 TRRT5 MU M g) z1Inject layers based on peft_config.target_modulescRSnX;aURU5up#nX$-nX;aMU$)Nz .base_layer) rpartition)rMrJrr$r;s rTstrip_base_layer_from_name@BaseTuner._inject_parameters..strip_base_layer_from_names; !D%$/$:$:4$@!6$o % rnc >[T U5up4nT"U5nT RU5n[U[5(a?URR S:wa%[ SUS[U5R S35eT RT T U5 T (a[O[nU"5 T RT T UUUUURS5SS9 SSS5 g!,(df  g=f)N ParamWrapperz+Trying to wrap an `nn.Parameter` of layer 'z ' of type z, which is not a valid target. Make sure that this layer is not also targeted with `target_modules`. For some models, PEFT will do this automatically, try setting `target_modules=[]` to prevent it.r:r=)rr) r' get_submoduler4rrrrfrr1rrrrg)rMr param_namerrriunwrapped_module_nameunwrapped_moduler]rrrhrgrrhs rTcreate_and_replace_param>BaseTuner._inject_parameters..create_and_replace_params*9%*M 'FK$>{$K !$223HI *N;;@P@Z@Z@c@cgu@u ABWAXXbF|,,-.EE  2 2;{ S(9${C((  ##-#8#8#=b#A)s +(C C*parametrizationsr:NFrecursec3L># UHnTRSU35v M g7fr9rr target_keyrs rTr/BaseTuner._inject_parameters..s'3reqWaCLL1ZLAQ4R4Req!$) rr7rNr2r3rgrerr7rr)rrgrhrrrpunsorted_target_names target_namesrMrKtarget_module_namer$rmrnrSrrhs````` @@rTrMBaseTuner._inject_parameterss%   8!$K$A$A B34 #(#6#6#8 Kv122(C8;s8K5&:)8 v48@ ,[#zJ1188=()C;(O%)/)@)@)@)O%J231ZLAC|+3req3r0r0r1c:N55<# UHoRT:Hv M g7frr9rrr.s rTr,BaseTuner._replace_module..4sI5H88t+5H)setattrr3r,rrrerrGr9r8r2rrr{rnext parametersr) rr child_name new_modulechildrJrKrr.s @rTr!BaseTuner._replace_moduless" J/ 5, ' '$$Ez<00 % J uf%%"'**  5'4 ( 4z<00.3kk %%+#(;;  MM%,,-- .||F#&446LD{{d"5),,"]]FUE**"YYFUH--"\\FU$4d;G"11F!%"2"2"45FIV5F5F5HIIIIIfmm,7rncUR5 URR5H=n[U[5(dM[ U5 UR XS9 SSS5 M? g!,(df  MQ=f)a This method merges the adapter layers into the base model. Merging adapters can lead to a speed up of the forward pass. A copy of the adapter weights is still kept in memory, which is required to unmerge the adapters. In order to merge the adapter weights without keeping them in memory, please call `merge_and_unload`. Args: adapter_names (`list[str]`, *optional*): The list of adapter names that should be merged. If `None`, all active adapters will be merged. Defaults to `None`. safe_merge (`bool`, *optional*): If `True`, the merge operation will be performed in a copy of the original weights and check for NaNs before merging the weights. This is useful if you want to check if the merge operation will produce NaNs. Defaults to `False`. )rrN)rrhrr4rrUr)rrrrKs rT merge_adapterBaseTuner.merge_adapter7sY. !!#jj((*F&.11!&)LL}LT*)+))s A-- A< cURR5H>n[U[5(dM[ U5 UR 5 SSS5 M@ g!,(df  MR=f)zE This method unmerges all merged adapter layers from the base model. N)rhrr4rrUunmerge)rrKs rTunmerge_adapterBaseTuner.unmerge_adapterTsIjj((*F&.11!&)NN$*)+))s A A- c[XUS9 g)a Sets the active adapter(s) on auxiliary modules. If the subclass (e.g. `LoraModel`) supports auxiliary modules like `modules_to_save`, it should call this method in `set_adapter` to ensure that those auxiliary modules are being set correctly. Args: adapter_name (`str` or `list[str]`): The name(s) of the adapter(s) to be set as active. The adapters must be loaded first. inference_mode (bool, optional): Whether the activated adapter should be frozen (i.e. `requires_grad=False`). Default is False. rAN)rrrrBs rTset_auxiliary_adapters BaseTuner.set_auxiliary_adapters]s TGrncN[URXURS9 Xlg)a Set the active adapter(s). Args: adapter_name (str, list[str]): The name(s) of the adapter(s) to set as active inference_mode (bool, optional): Whether the activated adapter should be frozen (i.e. `requires_grad=False`). Default is False. )rrBrN)rPrhrrrs rTrPBaseTuner.set_adapterls&  JJ\\`\p\p +rnc[US[5n[US5(aUR5nU$[R "U5(a[R "U5nU$)a< This method gets the config from a model in dictionary form. If model has not attribute config, then this method returns a default config. Args: model (`nn.Module`): Model to get the config from. default (`dict|None`, *optional*):: What to return if model does not have a config attribute. rbto_dict)rerr3r dataclasses is_dataclassasdict)rhrs rTrEBaseTuner.get_model_configzs_uh0BC < + +'//1L % %l 3 3&--l;Lrnc/nURU5nURS5(aBURH2nURS5S[;dM!UR U5 M4 U$)Ntie_word_embeddingsr:r=)rErrrCrr7)rrhrr target_modules rTr"BaseTuner._get_tied_target_modulessk ,,U3   1 2 2!%!;!; !&&s+B/3HH'..}= "< #"rnc[U5$r)rrs rTr/BaseTuner._get_module_names_tied_with_embeddings 4T::rnc4Sn[R"U5 g)z This method adds modules to tie to `peft_config` so that those modules can be tied downstream. By default this method raises a warning, and each tuner class extending `BaseTuner` can choose to implement this. KModel has `tie_word_embeddings=True` and a tied layer is part of the adapter, but no implementation exists to tie the adapters. This can lead to complications, for example when merging the adapter or converting your model to formats other than safetensors. Check the discussion here: https://github.com/huggingface/peft/issues/2777N)rr)rrgtied_weight_keysrs rT_add_modules_to_tieBaseTuner._add_modules_to_ties  Y   crnc[[US/5=(d /5n[SU55nUR5n[USS5(adU(aU(aUR X%5 g U(dU(a[ R "S5 g U(d[ R "S5 g g U(aKU(aC[US5(aSn[ R "U5 g Sn[ R "U5 g g g ) z Checks if any of the tied layers are targetted via `modules_to_save`. Updates the `peft_config.modules_to_tie` with any layers that needs to be tied modules_to_savec32# UH o[;v M g7fr)r)rrs rTr0BaseTuner._check_tied_modules..s"W!(=#=sensure_weight_tyingFz\You have requested `ensure_weight_tying`, but no tied modules are added in `modules_to_save`zUYou have requested `ensure_weight_tying`, but no tied modules were found in the modelaGModel has `tie_word_embeddings=True` and a tied layer is part of the adapter, but `ensure_weight_tying` is not set to True. This can lead to complications, for example when merging the adapter or converting your model to formats other than safetensors. Check the discussion here: https://github.com/huggingface/peft/issues/2777rN)rrerrrrrr3)rrhrgris_embedding_to_saverrs rTrDBaseTuner._check_tied_moduless gk3DbIORP""W"WWEEG ; 5u = =#(8((G).> r& uv&"&6{$9::a c"a c"%'7 !rnc~>[TU]U5$![a US:Xae[URU5s$f=f)z1Forward missing attributes to the wrapped module.rh)r __getattr__r rerh)rrJrs rTrBaseTuner.__getattr__sB -7&t, , -w4::t, , -s '<<)rrhrgrrFN) rgz(Union[PeftConfig, dict[str, PeftConfig]]rrrboolr!Optional[dict[str, torch.Tensor]]returnNoner list[str])rr rr )rh nn.Modulerbr&rrrr)rgr&rr?rr&)rgr&rhr)rgr&rrrbool | re.Match[str] | Noner)rgr&rrrrrirrrrrrz Optional[str]rr)rhrrrTrrrrrrrrrrrzstr | Sequence[str]rrrr)rbr&rr)rrr rrr)TFFN) rrr#rrrrOptional[list[str]]rr)FFN)r#rrrrrrtorch.nn.Module)rrrgr&rhrrir)TFN) rhrrrr rrrrrrr) rgr&rhrrrrrrrNF)rrrrrr)rstr | list[str]rBrrrF)rhrrr?)rhrrr)rhr)rJr).r __module__ __qualname____firstlineno____doc____annotations__rpropertyrrrrr staticmethodrrrrrrrrrr rrr'r+r.r1r4rrMr!rrrrPrErrrrDr__static_attributes__ __classcell__)rs@rTrrs$R K))0/#(8< #2>#2 #2  #2 6 #2 #2#2J## 3  8  <<")-              &       DZ00 0"/0 `2 q"L! -1   +   @im$ $ 59$ Re$ $ L>J i i(,"'8< W W W !% W  W 6 W  W rIB%IB.7IBGJIB_cIB IBV5-nU:% H +$ #; (#T--rnrc\rSrSr%SrSrS\S'SrS\S'SrS\S 'S r S \S '/r S \S'S%Sjr Sr \ S&Sj5r\ S&Sj5rS'S(SjjrS)Sjr\ S*Sj5r\ S*Sj5r\ S+Sj5rS,Sjr\ S5rS-SjrS.S/SjjrS0SjrS1SjrS2S3SjjrS4S5S jjr\S6S!j5r\S7S"j5rS8S#jrS$rg)9ria/ A tuner layer mixin that provides the common methods and attributes for all tuners. Args: is_pluggable (`bool`, *optional*): Whether the adapter layer can be plugged to any pytorch module active_adapters (Union[List[`str`], `str`], *optional*): The name of the active adapter. rztuple[str, ...]adapter_layer_namesother_param_namesFr_disable_adaptersdefaultr_active_adapterrmerged_adapterscjUn[US5(aURn[US5(aMU$)zt (Recursively) get the base_layer. This is necessary for the case that the tuner layer wraps another tuner layer. r,)r3r,rr,s rTr"BaseTunerLayer.get_base_layers6 j,//#..Jj,//rncUR5n[US5(dgURn[U[[ 45(a=[ R"X!RRURRS9$[U[ R5(a@UR5S:XaU$[R"SURS3[ 5 gg)a Extract embed_scale from base layer if present and valid. Some embedding layers (e.g., Gemma3TextScaledWordEmbedding) apply scaling to embeddings in their forward method. This method checks for the presence of an `embed_scale` attribute. If it exists, it is assumed to be a scalar. Its shape is validated accordingly. Returns: torch.Tensor or None: The embed_scale tensor if found and valid, None otherwise. embed_scaleN)r9dtyper(z'Found embed_scale attribute with shape z, expected scalar. Embedding scaling will not be applied. If this is unexpected, please open an issue at https://github.com/huggingface/peft/issues)r"r3rr4intfloatr8rFrr9rTensornumelrrrr$)rr,rs rT_get_embed_scaleBaseTunerLayer._get_embed_scales((* z=11 ,,  kC< 0 0<< 4E4E4L4LT^TeTeTkTkl l k5<< 0 0  "a'"" =k>O>O=PQAA  rnc|UR5n[US5(aURnU$URnU$)Nr)r"r3rr)rr,rs rTrBaseTunerLayer.weight(sC((* :y ) )''F  &&F rnc:UR5nUR$r)r"rrs rTrBaseTunerLayer.bias8s((* rnNc[err)rrrs rTrBaseTunerLayer.merge=!!rnc[errrs rTrBaseTunerLayer.unmerge@rrnc,[UR5$r)rrrs rTmergedBaseTunerLayer.mergedCsD(())rncUR$r)rrs rTdisable_adaptersBaseTunerLayer.disable_adaptersGs%%%rncUR$r)rrs rTrBaseTunerLayer.active_adapterLs###rnc[5nURHgn[X5n[U[R [R 45(dM?UR[UR555 Mi U$)z:Return all adapter names that can be found on this module.) rrrer4r ModuleDict ParameterDictrr@)radapters layer_namerKs rT_get_available_adapters&BaseTunerLayer._get_available_adaptersQs`522JT.Ffr}}b6F6F&GHH OOC . / 3 rncr[UR[5(a UR/$UR$rrrs rTrBaseTunerLayer.active_adapters[rrncU(a#URUR5 SUlgURHn[ X5nUR S5 M! SUlg)zToggle the enabling and disabling of adapters Takes care of setting the requires_grad flag for the adapter weights. Args: enabled (bool): True to enable adapters, False to disable adapters FTN)rPrrrrerequires_grad_)rrrrHs rTrBaseTunerLayer.enable_adaptersbsW    T11 2%*D "#66 1$$U+7&*D "rnc[U[5(aU/nURHWn[X5nUR 5H5upVXQ;aU(dUR S5 M$UR S5 M7 MY Xlg)aSet the active adapter(s). Additionally, this function will set the specified adapter to trainable (i.e., requires_grad=True) unless inference_mode is True. Args: adapter_name (`str` or `list[str]`): The name(s) of the adapter(s) to set as active. inference_mode (bool, optional): Whether the activated adapter should be frozen (i.e. `requires_grad=False`). Default is False. TFN)r4rrreitemsrr)rrrBr module_dictrrHs rTrPBaseTunerLayer.set_adaptertsw mS ) )*OM22J!$3K)//1 (>((.((/ 23 -rnc[5nURUR-H@n[X5n[ US5(dM!UR UR 55 MB [U5$)z3Return a sorted list of all available adapter namesr@)rrrrer3rr@rN)rrrJattrs rT_all_available_adapter_names+BaseTunerLayer._all_available_adapter_namess_ ,,t/E/EED4&DtV$$$$TYY[1 F m$$rncURUR-H nU[X5;dM[X5U M" XR;aURSSnUR U5 U(aUR U5 gUR 5nU(dUR /5 gUSn[R"SUSUS35 UR US5 gg)a Delete an adapter from the layer This should be called on all adapter layers, or else we will get an inconsistent state. This method will also set a new active adapter if the deleted adapter was an active adapter. It is important that the new adapter is chosen in a deterministic way, so that the same adapter is chosen on all layers. Args: adapter_name (`str`): The name of the adapter to delete Nrrz< was active which is now deleted. Setting active adapter to r:) rrrerremoverPr rr)rrr rremaining_adaptersnew_active_adapters rTrBaseTunerLayer.delete_adapters,,t/E/EEDwt22D' 5F // /"2215O  " "< 0  1&*%F%F%H")$$R();A)>&MM"<.0l-.a1$$%7%:;% 0rnc[U[5(aU1nO [U5nURH?n[ X5nUR 5HupgXc;dM UR U5 M MA g)rN)r4rrrrerr)rrradapter_names_setrrrrHs rTr BaseTunerLayer.set_requires_gradsg mS ) )!.  #M 2 22J!$3K)//1 +((723rnc^ UcnUR5n[U[R5(a URnSH,n[ X4S5nUcMUR nURn O g[R "S5m URUR-Hn[ XS5n[U[R[R[45(dMEX;aML[U 4SjUR555(aMvWRR (dURR"(aXR%UWS9X'MXR%U5X'M g)zE Move the adapter of the given name to the device of the base layer. N)rrr.c3@># UHoRT:Hv M g7frrrs rTrGBaseTunerLayer._move_adapter_to_device_of_base_layer..sH-G88t#-Grr)r"r4rrr\rer9rr8rrrrr)rris_floating_point is_complexrG) rrr9r, weight_namerradapter_layer_name adapter_layerr.s @rT%_move_adapter_to_device_of_base_layer4BaseTunerLayer._move_adapter_to_device_of_base_layers9 >,,.J*b&;&;<<'00 4  $?%#]]F"LLE 5||F# #'":":T=S=S"S #DdCMmbmmR=M=Mz-Z[[0H]-E-E-GHHH||--1H1H.;.I.L.LV[`.L.a +.;.I.L.LV.T +#Trncgrrrxrs rT_cast_input_dtype BaseTunerLayer._cast_input_dtypesFIrncgrrr!s rTr#r$sVYrnctUcg[USS5nU(aURU:XaU$URUS9$)aT Whether to cast the dtype of the input of the forward method. Usually, we want to enable this to align the input dtype with the dtype of the weight, but by setting layer.cast_input_dtype=False, this can be disabled if necessary. Enabling or disabling can be managed via the peft.helpers.disable_lora_input_dtype_casting context manager. Ncast_input_dtype_enabledTr)rerrG)rr"rr's rTr#r$s@ 9#*41KT#R (agg.>Htt%t  rn)rr)rr)r torch.Tensorr)rrrrrrr)rr)rr)rset[str]rr)rrrBrrrrrrrr)rrr9zOptional[torch.device]rr)r"rr torch.dtyperr)r"r(rr*rr()rr*) rrrrrrrrrrrr"rrrrrrrrrrrrrPr rrrr r#rrrnrTrrs,.-)++$t#(1O_0"$OY# #J  ""**&&$$## *$-8 %#>> from peft.tuners.tuners_utils import _find_minimal_target_modules >>> target_modules = [f"model.decoder.layers.{i}.self_attn.q_proj" for i in range(100)] >>> target_modules += [f"model.decoder.layers.{i}.self_attn.v_proj" for i in range(100)] >>> other_module_names = [f"model.encoder.layers.{i}.self_attn.k_proj" for i in range(100)] >>> _find_minimal_target_modules(target_modules, other_module_names) {"q_proj", "v_proj"} ``` Args: target_modules (`list[str]` | `set[str]`): The list of target modules. other_module_names (`list[str]` | `set[str]`): The list of other module names. They must not overlap with the target modules. Returns: `set[str]`: The minimal set of target modules that is sufficient to separate them from the other modules. Raises: ValueError: If `target_modules` is not a list or set of strings or if it contains an empty string. Also raises an error if `target_modules` and `other_module_names` contain common elements. z2target_modules should be a list or set of strings.r+z2target_modules should not contain an empty string.ztarget_modules and other_module_names contain common elements, this should not happen, please open a GitHub issue at https://github.com/huggingface/peft/issues with the code to reproduce this issuecURS5n[[U55Vs/sHnSRXS5PM snSSS2$s snf)Nr:r=)rCrangerrD)spartsrPs rTgenerate_suffixes7_find_minimal_target_modules..generate_suffixesGsH -23u:->?->r#->?"EE?sAc US$)Nr(r)tups rT._find_minimal_target_modules..Vs TWXYTZrn)rc3L># UHnTRSU-5v M g7fr9rv)r req_suffixitems rTr/_find_minimal_target_modules..]s$[IZ:t}}S:%566IZrz) r4rrfr isdisjointrNrradd) rother_module_namesrr0r8r;other_module_suffixestarget_modules_suffix_maprequired_suffixesrUs ` rTrIrIsPF.#&&nMNN(N ^MNN/0  $ $%7 8 8 v oF 1Ci0BQbcgQhvVQhV0BiLZ Z>4'8'>!>> Z!!:!@!@!BHZ[hF*f.M[IZ[[[!%%f-\ >"" /j![s ;D4D:c\rSrSrSrSrSrg)rLifzh A private helper method used to represent excluded modules in the check_target_module_exists function. cgrrrs rT__bool___ExcludedModule.__bool__ksrnrN)rrrrrrBrrrnrTrLrLfs rnrLc^[US5(aUR(a[UR[5(a1[R "URT5(a [ 5$OHTUR;a [ 5$[U4SjUR55(a [ 5$[USS5nU(a$[U4SjU55(a [ 5$URcURbg[UR[5(a[URT5nU$TUR;aSnU$[U4SjUR55n[US S5n[US S5nUSL=(a% [U[5(a[U5S :gOSnU(aU(aSnUb[U5S :Xa[R"S T5nOC[U[5(aU/OUnUH#n[R"S US3T5nUcM# O UcSnU$[UR!S55n[U[5(aXt:HnU$Xt;nU$)aA helper method to check if the passed module's key name matches any of the target modules in the adapter_config. Args: config (`PeftConfig`): A config to match target modules from. key (`str`): A key to search any matches in config Returns: `bool` | `re.Match[str]` | `None`: True or re.Match object if key matches any target modules from config, False or None if no match found. r@c3L># UHnTRSU35v M g7fr9rv)r exclude_keyrs rTr-check_target_module_exists..s&[DZ[+/00DZrzrNc3Z># UH n[R"SUS3T5v M" g7f)z(^|.*\.)z($|\..*)N)rematch)rrrs rTrrGs(O!rxx8A3h/55s(+FTc3L># UHnTRSU35v M g7fr9rvrws rTrrGs&!iShZ#,,:,/?"@"@Shrzr>r?rz.*\.[^.]*\.(\d+)\.z.*\.z \.(\d+)\.r()r3r@r4rrI fullmatchrLrrerr7r r>rrJrgroup) rbrrtarget_module_found layer_indexesr?is_using_layer_indexes layer_indexpatterns ` rTrrosXv())f.D.D f,,c 2 2||F22C88&((9 F** *"$ $ [FDZDZ[ [ ["$ $f&7>O OO O O"$ $%F,D,D,P&''--6v7L7LcRF E %% %"@ ="!iSYShSh!ii(=tD  )94@!.d!:" '1-'F'FC ! #D  "&9K%^)<)A hh'"="=H> r2K  , ,V 9 9  " ) )# .  $ + +C 0  ?sBc^[US5(dU$[UR[5(a"URR 5[ :XdU$[ RR[4nSn[5nUR5Hnunm[TU5(aURU5 M*[T[5(dMA[U4SjU55(dM]URU5 Mp [5n[U[5(aUR!5nUb@UR5VVs/sH upXXLdM UPM snnSn URU 5 O{UR"[$R&:Xa][(HSn[+XS5n U cMUR5VVs/sH upXXLdM UPM snnSn URU 5 O UR5HUun m[T[5(dMTR5H$upU (dMURU SU 35 M& MW XF-nX@lU$s snnfs snnf)z Helper function to update `target_modules` to all linear/Conv1D layers if provided as 'all-linear'. Adapted from the QLoRA repository: https://github.com/artidoro/qlora/blob/main/qlora.py r)rc3R># UHo[T5R;v M g7fr)rr)rrrKs rTr3_maybe_include_all_linear_layers..s!7i\hWXT&\=R=R8R\hs$'Nrr:)r3r4rrlowerrr8rrrrr2r;rrrget_output_embeddings task_typer#SEQ_CLSrre)rgrhlinear_classes linear_nameslinear_module_namesrJmodule_names_to_exclude output_embrKlast_module_namecls_headrr;rs ` rTrFrFs! ; 0 1 1 ;--s33  & & , , .2Q Qhhoov.NL%++- f fn - -  # #D )  / /C7i\h7i4i4i # #D )."e%))002  !9>9L9L9Ng9NRXRf9Nghij  # ' '(8 9  " "h&6&6 6+"55'AFATATAV'mAVZ`ZlAV'mno'p$+//0@A + --/ fn - -!'!5!5!7 6+//6(!F80DE"80 2!4 - h(ns I(I( I.I.cUc URn[U[5(a[SU<S35eUR(a[ UR 5nUVs/sH o3U;dM UPM nnU(aG[R"SSRUR 5SSRU5S35 U$[R"S5 U$s snf)z Helper function to check which adapters should be merged. Only return those adapters that are not already merged. Give a warning if some or all of the adapters are already merged. z/adapter_names should be a list of strings, got r:z'Already following adapters were merged ,z#. You are now additionally merging z/All adapters are already merged, nothing to do.) rr4rrfrrrrrrD)rKrrrJs rTcheck_adapters_to_mergerhs.. -%%J=J[[\]^^ }}f445*7W-$;V- W  MM9#((6CYCY:Z9[\447HH]4K3LAO   MMK L Xs  C,CFc[R"U5nSSjnU(a0UR5HupEU"XRRU55 M U$)zClone a module in a pytorch model. Clones a module of a model, optionally sharing all the parameters between the original and the clone. Simplifies reusing a module when manipulating the architecture of a model. cVURSS9Hup#URX#5 M g)NFrs)rregister_parameter)srcdstrJrSs rT_share_weights$clone_module.._share_weights)s*///>KD  " "4 /?rn)rlrrmr)copydeepcopyr2rl)rK share_weightsclonernrJ submodules rT clone_moduleru!sK MM& !E0%335OD 9&9&9$&? @ 6 Lrnc [US5(aURn[US5(aM[US5(a URnSnSn[US5(aSnURnOd[US5(a4[URS5(aSnURR nO[US5(aS nUR nU(a[U[R5(d [S 5e/nUHpupV[XV5H\n[U5nUR[X7S S 95 US R5Hn [U S5(dMXlM M^ Mr [R"U5nUS:XaX0lO/US:XaX0RlOUS :XaX0lO [S5e[UR"S5(a[U5UR"lgg)anReplicate layers in a transfomer model with weight sharing. This function looks for a module list attribute at model[(.model)*].layers and replicates the layers in the module list according to the layer map. For example the map `[[0, 4], [2, 5]]` will take the set of layers `[0, 1, 2, 3, 4]` and replace them with a module list containing `[0, 1, 2, 3, 2, 3, 4]`. rhbertNlayersllamaencoderrHhfalconzlCould not locate the layers attribute in the model. Expected Llama, Bert or Falcon compatible architectures.T)rrr= layer_idxz@Unexpected model type, need to handle post-processing of layers.num_hidden_layers)r3rhrwrxrzrHr{r4r ModuleListrfr-rr7rurr}rbr~) rh layer_maprcrx new_layersstartendrP current_idxrts rTreplicate_layersr4s % ! !  % ! !uf J Fuh   " "wu}}g'F'F $$     Z >> G  J u"Aj/K   l69DI J'^335 9k22*5'6 # ]]: &FW v $  x [\\u||011),Z &2rnc[XUS9 UR5H[n[XC5(dMUR(a&[R "S5 UR 5 URXS9 M] g)aeSet the active PEFT adapter(s) of the model. Active adapters are those adapters that participate in the forward pass. Use this function if you want to switch between multiple PEFT adapters. Args: model (`nn.Module`): The model on which the adapter(s) should be set. adapter_name (str, list[str]): The name(s) of the adapter(s) to set as active inference_mode (bool, optional): Whether the activated adapter should be frozen (i.e. `requires_grad=False`). Default is False. layer_cls (type, optional): The class of the adapter layer. Defaults to `BaseTunerLayer`. rAzJAdapter cannot be set when the model is merged. Unmerging the model first.N)rrr4rrrrrP)rhrrBrrKs rTrPrPmsX*^D--/ f ( (}} jk   |  K "rnc~UR5H)n[U[5(dMURXS9 M+ g)N)new_active_adapters)rr4rr)rhrrrKs rT_delete_auxiliary_adapterrs1--/ f6 7 7  ! !, ! X"rncUR5VVs/sHupEX$;dM UPM nnnSnUHHn[X5upXn[X5(dM#URU5 UbM9URSSnMJ [ XUS9 U$s snnf)as Delete an existing PEFT adapter. Note: This function does not delete the PEFT config on the model, if there is one. It will also not completely purge the PEFT layers if the last PEFT adapter is deleted. For this, consider using `model.unload()` if using a PEFT model instance, or just reloading the base model. Args: model (`nn.Module`): The model from which the adapter should be deleted. adapter_name (str): The name of the adapter to be deleted. prefix (str): The prefix of the PEFT method, e.g. "lora_" for LoRA. layer_cls (type, optional): The class of the adapter layer. Defaults to `BaseTunerLayer`. Returns: new_adapter (list[str] | None): The name of remaining adapter(s) after deletion, or `None` if there are no active adapters left. Use this to set the new active adapter of the model if necessary. N)rr)r2r'r4rrr) rhrrrrr$r%rrs rTrrs2#("5"5"7M"76;L"7HMK&u2 1 f ( (  ! !, /"$44Q7 eT_` Ns BBcU(dg[R[R1nUR5GHn[ U[ 5(dMUR5GHZn[ U[ R[ R[45(dM:X;aMA[ XQ[ R5(aFXQRU;a2XQRR5XQl M[ XQ[R5(a8XQRU;a#XQR5XQ'GMXQR!5HCnURU;dMURR5Ul ME GM] GM g)a1 A helper method to cast the adapter weights to the correct dtype. Currently, this only upcasts float16 and bfloat16 to float32. Args: adapter_name (`str`): The adapter name. autocast_adapter_dtype (`bool`, *optional*): Whether to autocast the adapter dtype. Defaults to `True`. N)r8float16bfloat16rr4rrrrr) ParameterrdatarGfloat32rr)rhrr dtypes_to_convert_to_fp32rKrtrSs rTrrsE "!& ?--/&.11 )Ii"--9I9I:)VWW,)12<<@@*004MM3<3J3O3O3R3RSXS`S`3aI+0)15<<@@*004MM.7.E.H.H.WI+"0;;=;;";;!&u}}!=EJ>#* "rncUR5H/n[U[[45(dM UR XS9 M1 g)aw Enable or disable gradients on the given adapter(s). Args: model (`nn.Module`): The model from which the adapter should be deleted. adapter_name (`str` or `Sequence[str]`): The name of the adapter(s) whose gradients should be enabled/disabled. requires_grad (`bool`, *optional*) Whether to enable (`True`, default) or disable (`False`). rN)rr4rrr)rhrrrKs rTrrs8--/ f~/GH I I  $ $= $ ^"rnr)rKrrz#tuple[int, int] | tuple[None, None])rlist[str] | set[str]r<rrr))rrrr)r)rVrrrrr?)rgr&rhrrr&r)rKrrrrrr)rKr)rhrrzlist[tuple[int, int]])rrrBrrrrr)rrrrrr) rhrrrrrrrrzlist[str] | Noner)rhrrrr rrrr)Y __future__rrprrArIrrabcrrcollections.abcr contextlibrrtypingr r r r r8accelerate.hooksr accelerate.utilsrr packagingrrr transformersrtransformers.pytorch_utilsr peft.mappingr peft.utilsrpeft.utils.constantsrrrrrpeft.utils.integrationsrpeft.utils.otherrrrr r!peft.utils.peft_typesr"r#peft.utils.warningr$rbr&utilsr' _buffer_dictr)parse __version___torch_supports_dtensorr is_availablerrUrmrModulerrrIrLrrWrFrhrurrPrrrrrrnrTrs#  #$211 -E(-4675*#$"--(9(9:gmmG>TT5Z%:K:K:X:X:Z@S@SF,:%zC- 3C-L s!Ss!l O(O>RO OdEP <~8&19x!&4 L!LL$ L  L<YYg$ $$'$14$AU$$N(>V_rn