oi"hSSKJr SSKJrJr SSKJrJr SSKJ r SSK J r \"SS\ 55r g) ) annotations) dataclassfield)OptionalUnion) PeftConfig)PeftTypec~^\rSrSr%Sr\"SSS0S9rS\S'\"S SS 0S9rS\S '\"S SS 0S9r S\S'\"SSS0S9r S\S'\"SSS0S9r S\S'\"SSS0S9r S\S'\"SSS0S9r S\S'\"SSS0S9rS\S '\"SSS!0S9rS\S"'\"\SS#0S$9rS%\S&'\"\SS'0S$9rS%\S('\"SSS)0S9rS*\S+'U4S,jrS-rU=r$). DeloraConfiga This is the configuration class to store the configuration of a [`DeloraModel`]. Args: r (`int`): The rank of the DeLoRA adapter. delora_lambda (`int`): The initial value of the boundary of the DeLoRA adapter. This variable sets an upper bound to the Frobenius norm of the weight change, avoiding the finetuned model to deviate too much from the original model. module_dropout (`float`): The dropout probability for disabling DeLoRA modules during training. target_modules (`Optional[Union[List[str], str]]`): The names of the modules to apply the adapter to. If this is specified, only the modules with the specified names will be replaced. When passing a string, a regex match will be performed. When passing a list of strings, either an exact match will be performed or it is checked if the name of the module ends with any of the passed strings. If this is specified as 'all-linear', then all linear/Conv1D modules are chosen, excluding the output layer. If this is not specified, modules will be chosen according to the model architecture. If the architecture is not known, an error will be raised -- in this case, you should specify the target modules manually. exclude_modules (`Optional[Union[List[str], str]]`): The names of the modules to not apply the adapter. When passing a string, a regex match will be performed. When passing a list of strings, either an exact match will be performed or it is checked if the name of the module ends with any of the passed strings. bias (`str`): Bias type for DeLoRA. Can be 'none', 'all' or 'delora_only'. If 'all' or 'delora_only', the corresponding biases will be updated during training. Be aware that this means that, even when disabling the adapters, the model will not produce the same output as the base model would have without adaptation. init_weights (`bool`): Whether to perform initialization of adapter weights. If `True` (default): A is initialized with kaiming uniform initialization, while B is initialized with zeros. If `False`: A and B are both initialized with kaiming uniform, immediately contributing a non-zero delta. This is generally discouraged for normal use. layers_to_transform (`Union[List[int], int]`): The layer indices to transform. If a list of ints is passed, it will apply the adapter to the layer indices that are specified in this list. If a single integer is passed, it will apply the transformations on the layer at this index. layers_pattern (`Optional[Union[List[str], str]]`): The layer pattern name, used only if `layers_to_transform` is different from `None`. This should target the `nn.ModuleList` of the model, which is often called `'layers'` or `'h'`. rank_pattern (`dict`): The mapping from layer names or regexp expression to ranks which are different from the default rank specified by `r`. For example, `{'^model.decoder.layers.0.encoder_attn.k_proj': 16}`. lambda_pattern (`dict`): The mapping from layer names or regexp expression to lambdas which are different from the default lambda specified by `delora_lambda`. For example, `{'^model.decoder.layers.0.encoder_attn.k_proj': 16}`. modules_to_save (`Optional[List[str]]`): List of modules apart from adapter layers to be set as trainable and saved in the final checkpoint. helpz DeLoRA rank)defaultmetadataintrzThe initial value of the boundary of the DeLoRA adapter. This variable sets an upper bound to the Frobenius norm of the weight change, avoiding the finetuned model to deviate too much from the original model. delora_lambdagzDThe dropout probability for disabling DeLoRA modules during trainingfloatmodule_dropoutNaList of module names or regex expression of the module names to replace with DeLoRA.For example, ['q', 'v'] or '.*decoder.*(SelfAttention|EncDecAttention).*(q|v)$' This can also be a wildcard 'all-linear' which matches all linear layers except the output layer.zOptional[Union[list[str], str]]target_moduleszTList of module names or regex expression of the module names to exclude from DeLoRA.exclude_modulesnonez,Bias type for DeLoRA. Can be 'none' or 'all'strbiasTa;Whether to perform initialization of adapter weights. If `True` (default): A is initialized with kaiming uniform initialization, while B is initialized with zeros. If `False`: A and B are both initialized with kaiming uniform, immediately contributing a non-zero delta. This is generally discouraged for normal use.bool init_weightszThe layer indexes to transform, is this argument is specified, PEFT will transform only the layers indexes that are specified inside this list. If a single integer is passed, PEFT will transform only the layer at this index.zOptional[Union[list[int], int]]layers_to_transformzThe layer pattern name, used only if `layers_to_transform` is different to None and if the layer pattern is not in the common layers pattern. This should target the `nn.ModuleList` of the model, which is often called `'layers'` or `'h'`.layers_patternzThe mapping from layer names or regexp expression to ranks which are different from the default rank specified by `r`. For example, `{'^model.decoder.layers.0.encoder_attn.k_proj': 16}`.)default_factoryrzOptional[dict] rank_patternzThe mapping from layer names or regexp expression to lambdas which are different from the default lambda specified by `delora_lambda`.lambda_patternaList of modules apart from DeLoRA layers to be set as trainable and saved in the final checkpoint. For example, in Sequence Classification or Token Classification tasks, the final layer `classifier/score` are randomly initialized and as such need to be trainable and saved.zOptional[list[str]]modules_to_savec>[TU]5 [RUl[ UR [5(a[UR 5O UR Ul[ UR [5(aURb [S5eUR(aUR(d [S5egg)NzD`layers_to_transform` cannot be used when `target_modules` is a str.zRWhen `layers_pattern` is specified, `layers_to_transform` must also be specified. ) super __post_init__r DELORA peft_type isinstancerlistsetrr ValueErrorr)self __class__s S/home/james-whalen/.local/lib/python3.13/site-packages/peft/tuners/delora/config.pyr&DeloraConfig.__post_init__s !(243F3F(M(MC## $SWSfSf  d))3 / /D4L4L4Xcd d   t'?'?qr r(@ )r(r)__name__ __module__ __qualname____firstlineno____doc__rr__annotations__rrrrrrrrdictr!r"r#r&__static_attributes__ __classcell__)r.s@r/r r s.`1 '> ?As? } M3"v'mnNE7< p 7N38=pq8O4f8f/ghD#h g L$rAs7#("" Bs:Bs Bsr1