oiSSKJr SSKrSSKJrJr SSKJrJrJ r SSK J r SSK J r SSKJr \"S S \ 55rg) ) annotationsN) dataclassfield)LiteralOptionalUnion) PeftConfig)PeftType) random_maskc^\rSrSr%Sr\"SSS0S9rS\S'\"S SS 0S9rS \S '\"S SS0S9r S\S'\"S SS0S9r S\S'\"SSS0S9r S\S'\"SSS0S9r S\S'\"S SS0S9r S\S'U4SjrSrU=r$) ShiraConfiga~ This is the configuration class to store the configuration of a [`ShiraModel`]. Args: r (`int`, *optional*, defaults to `32`): For a given target module, the number of SHiRA parameters is computed as r(m+n), where the original tensor dimensions are m x n. This means the number of SHiRA parameters is the same as that for a LoRA adapter. SHiRA is a high rank adapter. Setting this r parameter does not restrict the rank to this value. mask_type (`str`, defaults to `random`): Type of mask function. Defaults to a random sparse mask. An optional user-defined mask_fn to compute the mask value can also be supplied by instantiating `config = ShiraConfig(...)` and then setting `config.mask_fn = `. For a pretrained weight with shape m x n, the custom mask function must return only one mask (shape: m x n) which must be binary 0 or 1 with num_shira_parameters = r(m + n) for linear layers. Device and dtype of mask must be same as base layer's weight's device and dtype. Please see mask_functions.py for more details and to see the default random sparse mask implementation. random_seed (`int`, *optional*, defaults to `None`): random seed for the torch generator for random_mask. target_modules (`Union[List[str], str]`): List of module names or regex expression of the module names to replace with SHiRA. For example, ['q', 'v'] or '.*decoder.*(SelfAttention|EncDecAttention).*(q|v)$'. Only linear layers are supported. fan_in_fan_out (`bool`): Set this to True if the layer to replace stores weight like (fan_in, fan_out). For example, gpt-2 uses `Conv1D` which stores weights like (fan_in, fan_out) and hence this should be set to `True`. init_weights (`bool`, defaults to `True`): Initialize SHiRA weight to have zero values. If set to False, SHiRA weights are initialized to randn values instead of zeros and this is used only for testing. modules_to_save (`List[str]`): List of modules apart from SHiRA layers to be set as trainable and saved in the final checkpoint. helpa3For a given target module, the number of SHiRA parameters is computed as r(m+n), where the original tensor dimensions are m x n. This means the number of SHiRA parameters is the same as that for a LoRA adapter. SHiRA is a high rank adapter. Setting this r parameter does not restrict the rank to this value.)defaultmetadataintrrandomaoType of mask function. Defaults to a random sparse mask. An optional user-defined mask_fn to compute the mask value can also be supplied by instantiating `config = ShiraConfig(...)` and then setting `config.mask_fn = `. For a pretrained weight with shape m x n, the custom mask function must return only one mask (shape: m x n) which must be binary 0 or 1 with num_shira_parameters = r(m + n) for linear layers. Device and dtype of mask must be same as base layer's weight's device and dtype. Please see mask_functions.py for more details and to see the default random sparse mask implementation.zLiteral['random'] mask_typeNz3random seed for the torch generator for random_maskz Optional[int] random_seedzList of module names or regex expression of the module names to replace with SHiRA.For example, ['q', 'v'] or '.*decoder.*(SelfAttention|EncDecAttention).*(q|v)$'. Only linear layers are supported.zOptional[Union[list[str], str]]target_modulesFzMSet this to True if the layer to replace stores weight like (fan_in, fan_out)boolfan_in_fan_outTzInitialize SHiRA weight to have zero values. If set to False, SHiRA weights are initialized to randn values instead of zeros and this is used only for testing. init_weightsaList of modules apart from SHiRA layers to be set as trainable and saved in the final checkpoint. For example, in Sequence Classification or Token Classification tasks, the final layer `classifier/score` are randomly initialized and as such need to be trainable and saved.zOptional[list[str]]modules_to_savec>[TU]5 [RUl[ UR [5(a[UR 5O UR UlURS:Xa [Ul gUR(d%[R"SUR<S35 SUl g)NrzArgument self.mask_type=ze is not recognized, please supply your own masking function by calling `config.mask_fn = my_mask_fn`.)super __post_init__r SHIRA peft_type isinstancerlistsetrr mask_fninference_modewarningswarn)self __class__s R/home/james-whalen/.local/lib/python3.13/site-packages/peft/tuners/shira/config.pyr ShiraConfig.__post_init__ts !(243F3F(M(MC## $SWSfSf  >>X %&DL&& /01VW DL)r&r"r)__name__ __module__ __qualname____firstlineno____doc__rr__annotations__rrrrrrr __static_attributes__ __classcell__)r+s@r,rrs> s  As $) z  $I "'(]^"K7< 4  7N3 !ijND v L$ ,1 k  ,O(    r.r) __future__rr( dataclassesrrtypingrrr peft.configr peft.utilsr mask_functionsr rr.r,r>s=#(++"' e *e  e r.