oi9`SSKrSSKJrJr SSKJr SSKJr SSKJ r \"SS\55r g)N) dataclassfield)Optional) LoraConfig)PeftTypecR^\rSrSr%Sr\"SSS0S9r\\S'\"SSS 0S9r \\S '\"S SS 0S9r \\S '\"S SS0S9r \\S'\"SSS0S9r \\S'\"SSS0S9r \\S'\"SSS0S9r\\S'\"SSS0S9r\\S'\"SSS0S9r\\\S'\"SSS0S9r\\\S'U4SjrS rU=r$)! AdaLoraConfiga This is the configuration class to store the configuration of a [`~peft.AdaLora`]. AdaLoRA has three phases defined by `tinit`, `tfinal` and `total_step`. The initial phase can be understood as a step for pre-training the adapters so that when reducing their rank, there is already some information encoded that can be reduced instead of random matrices. This phase is defined by supplying `tinit`. After the initial phase is over (`tinit` steps have passed) and the final phase has not begun, AdaLoRA reduces the budget of how much rank each layer is allowed to have with each step. This is where the reduction of rank is happening. This goes on until `total_step - tfinal` steps are reached. The last phase, beginning once `total_step - tfinal` steps are reached, does not change the layer ranks anymore but fine-tunes the reduced-rank layers that resulted from the previous phase. A practical example: `tinit` is 10, `tfinal` is 20, `total_step` is 100. We spend 10 steps doing pre-training without rank reduction because our budget is constant (init phase), then we spend 80 (100-20) steps in the reduction phase where our budget decreases step-wise and, finally, 20 steps in the final fine-tuning stage without reduction. Args: target_r (`int`): The target average rank of incremental matrix. init_r (`int`): The initial rank for each incremental matrix. tinit (`int`): The steps of initial fine-tuning warmup. tfinal (`int`): The number of steps of final fine-tuning. deltaT (`int`): The time internval between two budget allocations. beta1 (`float`): The hyperparameter of EMA for sensitivity smoothing. beta2 (`float`): The hyperparameter of EMA for undertainty quantification. orth_reg_weight (`float`): The coefficient of orthogonal regularization. total_step (`int`): The total training steps that should be specified before training. rank_pattern (`list`): The allocated rank for each weight matrix by RankAllocator. helpzTarget Lora matrix dimension.)defaultmetadatatarget_r zInitial Lora matrix dimension.init_rrzThe steps of initial warmup.tinitzThe steps of final warmup.tfinalz!Step interval of rank allocation.deltaTg333333?zHyperparameter of EMA.beta1beta2g?z*The orthogonal regularization coefficient.orth_reg_weightNzThe total training steps. total_stepzThe saved rank pattern. rank_patternc>[TU]5 [RUlUR (a[ URS35eUR(a[ URS35e[UR[5(a[UR5O URUl [UR[5(a[UR5O URUl [UR[5(aURb [ S5eUR(aUR(d [ S5eUR S:wa["R$"S5 UR&bUR&S::a [ S5eUR(UR&UR*- :a [ S 5eg) Nz does not support DoRA.z does not support LOFTQ.zD`layers_to_transform` cannot be used when `target_modules` is a str.zRWhen `layers_pattern` is specified, `layers_to_transform` must also be specified. r zwNote that `r` is not used in AdaLora and will be ignored.If you intended to set the initial rank, use `init_r` instead.rzDAdaLoRA does not work when `total_step` is None, supply a value > 0.zsThe supplied schedule values don't allow for a budgeting phase. Decrease `tfinal`/`tinit` or increase `total_step`.)super __post_init__rADALORA peft_typeuse_dora ValueError loftq_config isinstancetarget_moduleslistsetexclude_modulesstrlayers_to_transformlayers_patternrwarningswarnrrr)self __class__s T/home/james-whalen/.local/lib/python3.13/site-packages/peft/tuners/adalora/config.pyrAdaLoraConfig.__post_init__Fs~ !)) ==//FGH H   //GHI I)343F3F(M(MC## $SWSfSf *4D4H4H$)O)OC$$ %UYUiUi  d))3 / /D4L4L4Xcd d   t'?'?qr r 66Q; MMQ  ?? "doo&:cd d ::$//DKK7 8)  9)r'rr$)__name__ __module__ __qualname____firstlineno____doc__rrint__annotations__rrrrrfloatrrrrrdictr__static_attributes__ __classcell__)r/s@r0r r s D!v7V.WXHcXf6V-WXFCXqF4R+STE3TV5Q,RSFCSV5X,YZFCZ9Q0RSE5S9Q0RSE5S"3&Bn9opOUp %dfFa=b cJ c#(Ib@c#dL(4.d&&r2r ) r, dataclassesrrtypingrpeft.tuners.lorar peft.utilsrr r2r0rCs4(' TJT Tr2