oiMSSKJr SSKrSSKJr SSKrSSKJr SSKJr SSK J r Sr S r "S S \R5rSS jrSS jrSSjrSSSjjrg)) annotationsN)Any)nn)PreTrainedModel) ArrowConfigtask_gks_c^\rSrSrSrU4Sjr\R"5S5rS Sjr \R"5S5r \R"5S5r S Sjr S r S rU=r$) ArrowLoraLinearLayerzW This class represent the main logic of the arrow routing algorithm for linear layers. cZ>[TU]5 XlSUlURUlUR UlURUlURR5UlURUl URUl SUl /Ul XlSUlg)NFT)super__init__ in_features _protos_readytop_krouter_temperature temperaturerng_seedtask_adapter_namescopygks_adapter_namesuse_gksgks_donegks_added_adapter_namescast_input_dtype_enabled)selfr arrow_config __class__s P/home/james-whalen/.local/lib/python3.13/site-packages/peft/tuners/lora/arrow.pyrArrowLoraLinearLayer.__init__#s &"!'' '::$--  + + 0 0 2   * * $++  ')$&(,%cUR5Vs/sHMnX2;dM US:wdMURS5(a#U[S5SR5(aMKUPMO nn[ UR 5[ U5:Xag[UR 5[U5:a)UVs/sHoUUR ;dMUPM snUlUR5UlSUlgs snfs snf)zv Called when adapters are added/removed/renamed so Arrow can refresh its internal state before the next forward pass. arrow_routerr NF) keys startswithlenisdigitsortedrrrr)rlora_Alora_Bkall_ts_adapter_namesxs r!on_adapter_change&ArrowLoraLinearLayer.on_adapter_change7s[[] "{  N2 <=LLE MM% ,kk&a0  \;5I!r#cnUR(dgUR(aUR(dg[R"UR Vs/sHo1UR PM snSS9RS5n[R"UR Vs/sHo2UR PM snSS9RS5nURSLabURHQnXR RRU5 X&R RRU5 MS OaURHQnXR RRU5 X&R RRU5 MS SUl/Ulgs snfs snf)a This function performs General Knowledge Subtraction. It takes an average of provided general_adapters, and subtract it from each task_adapter. This subtraction tries to purify the task adapters, based on "forgetting-via-negation" principle. Forgetting-via-negation is a task-arithmetic operation, explained in: https://huggingface.co/papers/2212.04089 The task adapters will be more focused and isolated, enhancing the performance on new tasks. Args: lora_A : Matrices A in LoRA layer. lora_B : Matrices A in LoRA layer. NrrPFT) rrrr7rVrrTmeanrdatasub_)rr+r,navg_Aavg_BrYs r! gen_know_sub!ArrowLoraLinearLayer.gen_know_subs]||  ]]4#?#? KK4;Q;Q R;Qa!1!1;Q RXYZ__EKK4;Q;Q R;Qa!1!1;Q RXYZ__E }}% 33DL'',,11%8L'',,11%84!88DL'',,11%8L'',,11%89 !DM+-D ()!S!Ss F-F2ctUcg[USS5nU(aURU:XaU$URUS9$)aT Whether to cast the dtype of the input of the forward method. Usually, we want to enable this to align the input dtype with the dtype of the weight, but by setting layer.cast_input_dtype=False, this can be disabled if necessary. Enabling or disabling can be managed via the peft.helpers.disable_lora_input_dtype_casting context manager. NrT)r4)getattrr4r6)rr/r4rs r!_cast_input_dtype&ArrowLoraLinearLayer._cast_input_dtypes@ 9#*41KT#R (agg.>Htt%t  r#cURXURSRR5nURGtpgnUR SU5n U R S5URR S5p[R"URV s/sHoU PM sn U RU RS9n [R"XRR-5n[R"XRSS9unnU RX4[!S55nUR#SUU5 [R$"UUR&- SS9n[R("URV s/sHoU RPM sn SS9n[R("URV s/sHoU RPM sn SS9n[R*"SU U5n[R*"SUU5nUU R SSS5-n[R*"S UU5nU"U5nUR S5nUR "U/UQUP76$s sn fs sn fs sn f) u Applies Arrow routing inside a LoRA layer. Steps: 1. Compute cosine similarity between each token representation and all adapter prototypes. 2. Select the top-k experts per token and normalize their scores with a softmax. 3. Project tokens into each selected expert’s low-rank space (A weights). 4. Map back to the output space (B weights). 5. Aggregate expert outputs via the weighted sum of their contributions. 6. Apply dropout, scaling, and return the reshaped delta. - Conceptually, this is a Mixture-of-Experts (MoE) over LoRA adapters, where coefficients are derived from prototype similarity. Returns: delta: LoRA output adjustment computed by Arrow routing. r)r3r4rrPz-infztf, erf -> terzter, eor -> teoz te, teo -> to)rjrrTr4shapeviewr?rRr7tensorr3absr9topkrnew_fullfloatscatter_softmaxrrVeinsum)rr/r+r,dropoutscalingrCrestF_intoktErc scales_tenssimtop_vidx full_scorecoeffA_stackB_stackzy delta_flatdeltaout_dims r!forwardArrowLoraLinearLayer.forwards$  " "1T-D-DQ-G&H&O&O&U&U V$ffRxx{DOO0031ll!%!8!8 9!8AQZ!8 9::)) iioo///0ZZZZQ7 s\\1&%-8 AsE* j4+;+;;C++9P9PQ9PAay//9PQWXY++9P9PQ9PAay//9PQWXY LL)3 8 LL*Aw 7    B* *\\/5!<  #//"%zz!,d,G,,M : RQs I,I1I6) rrrrrrrrrrr)g:0yE>)r4z torch.dtype)__name__ __module__ __qualname____firstlineno____doc__rr7no_gradr0rMr]rfrjr__static_attributes__ __classcell__)r s@r!r r sr-( ]]_##0+Z ]]_!"!"F ]]_&.&.P!"?-?-r#r c SnUGHFn[5nSnSnUR5Hupx[US5(dMX8R;dM)URURn UR URn UR S5Sn URU 5 U RSnU RU R4nM UcXVUS.nMXRS:wa[SUS US US35eXbS :wa[SUS US US 35eXBS :wdGM [SUS[U5S[US 535e [US 5n U [US54$)z After loading all adapters into `model`, check they share: - the same LoRA rank (r) - identical weight shapes - identical sets of target_modules Returns (sorted list of target module names, agreed rank r). Nr+.rmr)rshapesmodulesr[z] rank mismatch: z != rz] shape mismatch: rz-] target_modules mismatch: this adapter -> z reference -> ) set named_moduleshasattrr+rTr,splitaddrn ValueErrorr*r=) model adapter_names referencerY curr_modulescurr_r curr_shapes full_namemodulerBrCmod_nameagreed_moduless r!%check_loaded_lora_compatibility_arrowrsIu  !&!4!4!6 Ivx((T]]-BMM$'..MM$'..$??3/3  * ww0 "7  $ UI3' 1TF*;F84 RUGW!XYY11 1TF*<[MiX`NaMb!cdd33 v))/ )=(>?((.y/C(D'EG/:Ii01N 3y~. ..r#c SSKJn SnSSKnURRnSnSSKJn URUR4nUbXc4-nUbXe4-n/nUR5Hup[U S5(dMUHvn U [U S05;dM[U SS5=(d [U SS5n U bU OU n [X5(aMPURX[U 5R45 Mx M U(aCS/n UH upnU RSU S US U35 M" [!S R#U 55eg![a GN3f=f![a GN;f=f) z Validate that every module holding LoRA weights for any of `adapter_names` is Linear-like: nn.Linear, bitsandbytes.nn.Linear4bit, nn.Conv1d, or transformers.models.gpt2.modeling_gpt2.Conv1D. If not, raise. rN)Conv1Dr+ base_layeroriginal_modulezzLoRA adapters must only target Linear-like layers (nn.Linear, nn.Conv1d, HF Conv1D, or bitsandbytes.nn.Linear4bit). Found:z - adapter 'z ' on module 'z ' of type  )torch.nnr bitsandbytes Linear4bit ImportError&transformers.models.gpt2.modeling_gpt2rLinearConv1drrri isinstancerUr;r TypeErrorjoin)rrrrbnbHFConv1D allowed_types offendersrrrYbaselayer_to_checklinestnames r!)ensure_adapters_target_linear_layers_onlyrHs J "VV&& H MYY *M% 5 % 3 I"002 68 $ $%768R88"6<>j'&RceiBjD-1-=T6N%nDD!(($4;O;X;X)YZ &3 W '0 "DU LL=mI;jQVPWX Y'0 %())7       s"E E EE E)(E)c[RRU5(aU[RR[RR US55(d[ SUS35eUS4$UR S5RS5n[U5S:a>SR USS5n[U5S:aSR USS5nX#4$US4$US4$)aU Resolve a user-provided adapter `path` into (model_id, subfolder). Supports: - Local path to a folder that contains `adapter_config.json` - Hub path with subfolder, e.g. "user/repo/ts_expert_0[/more/...]", which becomes: model_id="user/repo", subfolder="ts_expert_0[/more/...]" - Plain Hub repo id "user/repo" (no subfolder) zadapter_config.jsonzLocal adapter path 'z)' does not contain 'adapter_config.json'.N/) ospathisdirisfilerrstriprr()rpartsmodel_id subfolders r!_resolve_adapter_sourcerys ww}}Tww~~bggll41FGHH3D69bcd dTz JJsO ! !# &E 5zQ88E"1I& u:>qr+I& &~ :r#c Ub[U5S:Xa [S5eSSKJnJn [ US5upx[ S3n [U5n Ub SU ;aXS'UR"U4UU S.U D6n [S[U55HJn [ U 3n [ X5up[U5nUb SU;aUUS'U R"S UU S.UD6 ML [[U55V s/sH n [ U 3PM sn Ul UR(aUb[U5S:Xa [S5e[[U55HJn [U 3n[ X<5up[U5nUb SU;aUUS'U R"S UUS.UD6 ML [[U55V s/sH n [U 3PM sn UlO/Ul[XRUR-S 9unn[!XRUR-S 9 U"UUUS 9nU R#S US 9 U R%S 5 U $s sn fs sn f)NrzF`task_specific_adapter_paths` should contain at least one adapter path) LoraConfig PeftModel0r)r adapter_namerzDYou should provide general LoRA paths if you want to use GenKnowSub.)r)rtarget_modulesrr%)r peft_config)r(rpeftrrrTASK_ADAPTER_PREFIXdictfrom_pretrainedrA load_adapterrrGKS_ADAPTER_PREFIXrrr add_adapter set_adapter) base_modeltask_specific_adapter_pathsrgeneral_adapter_pathsadapter_kwargsrr model_id0sub0initial_ts_expert_name first_kwargsrits_expert_namemidsub more_kwargsgen_expert_name gks_kwargsrr router_cfgs r!create_arrow_modelrs#*c2M.NRS.Sabb*-.I!.LMOI 34A6'L K|;$([!  % % +    E1c56 7/04*+F+IJ>* ?{+='*K $  '  8MRRUVqRrLs&tLsq*=)>qc'BLs&tL# (C0E,F!,Kcd ds012A!3 4QC8O./D/GHHCn-J;j#@*- ;'    ,   3OTTWXmTnNo)pNo-?,@*DNo)p &)+ &= <<|?]?]]NA. <<|?]?]]!% J  >zJ n% LI'u *qs &H:(H?)r list[str])rstrreturnztuple[str, str | None])N) rrrrrrrzlist[str] | Nonerr) __future__rrtypingrr7r transformersrconfigrrrModuler rrrrrr#r!rs#  (|-299|-~(/V.*b</3 II!*II, I  Ir#