oiSSKJr SSKrSSKJrJr SSKrSSKJr SSK J r \\R\R\R4r"SS\R5r"SS \R5rg) ) annotationsN)OptionalUnion) XLoraConfigc2^\rSrSrSU4SjjrSrSrU=r$)TemperatureScaledSoftmaxc`>[TU]5 Xl[R"SS9Ulg)N)dim)super__init__ temperaturennSoftmaxsoftmax)selfr __class__s V/home/james-whalen/.local/lib/python3.13/site-packages/peft/tuners/xlora/classifier.pyr!TemperatureScaledSoftmax.__init__s$ &zzb) c@XR- nURU5$)N)rr)rlogits scaled_logitss rforward TemperatureScaledSoftmax.forward"s!1!11 ||M**r)rr)g?)__name__ __module__ __qualname____firstlineno__rr__static_attributes__ __classcell__rs@rr r s* ++rr c^\rSrSrSrS U4SjjrS S SjjrS S SjjrS SjrS Sjr Sr U=r $)XLoraClassifier)z/ A classifier to select LoRA layers for XLora. c >[T U]5 X0lX@lX l/Ul[ URRS9UlURUl SUl [UR55RUlURS:n/nURR S:XaUR"(aL[$R&"UR(X4-SS9R+U5R+UR5nGO[$R&"UR(USS9R+U5R+UR5nGO\URR S::a [-S5eUR/[$R&"UR(UR0SS9R+U5R+UR55 UR/[$R2"55 U(a-UR/[$R4"URS 95 [7UR S - 5Hn UR/[$R&"UR0UR0SS9R+U5R+UR55 UR/[$R2"55 U(dMUR/[$R4"URS 95 M UR"(aK[$R&"UR0X4-SS9R+U5R+UR5nOH[$R&"UR0USS9R+U5R+UR5n[$R8"/UQUP76Ulg ) z Construct an X-LoRA classifier from a model, config and some metadata. Note that n_layers is the number of LoRA adapter layers, not the number of model layers. )rFgrT)biasrz'X-LoRA depth must be strictly positive.)pN)rr n_classesn_layersconfig log_scalingsr softmax_temperaturerscaling_pass_valueoverride_scaling_pass_valuescalings_loggingnext parametersdtypexlora_dropout_p xlora_depthlayerwise_scalingsrLinear hidden_sizeto ValueErrorappend xlora_sizeReLUDropoutrange Sequentiallayers) rmodelr.r,r-device add_dropoutrDlast_rs rrXLoraClassifier.__init__.s "  /DKK<[<[\ 393L3L( %%**,-33 ,,s2  ;; " "a '((yy!3!3Y5IPTUXXY_`ccdhdndnoyy!3!3YTJMMfUXXY]YcYcd{{&&!+ !JKK MM"))F$6$68I8IPTUXXY_`ccdhdndno p MM"'') $ bjj6+A+ABC6--12 bii(9(96;L;LSWX[[\bcffgkgqgqrs bggi(;MM"**v/E/E"FG 3((yy!2!2I4HtTWWX^_bbcgcmcmnyy!2!2IDILLVTWWX\XbXbcmm2V2T2 rcPUb+URSnURnURSnO*URSnURnURSn[R"XWURUR 4UR 5RX`RS9$)a0 Make some dummy scalings for the scalings pass (the one to get the logits for the X-LoRA classifier). These are of shape (batch_size, seq_len, n_layers, n_classes) and filled with the override scalings pass value. Note that n_layers is the number of LoRA adapter layers, not the number of model layers. rr)rFr6) shaperFtorchfullr-r,r2r<r6)r input_ids inputs_embedsargskwargs batch_sizerFseq_lens rmake_dummy_scalings#XLoraClassifier.make_dummy_scalingses  "+J%%Fooa(G&,,Q/J"))F#))!,Gzz $-- @  , ,  "F**" - .rcpUbURSnURSnOURSnURSnURnUSn URRU 5n URR (d/U R S5n U RSSURS5n U RXgURUR5n URR(aURU 5n UR(aURRU 5 U $)zd Using the hidden states of the model, predict `n_classes` LoRA alpha values. Returns the scalings. rrr r+)rL hidden_statesrDrr.r9 unsqueezeexpandr-reshaper,enable_softmaxrr3r/r>) rresultrOrPrQrRrSrTrX hidden_staterscalingss rrXLoraClassifier.forwards  "+Jooa(G&,,Q/J#))!,G,, $R( $$\2 {{--%%a(F]]2r4=="=F>>*t}}dnnU ;; % %||H-H     $ $X .rc0n[UR5HOup#URSnXA;a U/U/4X'M#XSRU5 XSRU5 MQ U$)a, Returns bucketed scalings, bucketed by seq_len. Each value consists of the positions (the first) and the associated tensors. The positions are paired with the associated tensors and give the position in the scaling log. Each scaling is a tensor of shape (batch_size, seq_len, n_layers, n_classes)). rr) enumerater/rLr>)r seqlens_mapiscalingrTs r_get_bucketed_scalings&XLoraClassifier._get_bucketed_scalingss{ HJ #D$5$56JAmmA&G))*gY'7 $$Q'..q1$Q'..w7 7rcvUcSUR- UlOXlURURlg)Nr)r,r2r.r1)rvalues r _set_override_scaling_pass_value0XLoraClassifier._set_override_scaling_pass_values0 =/04>>/AD ,/4 ,)-)I)I &r) r.r6rDr/r,r-r2r3r) rEz nn.Moduler.rr,intr-rlrFz torch.device)NN)rOzOptional[torch.LongTensor]rPzOptional[torch.FloatTensor]returnz torch.Tensor)rmz/dict[int, tuple[list[int], list[torch.Tensor]]])rizUnion[Number, None]) rrr r!__doc__rrUrrfrjr"r#r$s@rr&r&)s535353 53  53  53r1559.-.3.  .:1559 ,.,3 , ,\"JJrr&) __future__rbuiltinstypingrrrMtorch.nnrr.rrlfloatboolNumberModuler r&rrrxs\#"  x||X^^X]]: ; +ryy +ZJbiiZJr