hJSrSSKJr SSKrSSKJr SSKJs Jr SSK J r SSK J r J r JrJrJrJr SSKJr Sr"S S \R*5r"S S \R*5r"S S\R*5r"SS\R*5r"SS\R*5r"SS\R*5r"SS\R*5r"SS\R*5r"SS\R*5r"SS\R*5r"SS\5r "SS \R*5r!"S!S"\5r""S#S$\5r#"S%S&\R*5r$"S'S(\R*5r%"S)S*\R*5r&"S+S,\R*5r'"S-S.\R*5r("S/S0\R*5r)"S1S2\R*5r*"S3S4\R*5r+"S5S6\R*5r,"S7S8\R*5r-"S9S:\%5r."S;S<\5r/"S=S>\R*5r0"S?S@\05r1"SASB\R*5r2"SCSD\R*5r3"SESF\R*5r4"SGSH\R*5r5"SISJ\R*5r6"SKSL\R*5r7"SMSN\5r8"SOSP\5r9"SQSR\R R*5r:"SSST\R*5r;"SUSV\5r<"SWSX\R*5r="SYSZ\R*5r>"S[S\\R*5r?"S]S^\R*5r@"S_S`\5rA"SaSb\R*5rB"ScSd\R*5rC"SeSf\R*5rD"SgSh\R*5rE"SiSj\R*5rF"SkSl\R*5rG"SmSn\R*5rH"SoSp\R*5rIg)qzBlock modules.) annotationsN)fuse_conv_and_bn)ConvDWConv GhostConv LightConvRepConvautopad)TransformerBlock)'DFLHGBlockHGStemSPPSPPFC1C2C3C2fC2fAttnImagePoolingAttnContrastiveHeadBNContrastiveHeadC3xC3TRC3GhostGhostBottleneck Bottleneck BottleneckCSPProtoRepC3 ResNetLayer RepNCSPELAN4ELAN1ADownAConvSPPELANCBFuseCBLinearC3k2C2fPSAC2PSARepVGGDWCIBC2fCIB AttentionPSASCDown TorchVisionc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r :z Integral module of Distribution Focal Loss (DFL). Proposed in Generalized Focal Loss https://ieeexplore.ieee.org/document/9792391 c\>[TU]5 [R"USSSS9R S5Ul[ R"U[ RS9n[R"URSUSS55UR RRSS&Xl g)zx Initialize a convolutional layer with a given number of input channels. Args: c1 (int): Number of input channels. rFbias)dtypeN)super__init__nnConv2drequires_grad_convtorcharangefloat Parameterviewweightdatac1)selfrGx __class__s V/home/james-whalen/.local/lib/python3.13/site-packages/ultralytics/nn/modules/block.pyr; DFL.__init__As| IIb!QU3BB5I LL5;; /#%<<q"a0C#D a cURup#nURURUSURU5R SS5R S55RUSU5$)zCApply the DFL module to input tensor and return transformed output.r)shaper?rDrG transposesoftmax)rHrIb_as rKforward DFL.forwardNs[''ayy1dggq1;;AqAII!LMRRSTVWYZ[[rM)rGr?))rGintrI torch.Tensorreturnr\ __name__ __module__ __qualname____firstlineno____doc__r;rW__static_attributes__ __classcell__rJs@rKr r :s  \\rMr c>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r UzBUltralytics YOLO models mask Proto module for segmentation models.c >[TU]5 [XSS9Ul[R "X"SSSSS9Ul[X"SS9Ul[X#5Ulg)z Initialize the Ultralytics YOLO models mask Proto module with specified number of protos and masks. Args: c1 (int): Input channels. c_ (int): Intermediate channels. c2 (int): Output channels (number of protos). )krPrTr7N) r:r;rcv1r<ConvTranspose2dupsamplecv2cv3)rHrGc_c2rJs rKr;Proto.__init__XsQ !$**21aF !$<rMc ~URURURURU5555$)zEPerform a forward pass through layers using an upsampled input image.)rprornrlrHrIs rKrW Proto.forwardgs+xxtxx{!;<==rM)rlrorprn) )rGrZrqrZrrrZr[r^rfs@rKr r UsL   >>rMr c:^\rSrSrSrSU4SjjrSSjrSrU=r$)rlz StemBlock of PPHGNetV2 with 5 convolutions and one maxpool2d. https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/backbones/hgnet_v2.py c >[TU]5 [XSS[R"5S9Ul[X"S-SSS[R"5S9Ul[US-USSS[R"5S9Ul[US-USS[R"5S9Ul[X#SS[R"5S9Ul [R"SSSSS9Ul g) z Initialize the StemBlock of PPHGNetV2. Args: c1 (int): Input channels. cm (int): Middle channels. c2 (int): Output channels. rjrPactrrT) kernel_sizestridepadding ceil_modeN) r:r;rr<ReLUstem1stem2astem2bstem3stem4 MaxPool2dpool)rHrGcmrrrJs rKr;HGStem.__init__ss "!QBGGI6 2Qw1aRWWY? 27B1aRWWY? "q&"a : "!QBGGI6 LLQq!tT rMcbURU5n[R"U/SQ5nURU5n[R"U/SQ5nUR U5nUR U5n[ R"X2/SS9nURU5nURU5nU$)+Forward pass of a PPHGNetV2 backbone layer.)rrrrrdim) rFpadrrrr@catrr)rHrIx2x1s rKrWHGStem.forwards JJqM EE!\ " [[^ UU2| $ [[_ YYq\ IIrhA & JJqM JJqMrM)rrrrrr)rGrZrrZrrrZr[r^rfs@rKrrls U"  rMrc^\rSrSrSrSSSS\R "54S U4SjjjrS SjrSr U=r $) rz HG_Block of PPHGNetV2 with 2 convolutions and LightConv. https://github.com/PaddlePaddle/PaddleDetection/blob/develop/ppdet/modeling/backbones/hgnet_v2.py rjFc <>^^^^^ [T U]5 U(a[O[m [R "UU UUU4Sj[ U555Ul[TUT--US-SSTS9Ul[US-USSTS9Ul U=(a TU:HUl g)ap Initialize HGBlock with specified parameters. Args: c1 (int): Input channels. cm (int): Middle channels. c2 (int): Output channels. k (int): Kernel size. n (int): Number of LightConv or Conv blocks. lightconv (bool): Whether to use LightConv. shortcut (bool): Whether to use shortcut connection. act (nn.Module): Activation function. c3F># UHnT"US:XaTOTTTTS9v M g7f)rrkr}N).0ir}blockrGrrks rK #HGBlock.__init__..s'_V^QRu16Rr2LV^!rPrr|N) r:r;r rr< ModuleListrangemscecadd) rHrGrrrrkn lightconvshortcutr}rrJs `` ` `@rKr;HGBlock.__init__s0 & D__V[\]V^__rAF{B!GQs;rQwAqc2(brMc^U/mTRU4SjUR55 URUR[R "TS555mUR (aTU-$T$)rc38># UHo"TS5v M g7fNrrrys rKr"HGBlock.forward..*6a1R56r)extendrrrr@rrrHrIrs @rKrWHGBlock.forwardsV C *466** GGDGGEIIaO, -q1u'a'rM)rrrr)rGrZrrZrrrZrkrZrrZrboolrrr} nn.Moduler[) r_r`rarbrcr<rr;rWrdrerfs@rKrrs) ) )  )  )  )))))>((rMrc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) rzDSpatial Pyramid Pooling (SPP) layer https://arxiv.org/abs/1406.4729.c $>[TU]5 US-n[XSS5Ul[U[ U5S--USS5Ul[ R"UVs/sHn[ R"USUS-S9PM sn5Ul gs snf)z Initialize the SPP layer with input/output channels and pooling kernel sizes. Args: c1 (int): Input channels. c2 (int): Output channels. k (tuple): Kernel sizes for max pooling. rPrr~rrN) r:r;rrllenror<rrr)rHrGrrrkrqrIrJs rKr; SPP.__init__s  1W1%c!fqj)2q!4_`a_`Z[ 1aSTf U_`abas#B c URU5nUR[R"U/URVs/sH o""U5PM sn-S55$s snf)zBForward pass of the SPP layer, performing spatial pyramid pooling.r)rlror@rr)rHrIrs rKrW SPP.forwardsJ HHQKxx 1#tvv(>v!1v(>">BCC(>sArlror)) )rGrZrrrZrktuple[int, ...]r[r^rfs@rKrrsN c cDDrMrc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) rzGSpatial Pyramid Pooling - Fast (SPPF) layer for YOLOv5 by Glenn Jocher.c>[TU]5 US-n[XSS5Ul[US-USS5Ul[ R "USUS-S9Ulg)z Initialize the SPPF layer with given input/output channels and kernel size. Args: c1 (int): Input channels. c2 (int): Output channels. k (int): Kernel size. Notes: This module is equivalent to SPP(k=(5, 9, 13)). rPrrOrN)r:r;rrlror<rr)rHrGrrrkrqrJs rKr; SPPF.__init__sW  1W1%QAq)!AqAvFrMc^^TRU5/mTRUU4Sj[S555 TR[R "TS55$)zRApply sequential pooling operations to input and return concatenated feature maps.c3L># UHnTRTS5v M g7frr)rrUrHrs rKrSPPF.forward..s11"s!$rjr)rlrrror@rrs` @rKrW SPPF.forwardsA XXa[M 1a11xx !Q((rMrr)rGrZrrrZrkrZr[r^rfs@rKrrsQGG$))rMrc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) rz"CSP Bottleneck with 1 convolution.c>^[TU]5 [UTSS5Ul[R "U4Sj[ U556Ulg)z Initialize the CSP Bottleneck with 1 convolution. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of convolutions. rc3>># UHn[TTS5v M g7f)rjN)r)rrUrrs rKrC1.__init__..s C(Qb"a(N)r:r;rrlr< Sequentialrr)rHrGrrrrJs ` rKr; C1.__init__s< B1% C%( CDrMcLURU5nURU5U-$)z:Apply convolution and residual connection to input tensor.rlrrs rKrW C1.forwards! HHQKvvay1}rMrr)rGrZrrrZrrZr[r^rfs@rKrrs, E ErMrc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) ri z#CSP Bottleneck with 2 convolutions.c$>^^^[TT]5 [X&-5Tl[ USTR-SS5Tl[ STR-US5Tl[R"UUU4Sj[U556Tl g)a  Initialize a CSP Bottleneck with 2 convolutions. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of Bottleneck blocks. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. rPrc 3h># UH'n[TRTRTTSSS9v M) g7f)rjrjr?rkeNrcrrUgrHrs rKrC2.__init__..s- vmuhiDFFDFFHaK[_b!cmu/2N r:r;rZrrrlror<rrrrHrGrrrrrrrJs` `` rKr; C2.__init__ sl RVAJ1-DFF B* vmrstmu vwrMcURU5RSS5up#UR[R"UR U5U4S55$)z^^^[TT]5 [X&-5Tl[ USTR-SS5Tl[ SU-TR-US5Tl[R"UUU4Sj[U555Tl g)a  Initialize a CSP bottleneck with 2 convolutions. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of Bottleneck blocks. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. rPrc 3h># UH'n[TRTRTTSSS9v M) g7frrrs rKrC2f.__init__..9-tksfgz$&&$&&(AIY]`aksrN) r:r;rZrrrlror<rrrrs` `` rKr; C2f.__init__)sq RVAJ1-Q$&&("a0tkpqrksttrMc^[URU5RSS55mTRU4SjUR55 UR [ R"TS55$)zForward pass through C2f layer.rPrc38># UHo"TS5v M g7frrrs rKrC2f.forward..>rr)listrlrrrror@rrs @rKrW C2f.forward;sQ !""1a( ) *466**xx !Q((rMc^URU5RURUR4S5mTSTS/mTRU4SjUR55 UR [ R"TS55$).Forward pass using split() instead of chunk().rrc38># UHo"TS5v M g7frrrs rKr$C2f.forward_split..Err)rlsplitrrrror@rrs @rK forward_splitC2f.forward_splitAsj HHQK  tvvtvv. 2 qT1Q4L *466**xx !Q((rMrrFrrrr[ r_r`rarbrcr;rWrrdrerfs@rKrr&s!Fuu$) ))rMrc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) riIz#CSP Bottleneck with 3 convolutions.c>^^^[TU]5 [X&-5m[UTSS5Ul[UTSS5Ul[ST-US5Ul[R"UUU4Sj[U556Ul g)a" Initialize the CSP Bottleneck with 3 convolutions. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of Bottleneck blocks. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. rrPc 3@># UHn[TTTTSSS9v M g7f)))rrrrrNrrrUrqrrs rKrC3.__init__..]s% nem`aBHaCSWZ![emN) r:r;rZrrlrorpr<rrr rHrGrrrrrrrqrJs `` @rKr; C3.__init__Lsp  [B1%B1%BA& nejklem norMc UR[R"URUR U55UR U54S55$)z^^^[TT]XUTTU5 [X&-5Tl[R "UUU4Sj[ U556Tlg)a Initialize C3 module with cross-convolutions. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of Bottleneck blocks. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. c 3h># UH'n[TRTRTTSSS9v M) g7f)))rrjrjrrrN)rrqrs rKrC3x.__init__..us- vmuhiDGGTWWhM]ab!cmurN)r:r;rZrqr<rrrrs` `` rKr; C3x.__init__gsD Ha3bf+ vmrstmu vwrM)rqrrrr_r`rarbrcr;rdrerfs@rKrrds,xxrMrc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r!ixzRep C3.c h>[TU]5 [X$-5n[XSS5Ul[XSS5Ul[ R"[U5Vs/sHn[XU5PM sn6Ul XR:wa[XRSS5Ul g[ R"5Ul gs snf)z Initialize CSP Bottleneck with a single convolution. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of RepConv blocks. e (float): Expansion ratio. rN) r:r;rZrrlror<rrr rIdentityrp)rHrGrrrrrqrUrJs rKr;RepC3.__init__{s  [1%1%%( C(Q( CD)+41%r{{}!DsB/cURURURU55URU5-5$)zForward pass of RepC3 module.)rprrlrorus rKrW RepC3.forwards/xxtxx{+dhhqk9::rMr)rjrrGrZrrrZrrZrrBr[r^rfs@rKr!r!xsEE";;rMr!c4^\rSrSrSrSSU4SjjjrSrU=r$)riz"C3 module with TransformerBlock().cf>[TU]XX4XV5 [X&-5n[XwSU5Ulg)a Initialize C3 module with TransformerBlock. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of Transformer blocks. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. rON)r:r;rZr rrs rKr; C3TR.__init__s1 a3 [!"!Q/rMrrrrrfs@rKrrs,00rMrc4^\rSrSrSrSSU4SjjjrSrU=r$)riz!C3 module with GhostBottleneck().c>^[TU]XX4XV5 [X&-5m[R"U4Sj[ U556Ulg)a  Initialize C3 module with GhostBottleneck. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of Ghost bottleneck blocks. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. c3<># UHn[TT5v M g7f)N)r)rrUrqs rKr#C3Ghost.__init__..s K(QR!8!8(sNr:r;rZr<rrrrs @rKr;C3Ghost.__init__s; a3 [ K%( KLrMrrrrrfs@rKrrs+MMrMrc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) rizGGhost Bottleneck https://github.com/huawei-noah/Efficient-AI-Backbones.c x>[TU]5 US-n[R"[ XSS5US:Xa [ XUX4SS9O[R "5[ XRSSSS95UlUS:Xa0[R"[ XX4SS9[XSSSS95Ul g[R "5Ul g)z Initialize Ghost Bottleneck module. Args: c1 (int): Input channels. c2 (int): Output channels. k (int): Kernel size. s (int): Stride. rPrFr|N) r:r;r<rrrr!r?rr)rHrGrrrksrqrJs rKr;GhostBottleneck.__init__s  1WMM ba #/0AvF21U +2;;= ba .  ^_bc]cBMM&5941RW;X Y ikititiv rMcHURU5URU5-$)z8Apply skip connection and concatenation to input tensor.r?rrus rKrWGhostBottleneck.forwardsyy|dmmA...rMr4rrGrZrrrZrkrZr1rZr[r^rfs@rKrrsQ  *//rMrcV^\rSrSrSrSSU4SjjjrSSjrSrU=r$) rizStandard bottleneck.c>[TU]5 [X&-5n[XUSS5Ul[XrUSSUS9UlU=(a X:HUlg)a Initialize a standard bottleneck module. Args: c1 (int): Input channels. c2 (int): Output channels. shortcut (bool): Whether to use shortcut connection. g (int): Groups for convolutions. k (tuple): Kernel sizes for convolutions. e (float): Expansion ratio. rrrN)r:r;rZrrlror rHrGrrrrrkrrqrJs rKr;Bottleneck.__init__sS  [!a(!a1-(rMcUR(a"XRURU55-$URURU55$)z3Apply bottleneck with optional shortcut connection.)rrorlrus rKrWBottleneck.forwards8,0HHq88DHHQK((O$((488A;:OOrM)rrlroTrrr rGrZrrrZrrrrZrkztuple[int, int]rrBr[r^rfs@rKrrsSlo)))*.):=)FU)ch))(PPrMrc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) rizGCSP Bottleneck https://github.com/WongKinYiu/CrossStagePartialNetworks.c>^^^[TU]5 [X&-5m[UTSS5Ul[ R "UTSSSS9Ul[ R "TTSSSS9Ul[ST-USS5Ul [ R"ST-5Ul [ R"5Ul [ R"UUU4Sj[U556Ulg)a  Initialize CSP Bottleneck. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of Bottleneck blocks. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. rFr7rPc 3>># UHn[TTTTSS9v M g7frrNrrs rKr)BottleneckCSP.__init__.. s ZQYABHa3!GQYrN)r:r;rZrrlr<r=rorpcv4 BatchNorm2dbnSiLUr}rrrrs `` @rKr;BottleneckCSP.__init__s  [B1%99RQ699RQ6BAq)..R(779 ZQVWXQY Z[rMc  URURURU555nURU5nUR UR UR [R"X#4S5555$)z)Apply CSP bottleneck with 3 convolutions.r) rprrlrorFr}rHr@r)rHrIy1y2s rKrWBottleneckCSP.forward s\ XXdffTXXa[) * XXa[xxB8Q)?!@ABBrM)r}rHrlrorprFrrrr[r^rfs@rKrrsQ\\,CCrMrc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) ResNetBlockiz.ResNet block with standard convolution layers.c &>[TU]5 XB-n[XSSSS9Ul[X"SUSSS9Ul[X%SSS9UlUS:wdX:wa&[ R"[XSUSS95Ul g[ R"5Ul g) z Initialize ResNet block. Args: c1 (int): Input channels. c2 (int): Output channels. s (int): Stride. e (int): Expansion ratio. rTrkr1r}rjrkr1pr}FrN) r:r;rrlrorpr<rr!r)rHrGrrr1rc3rJs rKr;ResNetBlock.__init__s  V!qd3!qA48!/LMQRFVXV^ d2Q!&GH dfdododq rMc [R"URURUR U555UR U5-5$)z&Forward pass through the ResNet block.)rrelurprorlrrus rKrWResNetBlock.forward&s9vvdhhtxx 45 a8HHIIrM)rlrorpr)rrO)rGrZrrrZr1rZrrZr[r^rfs@rKrPrPs8rr"JJrMrPc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r"i+z)ResNet layer with multiple ResNet blocks.c >[T U]5 X@lUR(a<[R"[ XSSSSS9[R "SSSS95Ulg [XX6S9/nUR[US- 5Vs/sHn[Xb-USUS9PM sn5 [R"U6Ulg s snf) z Initialize ResNet layer. Args: c1 (int): Input channels. c2 (int): Output channels. s (int): Stride. is_first (bool): Whether this is the first layer. n (int): Number of ResNet blocks. e (int): Expansion ratio. rPrjTrSrrrDN) r:r;is_firstr<rrrlayerrPrr) rHrGrrr1r]rrblocksrUrJs rKr;ResNetLayer.__init__.s  ==RqA5r||PQZ[ef7gDJ""!12F MME!a%LQLq;qvr1:LQ R/DJRs Cc$URU5$)z&Forward pass through the ResNet layer.)r^rus rKrWResNetLayer.forwardFszz!}rM)r]r^)rFrrO) rGrZrrrZr1rZr]rrrZrrZr[r^rfs@rKr"r"+s3000rMr"c>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) MaxSigmoidAttnBlockiKzMax Sigmoid attention block.c>[TU]5 X0lX#-UlX:wa [ XSSS9OSUl[ R"XT5Ul[ R"[R"U55Ul [ XSSSS9Ul U(a3[ R"[R"SUSS55UlgSUlg)a Initialize MaxSigmoidAttnBlock. Args: c1 (int): Input channels. c2 (int): Output channels. nh (int): Number of heads. ec (int): Embedding channels. gc (int): Guide channels. scale (bool): Whether to use learnable scale parameter. rFrNrjrRr)r:r;nhhcrrr<LinearglrCr@zerosr8 proj_convonesscale)rHrGrrrfrgcrmrJs rKr;MaxSigmoidAttnBlock.__init__Ns (24($r.))B#LLR1 bQE:>CR\\%**QAq"9:  rMcURup4pVURU5nURX2RSURUR5nUR bUR U5OUnURX0RURXV5n[ R"SXr5nURSS9SnXRS-- nXRSSS2SS4-nUR5UR-nURU5nURX0RSXV5nXRS5-nURUSXV5$) z Forward pass of MaxSigmoidAttnBlock. Args: x (torch.Tensor): Input tensor. guide (torch.Tensor): Guide tensor. Returns: (torch.Tensor): Output tensor after attention. rNzbmchw,bnmc->bmhwnrrrrrP)rQrirDrfrgrr@einsummaxr8sigmoidrmrk unsqueeze) rHrIguidebsrUhwembedaws rKrWMaxSigmoidAttnBlock.forwardcsgg q 2{{1~tww@"gg1 q 2ww6 \\-u < VVV^A  77C<  ))D!T4/0 0 ZZ\DJJ & NN1  FF2wwA ) Q vvb"a##rM)r8rrirgrfrkrm)rF) rGrZrrrZrfrZrrZrnrZrmrrIr\rur\r]r\r^rfs@rKrdrdKs&MM*$$rMrdcx^\rSrSrSrSSU4SjjjrS SjrS SjrSrU=r $) riz*C2f module with an additional attn module.c z>^^^[T T]5 [X)-5Tl[ USTR-SS5Tl[ SU-TR-US5Tl[R"UUU4Sj[U555Tl [TRTRXdUS9Tl g)a Initialize C2f module with attention mechanism. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of Bottleneck blocks. ec (int): Embedding channels for attention. nh (int): Number of heads for attention. gc (int): Guide channels for attention. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. rPrrjc 3h># UH'n[TRTRTTSSS9v M) g7frrrs rKr#C2fAttn.__init__..rr)rnrrfN) r:r;rZrrrlror<rrrrdattn) rHrGrrrrrfrnrrrrJs ` `` rKr;C2fAttn.__init__s4 RVAJ1-Q$&&("a0tkpqrkstt'2L rMc6^[URU5RSS55mTRU4SjUR55 TR UR TSU55 UR[R"TS55$)z Forward pass through C2f layer with attention. Args: x (torch.Tensor): Input tensor. guide (torch.Tensor): Guide tensor for attention. Returns: (torch.Tensor): Output tensor after processing. rPrc38># UHo"TS5v M g7frrrs rKr"C2fAttn.forward..rrr) rrlrrrappendrror@rrHrIrurs @rKrWC2fAttn.forwardsn !""1a( ) *466** 1R5%()xx !Q((rMcb^[URU5RURUR4S55mTR U4SjUR 55 TR URTSU55 UR[R"TS55$)z Forward pass using split() instead of chunk(). Args: x (torch.Tensor): Input tensor. guide (torch.Tensor): Guide tensor for attention. Returns: (torch.Tensor): Output tensor after processing. rc38># UHo"TS5v M g7frrrs rKr(C2fAttn.forward_split..rrr) rrlrrrrrrror@rrs @rKrC2fAttn.forward_splits{ !""DFFDFF#3Q7 8 *466** 1R5%()xx !Q((rM)rrrlror)rr|rr}Frr)rGrZrrrZrrZrrZrfrZrnrZrrrrZrrBr~r rfs@rKrrs4 M M M  M  M  M MM M MMB) ))rMrcV^\rSrSrSrSSU4SjjjrSSjrSrU=r$) rizKImagePoolingAttn: Enhance the text embeddings with image-aware information.c >[T U]5 [U5n[R"[R "U5[R "X155Ul[R"[R "U5[R "X55Ul[R"[R "U5[R "X55Ul [R "X5Ul U(a*[R"[R"S/5SS9OSUl[R"UVs/sHn[R "XSS9PM sn5Ul[R"[%U5V s/sHn [R&"XU45PM sn 5UlXlX@lXplX-UlXPlgs snfs sn f)aC Initialize ImagePoolingAttn module. Args: ec (int): Embedding channels. ch (tuple): Channel dimensions for feature maps. ct (int): Channel dimension for text embeddings. nh (int): Number of attention heads. k (int): Kernel size for pooling. scale (bool): Whether to use learnable scale parameter. gT requires_gradrr)r~N)r:r;rr<r LayerNormrhquerykeyvalueprojrCr@tensorrmrr= projectionsrAdaptiveMaxPool2dim_poolsrrfnfrgrk) rHrchctrfrkrmr in_channelsrUrJs rKr;ImagePoolingAttn.__init__s>  W]]2<<#3RYYr5FG ==b!1299R3DE]]2<<#3RYYr5FG IIb% NSR\\%,,u"5TJY\ ==gi)jgiXc"))KQR*Sgi)jk USUY&WYr';';QF'CY&WX ( *k&Ws G%!G*c USRSn[U5UR:XdeURS-n[ XR UR 5VVVs/sH$upof"U"U55RUSU5PM& snnnn[R"USS9RSS5nURU5nURU5nURU5n URUSURUR 5nURUSURUR 5nU RUSURUR 5n [R""SXx5n XR S-- n [$R&"U SS9n [R""SX5nUR)URUSUR*55nXR,-U-$s snnnf) z Forward pass of ImagePoolingAttn. Args: x (list[torch.Tensor]): List of input feature maps. text (torch.Tensor): Text embeddings. Returns: (torch.Tensor): Enhanced text embeddings. rrPrrrzbnmc,bkmc->bmnkrzbmnk,bkmc->bnmc)rQrrrkziprrrDr@rrRrrrreshaperfrgrqrrSrrrm) rHrItextrv num_patchesrrqrkvrzs rKrWImagePoolingAttn.forwardsqTZZ]1v   ffai LOPQScSceiererLs tLs!4T$q']  B 4Ls t IIaR * *1a 0 JJt  HHQK JJqM IIb"dggtww / IIb"dggtww / IIb"dggtww / \\+Q 2 77C<  YYrr " LL*B 2 IIaiiB0 1::~$$# us!+G2) rrgrrkrrrfrrrrmr)rwrr}rjF) rrZrrrrZrfrZrkrZrmr)rIlist[torch.Tensor]rr\r]r\r^rfs@rKrrsQUns!0;>JMVYfj<%%rMrc6^\rSrSrSrU4SjrSSjrSrU=r$)ri zZImplements contrastive learning head for region-text similarity in vision-language models.c*>[TU]5 [R"[R "S/55Ul[R"[R"/5[R "S5R5-5Ul g)zBInitialize ContrastiveHead with region-text similarity parameters.$g$I$I,@N) r:r;r<rCr@rr8rllog logit_scale)rHrJs rKr;ContrastiveHead.__init__ sY LLug!67 << 2h9O9S9S9U(UVrMc[R"USSS9n[R"USSS9n[R"SX5nXRR 5-UR -$)z Forward function of contrastive learning. Args: x (torch.Tensor): Image features. w (torch.Tensor): Text features. Returns: (torch.Tensor): Similarity scores. rrPrrTrbchw,bkc->bkhw)r normalizer@rqrexpr8rHrIrxs rKrWContrastiveHead.forwardsZ KKqA & KKrQ ' LL)1 0##''))DII55rM)r8rrIr\rxr\r]r\r^rfs@rKrr sdW66rMrcJ^\rSrSrSrSU4SjjrSrS SjrS SjrSr U=r $) ri$z Batch Norm Contrastive Head using batch norm instead of l2-normalization. Args: embed_dims (int): Embed dimensions of text and image features. c>[TU]5 [R"U5Ul[R "[ R"S/55Ul[R "S[ R"/5-5Ul g)z_ Initialize BNContrastiveHead. Args: embed_dims (int): Embedding dimensions for features. rgN) r:r;r<rGnormrCr@rr8rlr)rH embed_dimsrJs rKr;BNContrastiveHead.__init__,sY NN:. LLug!67 <<uzz"~(=>rMc2U?U?U?URUlg)zCFuse the batch normalization layer in the BNContrastiveHead module.N)rr8r forward_fuserW)rHs rKfuseBNContrastiveHead.fuse:s I I  (( rMcU$)zPasses input out unchanged.rrs rKrBNContrastiveHead.forward_fuseAsrMcURU5n[R"USSS9n[R"SX5nXR R 5-UR-$)z Forward function of contrastive learning with batch normalization. Args: x (torch.Tensor): Image features. w (torch.Tensor): Text features. Returns: (torch.Tensor): Similarity scores. rrPrr)rrrr@rqrrr8rs rKrWBNContrastiveHead.forwardEsU IIaL KKrQ ' LL)1 0##''))DII55rM)r8rWrr)rrZr) r_r`rarbrcr;rrrWrdrerfs@rKrr$s! ?)66rMrcL^\rSrSrSrSSU4SjjjrSrU=r$) RepBottleneckiWzRep bottleneck.cl>[TU]XX4XV5 [X&-5n[XUSS5Ulg)a  Initialize RepBottleneck. Args: c1 (int): Input channels. c2 (int): Output channels. shortcut (bool): Whether to use shortcut connection. g (int): Groups for convolutions. k (tuple): Kernel sizes for convolutions. e (float): Expansion ratio. rrN)r:r;rZr rlr:s rKr;RepBottleneck.__init__Zs5 a3 [21Q4+rMrlr>r?rrfs@rKrrWsGlo,,,*.,:=,FU,ch,,rMrc4^\rSrSrSrSSU4SjjjrSrU=r$)RepCSPimzXRepeatable Cross Stage Partial Network (RepCSP) module for efficient feature extraction.c>^^^[TU]XUTTU5 [X&-5m[R"UUU4Sj[ U556Ulg)a  Initialize RepCSP layer. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of RepBottleneck blocks. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. c 3>># UHn[TTTTSS9v M g7frC)rrs rKr"RepCSP.__init__..~s ]T\qr2xc!JT\rNr-rs `` @rKr;RepCSP.__init__psB Ha3 [ ]TYZ[T\ ]^rMrrrrrfs@rKrrmsb__rMrcH^\rSrSrSrSSU4SjjjrS SjrS SjrSrU=r $) r#iz CSP-ELAN.c V>[TU]5 US-Ul[XSS5Ul[ R "[US-XE5[XDSS55Ul[ R "[XDU5[XDSS55Ul [USU--USS5Ul g)z Initialize CSP-ELAN layer. Args: c1 (int): Input channels. c2 (int): Output channels. c3 (int): Intermediate channels. c4 (int): Intermediate channels for RepCSP. n (int): Number of RepCSP blocks. rPrrjN) r:r;rrrlr<rrrorprF)rHrGrrrUc4rrJs rKr;RepNCSPELAN4.__init__s q1%==a!7ba9KL==!2DA4FGa"f r1a0rMc^[URU5RSS55mTRU4SjURUR 455 UR [R"TS55$)z(Forward pass through RepNCSPELAN4 layer.rPrc38># UHo"TS5v M g7frrrs rKr'RepNCSPELAN4.forward..s:%9!AbE((%9r) rrlrrrorprFr@rrs @rKrWRepNCSPELAN4.forwardsZ !""1a( ) :dhh%9::xx !Q((rMc2^[URU5RURUR4S55mTR U4SjUR UR 455 UR[R"TS55$)rrc38># UHo"TS5v M g7frrrs rKr-RepNCSPELAN4.forward_split..s8#7a1R5#7r) rrlrrrrorprFr@rrs @rKrRepNCSPELAN4.forward_splitsg !""DFFDFF#3Q7 8 8DHHdhh#788xx !Q((rMrrlrorprFr) rGrZrrrZrUrZrrZrrZr[r rfs@rKr#r#s11$) ))rMr#c0^\rSrSrSrSU4SjjrSrU=r$)r$iz!ELAN1 module with 4 convolutions.c>[TU]XX45 US-Ul[XSS5Ul[US-USS5Ul[XDSS5Ul[USU--USS5Ulg)z Initialize ELAN1 layer. Args: c1 (int): Input channels. c2 (int): Output channels. c3 (int): Intermediate channels. c4 (int): Intermediate channels for convolutions. rPrrjN)r:r;rrrlrorprF)rHrGrrrUrrJs rKr;ELAN1.__init__so (q1%aQ*1%a"f r1a0rMr)rGrZrrrZrUrZrrZrrfs@rKr$r$s+11rMr$c:^\rSrSrSrSU4SjjrSSjrSrU=r$)r&izAConv.cH>[TU]5 [XSSS5Ulg)z^ Initialize AConv module. Args: c1 (int): Input channels. c2 (int): Output channels. rjrPrN)r:r;rrlrHrGrrrJs rKr;AConv.__init__s" 1a(rMc[RRRUSSSSS5nUR U5$)z!Forward pass through AConv layer.rPrrFT)r@r< functional avg_pool2drlrus rKrW AConv.forwards4 HH   * *1aAud Cxx{rMrrGrZrrrZr[r^rfs@rKr&r&s )rMr&c:^\rSrSrSrSU4SjjrSSjrSrU=r$)r%izADown.c>[TU]5 US-Ul[US-URSSS5Ul[US-URSSS5Ulg)z^ Initialize ADown module. Args: c1 (int): Input channels. c2 (int): Output channels. rPrjrrN)r:r;rrrlrors rKr;ADown.__init__sS qaAq1aAq1rMcR[RRRUSSSSS5nUR SS5up#UR U5n[RRR USSS5nURU5n[R"X#4S5$)z!Forward pass through ADown layer.rPrrFTrj) r@r<rrrrl max_pool2dror)rHrIrrs rKrW ADown.forwards HH   * *1aAud CA XXb\ XX + +B1a 8 XXb\yy"1%%rM)rrlrorr[r^rfs@rKr%r%s 2&&rMr%c>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r'iz SPP-ELAN.c2>[TU]5 X0l[XSS5Ul[ R "USUS-S9Ul[ R "USUS-S9Ul[ R "USUS-S9Ul [SU-USS5Ul g)z Initialize SPP-ELAN block. Args: c1 (int): Input channels. c2 (int): Output channels. c3 (int): Intermediate channels. k (int): Kernel size for max pooling. rrPrrON) r:r;rrrlr<rrorprFcv5)rHrGrrrUrkrJs rKr;SPPELAN.__init__s 1%<# UHo"TS5v M g7frrrs rKr"SPPELAN.forward..sB#Aa1R5#Arr)rlrrorprFrr@rrs @rKrWSPPELAN.forwardsP XXa[M BDHHdhh#ABBxx !Q((rM)rrlrorprFrr)rGrZrrrZrUrZrkrZr[r^rfs@rKr'r's**$))rMr'c>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r)iz CBLinear.c >[TU]5 X l[R"U[ U5X4[ X55USS9Ulg)z Initialize CBLinear module. Args: c1 (int): Input channels. c2s (list[int]): List of output channel sizes. k (int): Kernel size. s (int): Stride. p (int | None): Padding. g (int): Groups. T)groupsr8N)r:r;c2sr<r=sumr r?)rHrGrrkr1rTrrJs rKr;CBLinear.__init__s8 IIb#c(A'!-PTU rMcTURU5RURSS9$)z$Forward pass through CBLinear layer.rr)r?rrrus rKrWCBLinear.forwards$yy|!!$((!22rM)rr?)rrNr) rGrZr list[int]rkrZr1rZrTz int | NonerrZ)rIr\r]rr^rfs@rKr)r)sVV 33rMr)c:^\rSrSrSrSU4SjjrSSjrSrU=r$)r(izCBFuse.c.>[TU]5 Xlg)zV Initialize CBFuse module. Args: idx (list[int]): Indices for feature selection. N)r:r;idx)rHrrJs rKr;CBFuse.__init__s rMc USRSSn[USS5VVs/sH*up4[R"X@RUUSS9PM, nnn[ R "[ R"XQSS-5SS9$s snnf)z Forward pass through CBFuse layer. Args: xs (list[torch.Tensor]): List of input tensors. Returns: (torch.Tensor): Fused output tensor. rrPNnearest)sizemoderr)rQ enumerater interpolaterr@rstack)rHxs target_sizerrIress rKrWCBFuse.forward'sfll12& [deghkikel[mn[mSWSTq}}Qxx{^+IN[mnyySbc7]3;;os1B)r)rr)r rr]r\r^rfs@rKr(r(s < ^\rSrSrSrSSU4SjjjrSSjrSrU=r$) C3fi6rc>^^^[TU]5 [X&-5m[UTSS5Ul[UTSS5Ul[SU-T-US5Ul[R"UUU4Sj[U555Ul g)a& Initialize CSP bottleneck layer with two convolutions. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of Bottleneck blocks. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. rrPc 3@># UHn[TTTTSSS9v M g7frrrs rKrC3f.__init__..Js%lck^_z"b(AAQUXYckrN) r:r;rZrrlrorpr<rrrrs `` @rKr; C3f.__init__9st  [B1%B1%Q" b!,lchijckllrMc^URU5URU5/mTRU4SjUR55 UR [ R "TS55$)zForward pass through C3f layer.c38># UHo"TS5v M g7frrrs rKrC3f.forward..Orrr)rorlrrrpr@rrs @rKrW C3f.forwardLsL XXa[$((1+ & *466**xx !Q((rMrr rr[r^rfs@rKrr6sFmm&))rMrcP^\rSrSrSrSSU4SjjjrSrU=r$)r*iSrc>^^^^[TT]XUTTU5 [R"UUUU4Sj[ U555Tlg)a' Initialize C3k2 module. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of blocks. c3k (bool): Whether to use C3k blocks. e (float): Expansion ratio. g (int): Groups for convolutions. shortcut (bool): Whether to use shortcut connections. c3># UHQnT(a#[TRTRSTT5O![TRTRTT5v MS g7f)rPN)C3krr)rrUc3krrHrs rKr C3k2.__init__..fsG muhi3C8Q /JtvvtvvW_ab^^^^[T U]XUTTU5 [X&-5m[R"UUUU4Sj[ U556Ulg)a  Initialize C3k module. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of Bottleneck blocks. shortcut (bool): Whether to use shortcut connections. g (int): Groups for convolutions. e (float): Expansion ratio. k (int): Kernel size. c 3D># UHn[TTTTTT4SS9v M g7f)rrNr)rrUrqrrkrs rKrC3k.__init__..~s' d[cVWBHaAq6S!Q[cs Nr-) rHrGrrrrrrrkrqrJs `` `@rKr; C3k.__init__nsB Ha3 [ d[`ab[c derMr)rTrrrj)rGrZrrrZrrZrrrrZrrBrkrZrrfs@rKrrksrffrMrcr^\rSrSrSrSU4SjjrS SjrS Sjr\R"5S5r Sr U=r $) r-izfRepVGGDW is a class that represents a depth wise separable convolutional block in RepVGG architecture.c >[TU]5 [XSSSUSS9Ul[XSSSUSS9UlXl[ R"5Ulg)zM Initialize RepVGGDW module. Args: ed (int): Input and output channels. r\rrjFrr}N) r:r;rr?conv1rr<rIr})rHedrJs rKr;RepVGGDW.__init__sN AqBE: "!QRU; 779rMcfURURU5URU5-5$)z Perform a forward pass of the RepVGGDW block. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after applying the depth wise separable convolution. )r}r?r*rus rKrWRepVGGDW.forwards(xx ! tzz!}455rMcBURURU55$)z Perform a forward pass of the RepVGGDW block without fusing the convolutions. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after applying the depth wise separable convolution. )r}r?rus rKrRepVGGDW.forward_fusesxx ! %%rMc@[URRURR5n[URRURR5nURnUR nURnUR n[ RRRU/SQ5nX5-nXF-nURRRU5 UR RRU5 XlU?g)z Fuse the convolutional layers in the RepVGGDW block. This method fuses the convolutional layers and updates the weights and biases accordingly. )rPrPrPrPN) rr?rHr*rEr8r@r<rrrFcopy_) rHr?r*conv_wconv_bconv1_wconv1_b final_conv_w final_conv_bs rKr RepVGGDW.fuses   = $**--@,,**((%%))'<@' '  |, \* JrM)r}r?r*r)r+rZr]Noner[) r_r`rarbrcr;rWrr@no_gradrrdrerfs@rKr-r-s/p  6 & ]]_rMr-c>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r.ia Conditional Identity Block (CIB) module. Args: c1 (int): Number of input channels. c2 (int): Number of output channels. shortcut (bool, optional): Whether to add a shortcut connection. Defaults to True. e (float, optional): Scaling factor for the hidden channels. Defaults to 0.5. lk (bool, optional): Whether to use RepVGGDW for the third convolutional layer. Defaults to False. c J>[TU]5 [X$-5n[R"[ XSUS9[ USU-S5U(a[ SU-5O[ SU-SU-SSU-S9[ SU-US5[ X"SUS95UlU=(a X:HUlg)z Initialize the CIB module. Args: c1 (int): Input channels. c2 (int): Output channels. shortcut (bool): Whether to use shortcut connection. e (float): Expansion ratio. lk (bool): Whether to use RepVGGDW. rjr9rPrN) r:r;rZr<rrr-rlr)rHrGrrrrlkrqrJs rKr; CIB.__init__s  [== b ! QVQ  "HQV QVQVQ!b&(I RQ  b !  (rMclUR(aXRU5-$URU5$)zy Forward pass of the CIB module. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor. rrlrus rKrW CIB.forwards'#'((q88A;; ;rMrA)TrF) rGrZrrrZrrrrBr>rr[r^rfs@rKr.r.s )). < ^^^[TT]XUTXg5 [R"UUU4Sj[ U555Tlg)a6 Initialize C2fCIB module. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of CIB modules. shortcut (bool): Whether to use shortcut connection. lk (bool): Whether to use local key connection. g (int): Groups for convolutions. e (float): Expansion ratio. c 3f># UH&n[TRTRTSTS9v M( g7f)r)rr>N)r.r)rrUr>rHrs rKr"C2fCIB.__init__..s(]T\qs4664668srJT\s.1Nr ) rHrGrrrrr>rrrJs ` `` rKr;C2fCIB.__init__s5 Ha3]TYZ[T\]]rMr)rFFrr)rGrZrrrZrrZrrr>rrrZrrBrrfs@rKr/r/sZ nq^^^#&^6:^HL^Y\^ej^^rMr/c>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r0ia Attention module that performs self-attention on the input tensor. Args: dim (int): The input tensor dimension. num_heads (int): The number of attention heads. attn_ratio (float): The ratio of the attention key dimension to the head dimension. Attributes: num_heads (int): The number of attention heads. head_dim (int): The dimension of each attention head. key_dim (int): The dimension of the attention key. scale (float): The scaling factor for the attention scores. qkv (Conv): Convolutional layer for computing the query, key, and value. proj (Conv): Convolutional layer for projecting the attended values. pe (Conv): Convolutional layer for positional encoding. c 2>[TU]5 X lX-Ul[ URU-5UlUR S-UlUR U-nXS--n[XSSS9Ul[XSSS9Ul [XSSUSS9Ul g) z Initialize multi-head attention module. Args: dim (int): Input dimension. num_heads (int): Number of attention heads. attn_ratio (float): Attention ratio for key dimension. rPrFr|rjr)N) r:r; num_headshead_dimrZkey_dimrmrqkvrpe)rHrrK attn_rationh_kdrwrJs rKr;Attention.__init__(s "( 4==:56 \\4'  y( !)Ou-1%0 sA%8rMc 0URup#pEXE-nURU5nURX RURS-UR -U5R URURUR /SS9upn URSS5U -UR-n U RSS9n XRSS5-RX#XE5URU RX#XE55-nURU5nU$)z Forward pass of the Attention module. Args: x (torch.Tensor): The input tensor. Returns: (torch.Tensor): The output tensor after self-attention. rPrr) rQrNrDrKrMrLrrRrmrSrOrr) rHrIBCHWNrNrrkrrs rKrWAttention.forward<sWW a Ehhqk((1nndllQ.>.NPQRXX \\4<< 7QY a B#a'4::5|||# B' ' - -aA 9DGGAIIaTUDY^\rSrSrSrSSU4SjjjrSSjrSrU=r$) PSABlockiTa PSABlock class implementing a Position-Sensitive Attention block for neural networks. This class encapsulates the functionality for applying multi-head attention and feed-forward neural network layers with optional shortcut connections. Attributes: attn (Attention): Multi-head attention module. ffn (nn.Sequential): Feed-forward neural network module. add (bool): Flag indicating whether to add shortcut connections. Methods: forward: Performs a forward pass through the PSABlock, applying attention and feed-forward layers. Examples: Create a PSABlock and perform a forward pass >>> psablock = PSABlock(c=128, attn_ratio=0.5, num_heads=4, shortcut=True) >>> input_tensor = torch.randn(1, 128, 32, 32) >>> output_tensor = psablock(input_tensor) c >[TU]5 [XUS9Ul[R "[ XS-S5[ US-USSS95UlX@lg)z Initialize the PSABlock. Args: c (int): Input and output channels. attn_ratio (float): Attention ratio for key dimension. num_heads (int): Number of attention heads. shortcut (bool): Whether to use shortcut connections. rPrKrPrFr|N) r:r;r0rr<rrffnr)rHrrPrKrrJs rKr;PSABlock.__init__jsO a)L ==aQ!2DQ1%4PQrMcUR(aXRU5-OURU5nUR(aXRU5-nU$URU5nU$)z Execute a forward pass through PSABlock. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after attention and feed-forward processing. rrr_rus rKrWPSABlock.forwardzsR!%A ! diil#xxA O.2XXa[rMrb)rrOT) rrZrPrBrKrZrrr]r:r[r^rfs@rKr\r\Ts*  rMr\c>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r1ia PSA class for implementing Position-Sensitive Attention in neural networks. This class encapsulates the functionality for applying position-sensitive attention and feed-forward networks to input tensors, enhancing feature extraction and processing capabilities. Attributes: c (int): Number of hidden channels after applying the initial convolution. cv1 (Conv): 1x1 convolution layer to reduce the number of input channels to 2*c. cv2 (Conv): 1x1 convolution layer to reduce the number of output channels to c. attn (Attention): Attention module for position-sensitive attention. ffn (nn.Sequential): Feed-forward network for further processing. Methods: forward: Applies position-sensitive attention and feed-forward network to the input tensor. Examples: Create a PSA module and apply it to an input tensor >>> psa = PSA(c1=128, c2=128, e=0.5) >>> input_tensor = torch.randn(1, 128, 64, 64) >>> output_tensor = psa.forward(input_tensor) c >[TU]5 X:Xde[X-5Ul[ USUR-SS5Ul[ SUR-US5Ul[URSURS-S9Ul[R"[ URURS-S5[ URS-URSSS95Ul g) z| Initialize PSA module. Args: c1 (int): Input channels. c2 (int): Output channels. e (float): Expansion ratio. rPrr@r^Fr|N) r:r;rZrrrlror0rr<rr_)rHrGrrrrJs rKr; PSA.__init__s xxRVAJ1-DFF B*dff" M ==dffdffqj!!derMc URU5RURUR4SS9up#X0RU5-nX0R U5-nUR [ R"X#4S55$)z Execute forward pass in PSA module. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after attention and feed-forward processing. rr)rlrrrr_ror@rrs rKrW PSA.forwardskxx{  $&&$&&!1q 9 !   Oxx 1&!,--rM)rrrlror_)r)rGrZrrrZrrBr[r^rfs@rKr1r1s.ff$ . .rMr1c>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r,ia C2PSA module with attention mechanism for enhanced feature extraction and processing. This module implements a convolutional block with attention mechanisms to enhance feature extraction and processing capabilities. It includes a series of PSABlock modules for self-attention and feed-forward operations. Attributes: c (int): Number of hidden channels. cv1 (Conv): 1x1 convolution layer to reduce the number of input channels to 2*c. cv2 (Conv): 1x1 convolution layer to reduce the number of output channels to c. m (nn.Sequential): Sequential container of PSABlock modules for attention and feed-forward operations. Methods: forward: Performs a forward pass through the C2PSA module, applying attention and feed-forward operations. Notes: This module essentially is the same as PSA module, but refactored to allow stacking more PSABlock modules. Examples: >>> c2psa = C2PSA(c1=256, c2=256, n=3, e=0.5) >>> input_tensor = torch.randn(1, 256, 64, 64) >>> output_tensor = c2psa(input_tensor) c*>^[TT]5 X:Xde[X-5Tl[ USTR-SS5Tl[ STR-US5Tl[R"U4Sj[U556Tl g)z Initialize C2PSA module. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of PSABlock modules. e (float): Expansion ratio. rPrc3h># UH'n[TRSTRS-S9v M) g7frrfr^Nr\rrrUrHs rKr!C2PSA.__init__..s* lck^_$&&SDFFVXL!YckrNrrHrGrrrrrJs` rKr;C2PSA.__init__su xxRVAJ1-DFF B* lchijck lmrMcURU5RURUR4SS9up#URU5nUR [ R "X#4S55$)z Process the input tensor through a series of PSA blocks. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after processing. rr)rlrrrror@rrs rKrW C2PSA.forwardsYxx{  $&&$&&!1q 9 FF1Ixx 1&!,--rMrrrr%r[r^rfs@rKr,r,s0nn$ . .rMr,c4^\rSrSrSrSSU4SjjjrSrU=r$)r+iaW C2fPSA module with enhanced feature extraction using PSA blocks. This class extends the C2f module by incorporating PSA blocks for improved attention mechanisms and feature extraction. Attributes: c (int): Number of hidden channels. cv1 (Conv): 1x1 convolution layer to reduce the number of input channels to 2*c. cv2 (Conv): 1x1 convolution layer to reduce the number of output channels to c. m (nn.ModuleList): List of PSA blocks for feature extraction. Methods: forward: Performs a forward pass through the C2fPSA module. forward_split: Performs a forward pass using split() instead of chunk(). Examples: >>> import torch >>> from ultralytics.models.common import C2fPSA >>> model = C2fPSA(c1=64, c2=64, n=3, e=0.5) >>> x = torch.randn(1, 64, 128, 128) >>> output = model(x) >>> print(output.shape) c>^X:Xde[TT]XX4S9 [R"U4Sj[ U555Tlg)z Initialize C2fPSA module. Args: c1 (int): Input channels. c2 (int): Output channels. n (int): Number of PSABlock modules. e (float): Expansion ratio. )rrc3h># UH'n[TRSTRS-S9v M) g7frmrnros rKr"C2fPSA.__init__.."s*jai\]x3$&&TV,WairNr rqs` rKr;C2fPSA.__init__s=xx 1*jafghaijjrMrrur%rrfs@rKr+r+s0 k krMr+c:^\rSrSrSrSU4SjjrSSjrSrU=r$)r2i%a SCDown module for downsampling with separable convolutions. This module performs downsampling using a combination of pointwise and depthwise convolutions, which helps in efficiently reducing the spatial dimensions of the input tensor while maintaining the channel information. Attributes: cv1 (Conv): Pointwise convolution layer that reduces the number of channels. cv2 (Conv): Depthwise convolution layer that performs spatial downsampling. Methods: forward: Applies the SCDown module to the input tensor. Examples: >>> import torch >>> from ultralytics import SCDown >>> model = SCDown(c1=64, c2=128, k=3, s=2) >>> x = torch.randn(1, 64, 128, 128) >>> y = model(x) >>> print(y.shape) torch.Size([1, 128, 64, 64]) c h>[TU]5 [XSS5Ul[X"X4USS9Ulg)z Initialize SCDown module. Args: c1 (int): Input channels. c2 (int): Output channels. k (int): Kernel size. s (int): Stride. rF)rkr1rr}N)r:r;rrlro)rHrGrrrkr1rJs rKr;SCDown.__init__=s2 1%!BE:rMcBURURU55$)z Apply convolution and downsampling to the input tensor. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Downsampled output tensor. )rorlrus rKrWSCDown.forwardKsxx $$rM)rlror6r[r^rfs@rKr2r2%s. ; % %rMr2cR^\rSrSrSrSSU4SjjjrSSjrSrU=r$) r3iXa. TorchVision module to allow loading any torchvision model. This class provides a way to load a model from the torchvision library, optionally load pre-trained weights, and customize the model by truncating or unwrapping layers. Attributes: m (nn.Module): The loaded torchvision model, possibly truncated and unwrapped. Args: model (str): Name of the torchvision model to load. weights (str, optional): Pre-trained weights to load. Default is "DEFAULT". unwrap (bool, optional): If True, unwraps the model to a sequential containing all but the last `truncate` layers. Default is True. truncate (int, optional): Number of layers to truncate from the end if `unwrap` is True. Default is 2. split (bool, optional): Returns output from intermediate child modules as list. Default is False. c>SSKn[TU] 5 [URS5(aURR XS9UlO+URRU"[U5S9UlU(a[UR R55n[US[R5(a#/[USR55QUSSQn[R"U(aUSU*OU6UlXPlgSUl[R"5=UR lUR lg)a. Load the model and weights from torchvision. Args: model (str): Name of the torchvision model to load. weights (str): Pre-trained weights to load. unwrap (bool): Whether to unwrap the model. truncate (int): Number of layers to truncate. split (bool): Whether to split the output. rN get_model)weights) pretrainedrF) torchvisionr:r;hasattrmodelsrr__dict__rrchildren isinstancer<rrr!headheads) rHmodelrunwraptruncaterrlayersrJs rKr;TorchVision.__init__is   ;%%{ 3 3 ''11%1IDF ''0074=QDF $&&//+,F&)R]]33C4q 2 2 45Cqr C]]8VJhY%7QDFJDJ)+ 6DFFK$&&,rMc^UR(a*U/mTRU4SjUR55 T$URU5mT$)z Forward pass through the model. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor | list[torch.Tensor]): Output tensor or list of tensors. c38># UHo"TS5v M g7frrrs rKr&TorchVision.forward..s.v!QquXXvr)rrrrs @rKrWTorchVision.forwardsD ::A HH.tvv. .q ArM)rr)DEFAULTTrPF) rstrrrrrrrZrrr[r^rfs@rKr3r3XsK"kp77#&7<@7SV7cg77<rMr3c>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) AAttnia Area-attention module for YOLO models, providing efficient attention mechanisms. This module implements an area-based attention mechanism that processes input features in a spatially-aware manner, making it particularly effective for object detection tasks. Attributes: area (int): Number of areas the feature map is divided. num_heads (int): Number of heads into which the attention mechanism is divided. head_dim (int): Dimension of each attention head. qkv (Conv): Convolution layer for computing query, key and value tensors. proj (Conv): Projection convolution layer. pe (Conv): Position encoding convolution layer. Methods: forward: Applies area-attention to input tensor. Examples: >>> attn = AAttn(dim=256, num_heads=8, area=4) >>> x = torch.randn(1, 256, 32, 32) >>> output = attn(x) >>> print(output.shape) torch.Size([1, 256, 32, 32]) c >[TU]5 X0lX lX-=UlnX@R-n[ XS-SSS9Ul[ XQSSS9Ul[ XQSSSUSS9Ulg)z Initialize an Area-attention module for YOLO models. Args: dim (int): Number of hidden channels. num_heads (int): Number of heads into which the attention mechanism is divided. area (int): Number of areas the feature map is divided. rjrFr|r\r)N) r:r;arearKrLrrNrrO)rHrrKrrL all_head_dimrJs rKr;AAttn.__init__sp  "#3 ..0 A-qe<A59 |!QSeDrMcURup#pEXE-nURU5RS5RSS5nURS:a=UR X R-X`R-US-5nURup&nUR X&URURS-5RSSSS5RURURUR/SS9upn U RSS5U -URS--n U RSS9n XRSS5-nURSSSS5nU RSSSS5n URS:aeUR X R-X`R-U5nU R X R-X`R-U5n URup&nUR X$XS5RSSSS5R5nU R X$XS5RSSSS5R5n XRU 5-nURU5$) z Process the input tensor through the area-attention. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after area-attention. rPrrjrrrTrrJ)rQrNflattenrRrrrDrKrLpermuterrS contiguousrOr) rHrIrUrVrWrXrYrNrUrrkrrs rKrW AAttn.forwardsWW a Ehhqk!!!$..q!4 99q=++a))mQ))^QUCCiiGA! HHQ4>>4==1+< = WQ1a UDMM4==$--@aU H a  B#a'DMM4,?@|||# r2& & IIaAq ! IIaAq ! 99q= !yy.!ii-;A !yy.!ii-;AggGA! IIaA ! ) )!Q1 5 @ @ B IIaA ! ) )!Q1 5 @ @ B  Nyy|rM)rrLrKrOrrNr)rrZrKrZrrZr[r^rfs@rKrrs2EE(%%rMrcH^\rSrSrSrSSU4SjjjrS SjrS SjrSrU=r $) ABlockiah Area-attention block module for efficient feature extraction in YOLO models. This module implements an area-attention mechanism combined with a feed-forward network for processing feature maps. It uses a novel area-based attention approach that is more efficient than traditional self-attention while maintaining effectiveness. Attributes: attn (AAttn): Area-attention module for processing spatial features. mlp (nn.Sequential): Multi-layer perceptron for feature transformation. Methods: _init_weights: Initializes module weights using truncated normal distribution. forward: Applies area-attention and feed-forward processing to input tensor. Examples: >>> block = ABlock(dim=256, num_heads=8, mlp_ratio=1.2, area=1) >>> x = torch.randn(1, 256, 32, 32) >>> output = block(x) >>> print(output.shape) torch.Size([1, 256, 32, 32]) c >[TU]5 [XUS9Ul[ X-5n[ R "[XS5[XQSSS95UlURUR5 g)a- Initialize an Area-attention block module. Args: dim (int): Number of input channels. num_heads (int): Number of heads into which the attention mechanism is divided. mlp_ratio (float): Expansion ratio for MLP hidden dimension. area (int): Number of areas the feature map is divided. )rKrrFr|N) r:r;rrrZr<rrmlpapply _init_weights)rHrrK mlp_ratiormlp_hidden_dimrJs rKr;ABlock.__init__s` #> S_-==c1!=tNYZ`e?fg 4%%&rMc[U[R5(aa[RR UR SS9 UR b+[RRUR S5 ggg)zk Initialize weights using a truncated normal distribution. Args: m (nn.Module): Module to initialize. g{Gz?)stdNr)rr<r=init trunc_normal_rEr8 constant_)rHrs rKrABlock._init_weightss\ a # # GG ! !!(( ! 5vv!!!!&&!," $rMcNXRU5-nXRU5-$)z Forward pass through ABlock. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after area-attention and feed-forward processing. rrrus rKrWABlock.forward%s$  ! 88A;rMr)g333333?r)rrZrKrZrrBrrZ)rrr[) r_r`rarbrcr;rrWrdrerfs@rKrrs!.''$ -  rMrct^\rSrSrSrSSU4SjjjrSSjrSrU=r$) A2C2fi3a Area-Attention C2f module for enhanced feature extraction with area-based attention mechanisms. This module extends the C2f architecture by incorporating area-attention and ABlock layers for improved feature processing. It supports both area-attention and standard convolution modes. Attributes: cv1 (Conv): Initial 1x1 convolution layer that reduces input channels to hidden channels. cv2 (Conv): Final 1x1 convolution layer that processes concatenated features. gamma (nn.Parameter | None): Learnable parameter for residual scaling when using area attention. m (nn.ModuleList): List of either ABlock or C3k modules for feature processing. Methods: forward: Processes input through area-attention or standard convolution pathway. Examples: >>> m = A2C2f(512, 512, n=1, a2=True, area=1) >>> x = torch.randn(1, 512, 32, 32) >>> output = m(x) >>> print(output.shape) torch.Size([1, 512, 32, 32]) c >^^^^ ^ ^ [T U]5 [X(-5m T S-S:XdS5e[UT SS5Ul[SU-T -US5UlT(a3U(a,[ R"S[R"U5-SS9OSUl [ R"UUU U UU 4S j[U555Ul g) a Initialize Area-Attention C2f module. Args: c1 (int): Number of input channels. c2 (int): Number of output channels. n (int): Number of ABlock or C3k modules to stack. a2 (bool): Whether to use area attention blocks. If False, uses C3k blocks instead. area (int): Number of areas the feature map is divided. residual (bool): Whether to use residual connections with learnable gamma parameter. mlp_ratio (float): Expansion ratio for MLP hidden dimension. e (float): Channel expansion ratio for hidden channels. g (int): Number of groups for grouped convolutions. shortcut (bool): Whether to use shortcut connections in C3k blocks. rxrz(Dimension of ABlock be a multiple of 32.rg{Gz?TrNc3># UHCnT(a([R"UUU4Sj[S556O[TTSTT5v ME g7f)c3F># UHn[TTS-TT5v M g7f)rxN)r)rrUrrqrs rKr+A2C2f.__init__...ps#T8aF2rRxDAA8rrPN)r<rrr)rrUa2rrqrrrs rKr!A2C2f.__init__..osJ  MMT5QR8T URQ!, -sA A)r:r;rZrrlror<rCr@rlgammarrr) rHrGrrrrrresidualrrrrrqrJs `` ` ``@rKr;A2C2f.__init__Ks8  [Bw!|GGG|B1%Q" b!,PRW_R\\$B"7tLei   1X   rMcP^URU5/mTRU4SjUR55 UR[R "TS55mUR b:XR RSUR RSSS5T--$T$)z Forward pass through A2C2f layer. Args: x (torch.Tensor): Input tensor. Returns: (torch.Tensor): Output tensor after processing. c38># UHo"TS5v M g7frrrs rKr A2C2f.forward..rrrrr) rlrrror@rrrDrQrs @rKrW A2C2f.forwardvsXXa[M *466** HHUYYq!_ % :: !zzr4::+;+;A+>1EII IrM)rlrorr)rTrFg@rrT)rGrZrrrZrrZrrrrZrrrrBrrBrrZrrr[r^rfs@rKrr3s6) )  )   )  )  ) ) )  )  ) ) ) VrMrc>^\rSrSrSrSSU4SjjjrSSjrSrU=r$) SwiGLUFFNiz@SwiGLU Feed-Forward Network for transformer-based architectures.c>[TU]5 [R"XU-5Ul[R"X2-S-U5Ulg)z Initialize SwiGLU FFN with input dimension, output dimension, and expansion factor. Args: gc (int): Guide channels. ec (int): Embedding channels. e (int): Expansion factor. rPN)r:r;r<rhw12w3)rHrnrrrJs rKr;SwiGLUFFN.__init__s< 99RR())AFaK,rMcURU5nURSSS9up4[R"U5U-nUR U5$)z.Apply SwiGLU transformation to input features.rPrr)rrrsilur)rHrIx12rrhiddens rKrWSwiGLUFFN.forwardsBhhqk1"%bwwvrM)rr)rO)rnrZrrZrrZr]r:r[r^rfs@rKrrsJ - -rMrc:^\rSrSrSrSU4SjjrSSjrSrU=r$)Residualiz7Residual connection wrapper for neural network modules.c">[TU]5 Xl[RR URR R5 [RR URR R5 g)zx Initialize residual module with the wrapped module. Args: m (nn.Module): Module to wrap with residual connection. N) r:r;rr<rzeros_rr8rE)rHrrJs rKr;Residual.__init__sQ  tvvyy~~& tvvyy''(rMc(XRU5-$)z,Apply residual connection to input features.rrus rKrWResidual.forwards66!9}rMr)rrr]r:r[r^rfs@rKrrsA )rMrc:^\rSrSrSrSU4SjjrSSjrSrU=r$)SAVPEizESpatial-Aware Visual Prompt Embedding module for feature enhancement.c >^[TU]5 [R"U4Sj[ U555Ul[R"U4Sj[ U555UlSUl[R"ST-US5Ul [R"ST-URSSS9Ul [R"SURSSS9Ul [R"[SUR-URS5[R"URURSSS95Ulg) z Initialize SAVPE module with channels, intermediate channels, and embedding dimension. Args: ch (list[int]): List of input channel dimensions. c3 (int): Intermediate channels. embed (int): Embedding dimension. c 3># UHeup[R"[UTS5[TTS5US;a[R"US-S9O[R"55v Mg g7f)rjrrPrP scale_factorNr<rrUpsampler!rrrIrUs rKr!SAVPE.__init__..se! & MMQARQTUY_T_!a%1Pegepeper  &sA-A0c3># UHYup[R"[UTS5US;a[R"US-S9O[R"55v M[ g7f)rrrPrNrrs rKrrsP! % MM$q"a.QRX["++1q5*I^`^i^i^k l l%sA!A$rYrjr)rrPN)r:r;r<rr rlrorr=rprFrrrcv6)rHrrUryrJs ` rKr;SAVPE.__init__s ==! "" !  ==! !" !   99QVUA.99QVTVVQ:99Q15==a$&&j$&&!!efrMcn[U5VVs/sHup4URU"U5PM nnnUR[R"USS95n[U5VVs/sHup4UR U"U5PM nnnUR [R"USS95nURupgpURSn URXgS5nURUSURX5RSU SSS5RXj-URX5nURXjSX5RXj-SX5nUR[R"XPRU54SS95nURXjURS5nURXjSS5nXR-[R"U5[R"UR 5R"--n [$R&"U SS9R)UR 5n U R+SS5URX`RXpR-S5R+SS5-n [$R,"U R+SS5RXjS5SSS9$s snnfs snnf)zJProcess input features and visual prompts to generate enhanced embeddings.rrrrTrPr)r rorFr@rrlrprQrDrrexpandrr logical_notfinfor9minrrStorRr) rHrIvprxirrUrVrWrXQscore aggregateds rKrW SAVPE.forwards*3A, 7,TXXa[_, 7 HHUYYqa( )*3A, 7,TXXa[_, 7 HHUYYqa( )WW a HHQK FF1  IIaDFFA ) 0 0QB C K KAESWSYSY[\ ` ZZa & . .qua > HHUYY88B<0a8 9 IIaDFFB ' ZZa $**2.QWW1E1I1III %R(++AGG4__R,qyyFFAKQS/T/^/^_ace/ff {{://B7??bIrUVWW1 8 8s !J+%!J1)rrlrorprFrr)rrrUrZryrZ)rIrrr\r]r\r^rfs@rKrrsOg8XXrMr)Jrc __future__rr@torch.nnr<torch.nn.functionalrrultralytics.utils.torch_utilsrr?rrrr r r transformerr __all__Moduler r rrrrrrrrrr!rrrrrrPr"rdrrrrrrr#r$r&r%r'r)r(rr*rr-r.r/r0r\r1r,r+r2r3rrrrrrrrMrKrs" :FF)( V\"))\6>BII>.#RYY#L+(bii+(\D"))D0)299)8,668 )")) )FJJ6x"x(;BII;2020(MbM(/bii/:PP8CBIIC@J"))J2"))@3$"))3$lB)biiB)J@%ryy@%F6bii6606 06f,J,,_R_()299)D1L1*BII(&BII&4)bii)83ryy30"))>BSBIISlARYYAHRBIIRj 0ryy,9XBII9XrM