hn:SSKJr SSKrSSKrSSKJrJr SSKJr "SS\R5r "SS\R5r "S S \R5r g) ) annotationsN)Tensornn)MLPBlockc^\rSrSrSr\R S4SU4SjjjrSSjrSr U=r $) TwoWayTransformer a0 A Two-Way Transformer module for simultaneous attention to image and query points. This class implements a specialized transformer decoder that attends to an input image using queries with supplied positional embeddings. It's useful for tasks like object detection, image segmentation, and point cloud processing. Attributes: depth (int): Number of layers in the transformer. embedding_dim (int): Channel dimension for input embeddings. num_heads (int): Number of heads for multihead attention. mlp_dim (int): Internal channel dimension for the MLP block. layers (nn.ModuleList): List of TwoWayAttentionBlock layers composing the transformer. final_attn_token_to_image (Attention): Final attention layer from queries to image. norm_final_attn (nn.LayerNorm): Layer normalization applied to final queries. Methods: forward: Process image and point embeddings through the transformer. Examples: >>> transformer = TwoWayTransformer(depth=6, embedding_dim=256, num_heads=8, mlp_dim=2048) >>> image_embedding = torch.randn(1, 256, 32, 32) >>> image_pe = torch.randn(1, 256, 32, 32) >>> point_embedding = torch.randn(1, 100, 256) >>> output_queries, output_image = transformer(image_embedding, image_pe, point_embedding) >>> print(output_queries.shape, output_image.shape) c R>[TU]5 XlX lX0lX@l[ R"5Ul[U5H-nURR[UUUUUUS:HS95 M/ [X#US9Ul [ R"U5Ulg)a# Initialize a Two-Way Transformer for simultaneous attention to image and query points. Args: depth (int): Number of layers in the transformer. embedding_dim (int): Channel dimension for input embeddings. num_heads (int): Number of heads for multihead attention. Must divide embedding_dim. mlp_dim (int): Internal channel dimension for the MLP block. activation (Type[nn.Module], optional): Activation function to use in the MLP block. attention_downsample_rate (int, optional): Downsampling rate for attention mechanism. r) embedding_dim num_headsmlp_dim activationattention_downsample_rateskip_first_layer_pedownsample_rateN)super__init__depthr r rr ModuleListlayersrangeappendTwoWayAttentionBlock Attentionfinal_attn_token_to_image LayerNormnorm_final_attn) selfrr r rrri __class__s d/home/james-whalen/.local/lib/python3.13/site-packages/ultralytics/models/sam/modules/transformer.pyrTwoWayTransformer.__init__*s(  *" mmo uA KK  $"/'#).G)*a  *3=]v)w&!||M:c.URS5RSSS5nURS5RSSS5nUnUnURHnU"UUUUS9upEM XC-nXR-nURXxUS9n XI-nUR U5nXE4$)aD Process image and point embeddings through the Two-Way Transformer. Args: image_embedding (torch.Tensor): Image to attend to, with shape (B, embedding_dim, H, W). image_pe (torch.Tensor): Positional encoding to add to the image, with same shape as image_embedding. point_embedding (torch.Tensor): Embedding to add to query points, with shape (B, N_points, embedding_dim). Returns: queries (torch.Tensor): Processed point embeddings with shape (B, N_points, embedding_dim). keys (torch.Tensor): Processed image embeddings with shape (B, H*W, embedding_dim). r r)querieskeysquery_pekey_peqkv)flattenpermuterrr) r image_embeddingimage_pepoint_embeddingr(r)layerr-r.attn_outs r#forwardTwoWayTransformer.forwardTs&*11!4<>> embedding_dim, num_heads = 256, 8 >>> block = TwoWayAttentionBlock(embedding_dim, num_heads) >>> queries = torch.randn(1, 100, embedding_dim) >>> keys = torch.randn(1, 1000, embedding_dim) >>> query_pe = torch.randn(1, 100, embedding_dim) >>> key_pe = torch.randn(1, 1000, embedding_dim) >>> processed_queries, processed_keys = block(queries, keys, query_pe, key_pe) ir Fc>[TU]5 [X5Ul[R "U5Ul[XUS9Ul[R "U5Ul[XU5Ul [R "U5Ul [R "U5Ul [XUS9Ul X`lg)a0 Initialize a TwoWayAttentionBlock for simultaneous attention to image and query points. This block implements a specialized transformer layer with four main components: self-attention on sparse inputs, cross-attention of sparse inputs to dense inputs, MLP block on sparse inputs, and cross-attention of dense inputs to sparse inputs. Args: embedding_dim (int): Channel dimension of the embeddings. num_heads (int): Number of attention heads in the attention layers. mlp_dim (int, optional): Hidden dimension of the MLP block. activation (Type[nn.Module], optional): Activation function for the MLP block. attention_downsample_rate (int, optional): Downsampling rate for the attention mechanism. skip_first_layer_pe (bool, optional): Whether to skip positional encoding in the first layer. rN)rrr self_attnrrnorm1cross_attn_token_to_imagenorm2rmlpnorm3norm4cross_attn_image_to_tokenr)r r r rrrrr"s r#rTwoWayAttentionBlock.__init__s0 "=<\\-0 )2=]v)w&\\-0 MJ?\\-0 \\-0 )2=]v)w r%cUR(aURXUS9nOX-nURXUUS9nX-nURU5nX-nX$-nURXWUS9nX-nUR U5nUR U5nX-nUR U5nX-nX$-nURXuUS9nX&-nURU5nX4$)ar Apply two-way attention to process query and key embeddings in a transformer block. Args: queries (torch.Tensor): Query embeddings with shape (B, N_queries, embedding_dim). keys (torch.Tensor): Key embeddings with shape (B, N_keys, embedding_dim). query_pe (torch.Tensor): Positional encodings for queries with same shape as queries. key_pe (torch.Tensor): Positional encodings for keys with same shape as keys. Returns: queries (torch.Tensor): Processed query embeddings with shape (B, N_queries, embedding_dim). keys (torch.Tensor): Processed key embeddings with shape (B, N_keys, embedding_dim). r,) rrLrMrNrOrPrQrSrR) r r(r)r*r+r-r6r.mlp_outs r#r7TwoWayAttentionBlock.forwards"  # #nnwWnEG"A~~'~:H(G**W%   M11Ad1C$**W%((7##**W%   M11Ag1Fzz$}r%) rSrNrPrMrOrQrRrLr)r r9r r9rr9rr:rr9rboolr;r<) r(r=r)r=r*r=r+r=r;r>r?rHs@r#rrsF&(gg)*$)%7%7%7 %7 $ %7 $' %7"%7 %7%7N,#,+7,CO,Ye, *,,r%rc|^\rSrSrSrSS U4Sjjjr\S Sj5r\S Sj5rS Sjr Sr U=r $) ra An attention layer with downscaling capability for embedding size after projection. This class implements a multi-head attention mechanism with the option to downsample the internal dimension of queries, keys, and values. Attributes: embedding_dim (int): Dimensionality of input embeddings. kv_in_dim (int): Dimensionality of key and value inputs. internal_dim (int): Internal dimension after downsampling. num_heads (int): Number of attention heads. q_proj (nn.Linear): Linear projection for queries. k_proj (nn.Linear): Linear projection for keys. v_proj (nn.Linear): Linear projection for values. out_proj (nn.Linear): Linear projection for output. Methods: _separate_heads: Separate input tensor into attention heads. _recombine_heads: Recombine separated attention heads. forward: Compute attention output for given query, key, and value tensors. Examples: >>> attn = Attention(embedding_dim=256, num_heads=8, downsample_rate=2) >>> q = torch.randn(1, 100, 256) >>> k = v = torch.randn(1, 50, 256) >>> output = attn(q, k, v) >>> print(output.shape) torch.Size([1, 100, 256]) c>[TU]5 XlUbUOUUlX-UlX lURU-S:XdS5e[ R"XR5Ul[ R"URUR5Ul [ R"URUR5Ul [ R"URU5Ul g)a Initialize the Attention module with specified dimensions and settings. Args: embedding_dim (int): Dimensionality of input embeddings. num_heads (int): Number of attention heads. downsample_rate (int, optional): Factor by which internal dimensions are downsampled. kv_in_dim (int | None, optional): Dimensionality of key and value inputs. If None, uses embedding_dim. Raises: AssertionError: If num_heads does not evenly divide the internal dim (embedding_dim / downsample_rate). Nrz$num_heads must divide embedding_dim.) rrr kv_in_dim internal_dimr rLinearq_projk_projv_projout_proj)r r r rr\r"s r#rAttention.__init__s& *&/&;)<"  9,1Y3YY1ii /@/@A ii0A0AB ii0A0AB  $"3"3]C r%cnURup#nURX#XU-5nURSS5$)zGSeparate the input tensor into the specified number of attention heads.r'r )shapereshape transpose)xr bncs r#_separate_headsAttention._separate_heads5s6''a IIaII~ 6{{1a  r%clURupp4URSS5nURXX$-5$)z9Recombine separated attention heads into a single tensor.r'r )rergrf)rhrin_headsn_tokens c_per_heads r#_recombine_headsAttention._recombine_heads<s6,-77(H KK1 yyg&:;;r%cURU5nURU5nURU5nURXR5nURX R5nURX0R5nUR u pEXR SSSS5-nU[R"U5- n[R"USS9nXc-nURU5nURU5$)a Apply multi-head attention to query, key, and value tensors with optional downsampling. Args: q (torch.Tensor): Query tensor with shape (B, N_q, embedding_dim). k (torch.Tensor): Key tensor with shape (B, N_k, embedding_dim). v (torch.Tensor): Value tensor with shape (B, N_k, embedding_dim). Returns: (torch.Tensor): Output tensor after attention with shape (B, N_q, embedding_dim). rr'r )dim) r_r`rarlr rer1mathsqrttorchsoftmaxrrrb)r r-r.r/_rqattnouts r#r7Attention.forwardCs KKN KKN KKN  NN 3  NN 3  NN 3 gg1a99Q1a((dii ++}}Tr*h##C(}}S!!r%)r r]r`r\r rbr_ra)r'N) r r9r r9rr9r\r9r;r<)rhr=r r9r;r=)rhrr;r)r-r=r.r=r/r=r;r=) r@rArBrCrDr staticmethodrlrrr7rFrGrHs@r#rrsD ! DDD D  D  DD>!! << ""r%r) __future__rrxrzrrultralytics.nn.modulesrModulerrrr%r#rsN# +q qhs299slk" k"r%