C1i0SrSSKJrJrJr SSKrSSKJrJ r J r "SS\ R5r "SS\ R5r "S S \ R5r"S S \ R5r"S S\ R5r"SS\ R5r"SS\ R$5r"SS\ R(5r"SS\ R,5r"SS\ R05r"SS\ R,5r"SS\ R05rg)z0Declares specification of the Transformer model.)OptionalTupleUnionN)attention_spec common_spec model_specc/X\rSrSrSS\R R S\RRSSSSSSSSSSSSSSSS4S\ S\ S \ S \ S \R S \ S \RS\ S\ S\ S\ S\ S\ S\ \ S\ \ S\ \ S\ S\ \ RS\S\S\ \ S\ \ S\ 4.SjjrSrg) TransformerEncoderSpec TFN' num_layers num_headspre_norm no_final_norm activationnum_source_embeddingsembeddings_mergelayernorm_embeddingrelative_positionrelative_attention_biasffn_glurms_normmulti_query_attention num_heads_kvhead_dim rotary_dimrotary_interleaverotary_scaling_typerotary_scaling_factor rotary_basesliding_windowqk_normpre_post_layer_normcfU (aUbUS:wa [S5eSnXl[R"S5R U5UlX0l[R"S5R U5Ul[R"S5R U5Ul[U5Vs/sHn[R"5PM snUl SUl U (dU (d[5UlU(a U(d[R "U S9UlU(a[R "U S9UlUb*[R"S5R U5Ul[U5Vs/sHn[)U U U U UUUUUUUUUS 9 PM snUlgs snfs snf) a/Initializes a Transformer encoder specification. Args: num_layers: Number of layers. num_heads: Number of attention heads. pre_norm: Enable the pre-norm Transformer architecture. no_final_norm: Disable the final layer norm in the pre-norm architecture. activation: Activation to apply in the feed-forward network. num_source_embeddings: Number of source embeddings. embeddings_merge: When :obj:`num_source_embeddings` > 1, specify how the embeddings are merged. layernorm_embedding: Apply layer normalization after the embedding layer. relative_position: Use relative position representations in the self-attention layers as described in https://arxiv.org/abs/1803.02155. relative_attention_bias: Use relative attention bias in the self-attention layers as described in the T5 paper https://arxiv.org/abs/1910.10683. ffn_glu: Use gated linear units in the FFN layers as described in https://arxiv.org/abs/2002.05202. rms_norm: Use the root mean square layer normalization. multi_query_attention: Use multi-query attention (alias for num_heads_kv=1). num_heads_kv: Number of attention heads for the key and value. head_dim: Number of dimensions per attention head. rotary_dim: Apply rotary embeddings to these first N dimensions. If 0, rotary embeddings are applied to all dimensions. rotary_interleave: Interleave the head dimensions when rotary embeddings are applied. Otherwise the head dimensions are sliced in half. rotary_scaling_type: Type of RoPE scaling. rotary_scaling_factor: Factor used in the RoPE scaling. rotary_base: The base period of the rotary embeddings. sliding_window: Max sequence length to retain in KV Cache. qk_norm: Apply layer normalization to the query and key projections. pre_post_layer_norm: Add post layer norm for each pre norm layer. Nr 5Enabling multi_query_attention implies num_heads_kv=1int16int8Trint32) rrrrrrrrrr r!r#r$) ValueErrorrnpdtypetyperrrrrangerEmbeddingsSpec embeddingsscale_embeddingsPositionEncoderSpecposition_encodings LayerNormSpec layer_normrr"TransformerEncoderLayerSpeclayer)selfrrrrrrrrrrrrrrrrrrr r!r"r#r$_s \/home/james-whalen/.local/lib/python3.13/site-packages/ctranslate2/specs/transformer_spec.py__init__TransformerEncoderSpec.__init__ sx !'LA,= KL%:"'*// : ((6*// ; " 0 5 56F G278M2N 2NQK & & (2N !% )@&9&;D # M)77JDO '2'@'@('SD $  %"$((7"3"8"8"HD $:&! ' ("3(?!)!%"3$7&;'$7 '!    s 5F)F.) rr1rr8r6rrrr4rr2r")__name__ __module__ __qualname____firstlineno__r ActivationRELUEmbeddingsMergeCONCATintboolrrRotaryScalingTypefloatr<__static_attributes__r;r r s #-8-C-C-H-H%&8C8S8S8Z8Z$)"'(-&+&*"&$("&JN'("(,"'$)1f f f  f  f  ** f  #f &55f "f  f "&f f f  $f smf 3-!f "SM#f $ %f &&n&F&FG'f ( %)f *+f ,! -f .$/f 0"1f f rLr cJ\rSrSrS\R R SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS4"S \S \S \S \R S \S\S\S\S\S\S\S\S\S\S\S\S\S\ \S\S\ \ RS\ S\ S\S \S!\S"\S#\S$\S%\ \S&\ \S'\ \S(\ \RS)\ \S*\ \S+\S,\ \4HS-jjr\S.5rS/rg)0TransformerDecoderSpectTFr Nr rrrrrrwith_encoder_attentionrproject_in_outrralignment_layeralignment_headsrralibialibi_use_positive_positions scale_alibirrrr r! original_max_position_embeddingsmax_position_embeddingsparallel_residualshared_layer_normr$rrrr" quant_typequant_group_size quant_bitsr# external_pre_post_encoder_layersc% [5UlU(a$U(d [S5eU(a [S5eU(aUbUS:wa [S5eSn[R"S5R U5UlX0l[R"S5R U5Ul[R"S5R U 5Ul [R"S5R U 5Ul [R"5Ul SUl[R UlXlUUlUUlUb*[R"S 5R U5UlU (d U (dU(dUc[-5UlU(a U(d[R0"US 9UlU(a[R0"US 9Ul[R6"5Ul[;U5V%s/sHHn%[=S$0S U_S U _S U _SU _SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU#_SU$_6PMJ sn%UlSUl U=(d UU:gURS 'U(a4[R6"5Ul![R6"5Ul"U (a.U URS!'U"URS"'U!URS#'ggs sn%f)%a Initializes a Transformer decoder specification. Args: num_layers: Number of layers. num_heads: Number of attention heads. pre_norm: Enable the pre-norm Transformer architecture. activation: Activation to apply in the feed-forward network. layernorm_embedding: Apply layer normalization after the embedding layer. with_encoder_attention: Enable the encoder attention sublayers. no_final_norm: Disable the final layer norm in the pre-norm architecture. project_in_out: Add linear transformations after the embedding layer and before the final layer. relative_position: Use relative position representations in the self-attention layers as described in https://arxiv.org/abs/1803.02155. relative_attention_bias: Use relative attention bias in the self-attention layers as described in the T5 paper https://arxiv.org/abs/1910.10683. alignment_layer: Layer index selected for alignment. alignment_heads: Number of attention heads selected for alignment. ffn_glu: Use gated linear units in the FFN layers as described in https://arxiv.org/abs/2002.05202. rms_norm: Use the root mean square layer normalization. alibi: Use attention with linear biases. alibi_use_positive_positions: Use positive positions in the ALiBi definition. scale_alibi: Apply the dot product scale factor to ALiBi. rotary_dim: Apply rotary embeddings to these first N dimensions. If 0, rotary embeddings are applied to all dimensions. rotary_interleave: Interleave the head dimensions when rotary embeddings are applied. Otherwise the head dimensions are sliced in half. rotary_scaling_type: Type of RoPE scaling. rotary_scaling_factor: Factor used in the RoPE scaling. rotary_base: The base period of the rotary embeddings. original_max_position_embeddings: The original max position embeddings for Su rope embeddings max_position_embeddings: The max position embeddings for Su rope embeddings parallel_residual: Use parallel residual connections in each layer block, as used by the GPT-J and GPT-NeoX models. shared_layer_norm: When using parallel residual, share the input and post attention layer norms. pre_post_layer_norm: Add post layer norm for each pre norm layer multi_query_attention: Use multi-query attention (alias for num_heads_kv=1). num_heads_kv: Number of attention heads for the key and value. sliding_window: Max sequence length to retain in KV Cache. quant_type: quantization type used (like awq... for lower bit quantization) quant_group_size: group size of the lower bit quantization quant_bits: number of bit of the quantization (ex: 4bit) external_pre_post_encoder_layers: if the encoder attention pre and processing is done outside the attention. z/The GPT-J block expects a pre-norm architecturez-The GPT-J block does not have cross attentionNr r&r'r(Tr*r)rQrrrrrrrr r!rXrYrZr[r$rrr"r#r_Frquantization_typequantization_bitsquantization_group_sizerK)#dict_configr+r,r-r.rrrrSrTrr0r1r2rOPTIONAL scale_outputsrUrVrWr"r3r4r5r6r LinearSpec projectionr/TransformerDecoderLayerSpecr8start_from_zero_embedding project_in project_out)&r9rrrrrrQrrRrrrSrTrrrUrVrWrrrr r!rXrYrZr[r$rrrr"r\r]r^r#r_r:s& r;r<TransformerDecoderSpec.__init__uspv  !RSS% !PQQ 'LA,= KL'*// : ((6*// ;!xx055oF!xx055oF%446 $'00 ,H)&  %"$((7"3"8"8"HD !+"&9&;D # M)77JDO '2'@'@('SD $%0020:&/ .'- ( '= "3 )@    "   &  #4 %8 '< ( 2R )@ #4 #4 %8  *! ""# $ .% & ' (2R) ,'/  2*/&0E1 I % ,- )446DO*557D  0:DLL, -0:DLL, -6FDLL2 3 E sAK<cUR$N)rer9s r;configTransformerDecoderSpec.configs ||rL)rerrUrVrTrSr1r8r6rrr4rrlrmrirWr2rgr"rk)r>r?r@rArrBrCrFrGrrrHrI Quantizationr<propertyrrrJrKrLr;rNrNts -8-C-C-H-H$)'+#$"'(-! -2!$("&JN'("01'("'"'$)&+&*"&(,9=*.$(;@KdGdGdG dG  ** dG " dG!%dGdGdG dG"&dGdGdGdGdG !dG"'+#dG$%dG&SM'dG( )dG*&n&F&FG+dG, %-dG./dG0+.1dG2"%3dG4 5dG6 7dG8"9dG: $;dG<sm=dG>3-?dG@! AdGB[556CdGD#3-EdGFSMGdGHIdGJ+34.KdGLrLrNc v\rSrSrS S\\S\S\\RS\ S\ S\4 S jjr S r g) r7i Nrrrr r!r$c[R"SUUUUUUUU U U U U S9 Ul[X4S9UlU(a[ R "US9Ul[ R "US9Ul[ R "US9Ul [ R "US9Ul [URS5 [URS5 gg)NT) self_attentionrrrrrr"rrrr r!r#glurr)r6) rMultiHeadAttentionSpecrxFeedForwardSpecffnrr5input_layer_normpost_attention_layer_normpre_feedforward_layer_normpost_feedforward_layer_normdelattr)r9rrrrrrr"rrrr r!r#r$s r;r<$TransformerEncoderLayerSpec.__init__!s"-CC/$;%)!/ 3"7# #wB $/$=$=x$PD !-8-F-F!.D */:.G.G!/D +0;/H/H!0D , D'' 6 DHHl + rL)r}r~rrrrx)FFFFNNNNTNr r FF) r>r?r@rArrFrGrrHrIr<rJrKrLr;r7r7 s %$("&JN'("$)/,SM/, /,&n&F&FG/, %/,/,"/,/,rLr7cF\rSrSrSSjrSrg)rjiSNc~[R"SUUUUUUU U U U UUUUS9UlU(a [R"UUUUUUSLS9Ul[ XES9UlU (aU(a[ R"5UlO4[ R"5Ul [ R"5Ul [URS5 [UR S5 U(a[ R"US9Ul [ R"US9Ul U(a9U(a2[ R"US9Ul [ R"US9Ul [ R"US9Ul[ R"US9Ul[URS5 [UR S5 gg)NT)rxrrrrrrr r!rXrYrrr"r#F)rrrr"r#has_normryr6r))rr{rx attentionr|r}rr5r[r~rr*external_post_encoder_attention_layer_norm)external_pre_encoder_attention_layer_normrr)r9rQrrrrrrrr r!rXrYrZr[r$rrr"r#r_s r;r<$TransformerDecoderLayerSpec.__init__Ts.-CC/$;!/ 3"7#-M$;%) $ "+BB!)!-9UB DN#wB  )4)B)B)D&(3(A(A(C%1<1J1J1L. D'' 6 DHHl + $/$=$=x$PD !-8-F-F!.D *&*J--x@? --x@> /:.G.G!/D +0;/H/H!0D , D'' 6 DHHl +1 rL) rrrr}r~rrrrxr[)TFFFFNTNr r rrFFFNNNFFr>r?r@rAr<rJrKrLr;rjrjSsF $ % )* !!).+W,rLrjc\rSrSrSSjrSrg)r|ic[R"US9Ul[R"5Ul[R"5UlU(a"5Ulgg)Nr))rr5r6rhlinear_0linear_1linear_0_noact)r9rzrs r;r<FeedForwardSpec.__init__sM%33XF#..0 #..0 "-"8"8":D  rL)r6rrrN)FFrrKrLr;r|r|s;rLr|c\rSrSrSrSrg)r3ic.[RUlgrp)rrf encodingsrqs r;r<PositionEncoderSpec.__init__s#,,rL)rNrrKrLr;r3r3s-rLr3c>^\rSrSrSrSS\\4U4SjjjrSrU=r $)TransformerConfigiz%Configuration for Transformer models.layer_norm_epsilonc *>[TU]"SSU0UD6 g)zInitializes the configuration for Transformer models. Args: layer_norm_epsilon: The layer norm epsilon value. **kwargs: Additional configuration. rNrKsuperr<r9rkwargs __class__s r;r<TransformerConfig.__init__ I,>I&IrLrKrp r>r?r@rA__doc__rrIr<rJ __classcell__rs@r;rrs/J8E?JJrLrc d^\rSrSrSrS\S\4U4Sjjr\SSS\ RRSS S \ RRSSSSS4 S \\\\\44S \S \S \S\S\ RS\S\S\S\ RS\S\S\S\S\4Sjj5r\S5r\S5rSrSrSrSrU=r$) TransformerSpecizDescribes a Transformer model. The specification is invariant to hidden dimensions but requires to explicitly set the number of layers and attention heads. encoderdecoderc>[U[5(d [S5e[U[5(d [S5e[TU]5 XlX lURRSUR R5 g)zInitializes a Transformer model specification. Args: encoder: The encoder specification. decoder: The decoder specification. 1encoder argument must be a TransformerEncoderSpec1decoder argument must be a TransformerDecoderSpecrN) isinstancer TypeErrorrNrr<rrre add_attributer)r9rrrs r;r<TransformerSpec.__init__so'#9::OP P'#9::OP P    "" #T\\%G%G rLFTrPr rrwith_relative_positionrrrrSrTrrrrrrrc[U[[45(aUunnOXnn[UUUUUU U U UU U UUS9 n[ UUUUUU UU UUU UUS9 nU"UU5$)aCreates a Transformer model specification. Args: num_layers: Number of encoder and decoder layers, or a 2-tuple if the number is different. num_heads: Number of attention heads. with_relative_position: Use relative position representations in the self-attention layers as described in https://arxiv.org/abs/1803.02155. pre_norm: Enable the pre-norm Transformer architecture. no_final_norm: Disable the final layer norm in the pre-norm architecture. activation: Activation to apply in the feed-forward network. alignment_layer: Layer index selected for alignment. alignment_heads: Number of attention heads selected for alignment. num_source_embeddings: Number of source embeddings. embeddings_merge: When :obj:`num_source_embeddings` > 1, specify how the embeddings are merged. layernorm_embedding: Apply layer normalization after the embedding layer. relative_attention_bias: Use relative attention bias in the self-attention layers as described in the T5 paper https://arxiv.org/abs/1910.10683. ffn_glu: Use gated linear units in the FFN layer as described in https://arxiv.org/abs/2002.05202. rms_norm: Use the root mean square layer normalization. multi_query_attention: Use multi-query attention. ) rrrrrrrrrrr) rrrrrrrSrTrrr)rlisttupler rN)clsrrrrrrrSrTrrrrrrrnum_encoder_layersnum_decoder_layersrrs r; from_configTransformerSpec.from_configsV j4- 0 05? 2  25? 2 (  '!"7- 34$;"7  )  '! 34$;++"7  7G$$rLcg)NrrKrqs r;nameTransformerSpec.name7s rLcg)NrKrqs r;revisionTransformerSpec.revision;rLc[5$rp)rrqs r;get_default_config"TransformerSpec.get_default_config?s  ""rLcURRVs/sHoRRSPM sn$s snfNrrr1weightshape)r9specs r;get_source_vocabulary_size*TransformerSpec.get_source_vocabulary_sizeBs3151H1HI1H !!!$1HIIIs">c\URRRRS$rrr1rrrqs r;get_target_vocabulary_size*TransformerSpec.get_target_vocabulary_sizeE#||&&--33A66rL)rr)r>r?r@rArr rNr< classmethodrrBrCrDrErrFrrGrrurrrrrrJrrs@r;rrss  - 8N * (-#-8-C-C-H-H! %&8C8S8S8Z8Z$)(-&+!O%#uS#X./O%O%!% O%  O%  O% **O%O%O% #O%&55O%"O%"&O%O%O% $!O%O%b!!#J77rLrc>^\rSrSrSrSS\\4U4SjjjrSrU=r $)TransformerDecoderModelConfigiIz-Configuration for Transformer decoder models.rc *>[TU]"SSU0UD6 g)zInitializes the configuration for Transformer decoder models. Args: layer_norm_epsilon: The layer norm epsilon value. **kwargs: Additional configuration. rNrKrrs r;r<&TransformerDecoderModelConfig.__init__LrrLrKrprrs@r;rrI7J8E?JJrLrc@^\rSrSrSrS\4U4Sjjr\S\RRSSSSSSSSSSSSSS S S SSSSSSSSSSS4S \ S \ S \ S\RS\ S\ S\ S\ S\ S\ S\ S\ S\ S\ \ S\ S\ \RS\S\S\ S\ S\ S \ S!\ S"\ S#\ \ S$\ \ S%\ \ S&\ \R"S'\ \ S(\ \ S)\ 4>S*jj5r\S+5r\S,5rS-rS.rS/rU=r$)0TransformerDecoderModelSpeciVz3Describes a Transformer decoder model (e.g. GPT-2).rc>[U[5(d [S5e[TU]5 XlUR R R5H up#URRX#5 M" g)zdInitializes a Transformer decoder model specification. Args: decoder: The decoder specification. rN) rrNrrr<rrritemsrer)r9rkeyvaluers r;r<$TransformerDecoderModelSpec.__init__Ys` '#9::OP P  ,,--335JC LL & &s 26rLTFNr r rrrrrrrrRrrrrUrVrWrrrr r!rXrYrZr[r$rrrr"r\r]r^r#c [UU40SU_SU_SU_SS_SU_SU_SU_S U _S U _S U _S U _S U _SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_SU_6n U"U 5$) aCreates a Transformer decoder model specification. Args: num_layers: Number of decoder layers. num_heads: Number of attention heads. pre_norm: Enable the pre-norm Transformer architecture. activation: Activation to apply in the feed-forward network. layernorm_embedding: Apply layer normalization after the embedding layer. no_final_norm: Do not apply layer normalization after the last decoder block. project_in_out: Add a linear layer after the embedding layer and another one before the final output projection. with_relative_position: Enable relative position representations modules. ffn_glu: Use gated linear units in the FFN layers as described in https://arxiv.org/abs/2002.05202. rms_norm: Use the root mean square layer normalization. alibi: Use attention with linear biases. alibi_use_positive_positions: Use positive positions in the ALiBi definition. scale_alibi: Apply the dot product scale factor to ALiBi. rotary_dim: Apply rotary embeddings to these first N dimensions. If 0, rotary embeddings are applied to all dimensions. rotary_interleave: Interleave the head dimensions when rotary embeddings are applied. Otherwise the head dimensions are sliced in half. rotary_scaling_type: Type of RoPE scaling. rotary_scaling_factor: Factor used in the RoPE scaling. rotary_base: The base period of the rotary embeddings. original_max_position_embeddings: The original max position embeddings for Su rope embeddings max_position_embeddings: The max position embeddings for Su rope embeddings parallel_residual: Use parallel residual connections in each layer block, as used by the GPT-J and GPT-NeoX models. shared_layer_norm: When using parallel residual, share the input and post attention layer norms. pre_post_layer_norm: add post layer norm for each pre norm layer multi_query_attention: Use multi-query attention (alias for num_heads_kv=1). num_heads_kv: Number of attention heads for the key and value. head_dim: Number of head sliding_window: max sequence length to retain KV cache quant_type: quantization type used (like awq... for lower bit quantization) quant_group_size: group size of the lower bit quantization quant_bits: number of bit of the quantization (ex: 4bit) rrrrQFrrRrrrrUrVrWrrrr r!rXrYrZr[r$rrrr"r\r]r^r#)rN)!rrrrrrrrRrrrrUrVrWrrrr r!rXrYrZr[r$rrrr"r\r]r^r#rs! r;r'TransformerDecoderModelSpec.from_configgsRX)  ! ! " ! !4 ! $) ! (! *! 5! ! ! ! *F! $! "! 0!! "!4#! $#8%! &$'! (.N)! *%<+! ,0-! .0/! 0!41! 2#83! 4&5! 67! 8*9! :";! <.=! >"?! @A! F7|rLcg)NrNrKrqs r;r TransformerDecoderModelSpec.name'rLcg)NrKrqs r;r$TransformerDecoderModelSpec.revisionrrLc[5$rp)rrqs r;r.TransformerDecoderModelSpec.get_default_config ,..rLc\URRRRS$rrrqs r;get_vocabulary_size/TransformerDecoderModelSpec.get_vocabulary_sizerrL)r)r>r?r@rArrNr<rrrBrCrFrGrrrHrIrtrrurrrrrJrrs@r;rrVsP= 3 6 3 -8-C-C-H-H$)#$',-2!$("&JN'("01'("'"'$)&+&*"&(,9=*.$(Annn n  ** n " nnn!%nnnn'+nnSMn  !n"&n&F&FG#n$ %%n&'n(+.)n*"%+n, -n. /n0"1n2 $3n4sm5n63-7n8! 9n:[556;n<#3-=n>SM?n@Ann`((/77rLrc>^\rSrSrSrSS\\4U4SjjjrSrU=r $)TransformerEncoderModelConfigiz-Configuration for Transformer encoder models.rc *>[TU]"SSU0UD6 g)zInitializes the configuration for Transformer encoder models. Args: layer_norm_epsilon: The layer norm epsilon value. **kwargs: Additional configuration. rNrKrrs r;r<&TransformerEncoderModelConfig.__init__rrLrKrprrs@r;rrrrLrc^\rSrSrSrS\R R4S\S\ S\R 4U4Sjjjr \ S5r \ S 5r S rS rS rU=r$) TransformerEncoderModelSpeciz2Describes a Transformer encoder model (e.g. BERT).Fr pooling_layerpooling_activationcf>[U[5(d [S5e[TU]5 XlUR RSUR R5 U(aE[R"5Ul [R"S5RU5Ulgg)zInitializes a Transformer encoder model specification. Args: encoder: The encoder specification. pooling_layer: Add the pooling layer. pooling_activation: The activation to apply after the pooling layer. rrr(N)rr rrr<rrerrrrh pooler_denser,r-r.pooler_activation)r9rrrrs r;r<$TransformerEncoderModelSpec.__init__s'#9::OP P   "" #T\\%G%G   + 6 6 8D %'XXf%5%:%:;M%ND " rLcg)Nr rKrqs r;r TransformerEncoderModelSpec.namerrLcg)Nr rKrqs r;r$TransformerEncoderModelSpec.revisionrrLc[5$rp)rrqs r;r.TransformerEncoderModelSpec.get_default_configrrLcbURRSRRS$rrrqs r;r/TransformerEncoderModelSpec.get_vocabulary_sizes(||&&q)0066q99rL)rrr)r>r?r@rArrrBTanhr rGr<rurrrrrJrrs@r;rrs< $5@5K5K5P5P O'OO(22 OO4((/::rLr)rtypingrrrnumpyr,ctranslate2.specsrrr LayerSpecr rNr7rjr|r3SequenceToSequenceModelConfigrSequenceToSequenceModelSpecrLanguageModelConfigrLanguageModelSpecrrrrKrLr;r s6))EEg Z11g TiZ11iX0,*"6"60,fX,*"6"6X,v;j**;-*..- J @@ J}7j<<}7@ JJ$B$B JN7*">">N7b JJ$B$B J):*">">):rL