bCiPHSrSSKJr SSKJr \R "\5r"SS\5r"SS\5r "SS \5r "S S \5r "S S \5r /SQr g)zBlt model configuration)PretrainedConfig)loggingcT^\rSrSrSrSrSU4SjjrSrU=r$)BltLocalEncoderConfigz: Configuration class for the Blt Local Encoder component. blt_local_encoderc l>XlX lX0lX@lXPlX`lU=(d UUlXV-UlU=(d [SU-S- 5Ul Xl Xl Xl Xl XlXlXlUUlUR%SS5 [&TU]P"S0UDSS0D6 gNrtie_word_embeddingsF vocab_sizecross_attn_all_layers cross_attn_khidden_size_global hidden_sizenum_attention_headsnum_key_value_headshead_dimintintermediate_sizenum_hidden_layers rms_norm_epsdropoutmax_position_embeddings rope_theta rope_scaling hidden_actinitializer_rangepopsuper__init__selfrrrrrrrrrrrrrrrr kwargs __class__s c/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/blt/configuration_blt.pyr#BltLocalEncoderConfig.__init__(%%:"("4&#6 #6#M:M #: !2!Nc!k/A:M6N!2( '>$$($!2  ($/ =6=u=rrrrrrrr rrrrrrrrr)FNh㈵>`ANsilu {Gz? __name__ __module__ __qualname____firstlineno____doc__ model_typer#__static_attributes__ __classcell__r's@r(rrsK%J#  %#(>(>r+rcT^\rSrSrSrSrSU4SjjrSrU=r$)BltLocalDecoderConfigJz: Configuration class for the Blt Local Decoder component. blt_local_decoderc l>XlX lX0lX@lXPlX`lU=(d UUlXV-UlU=(d [SU-S- 5Ul Xl Xl Xl Xl XlXlXlUUlUR%SS5 [&TU]P"S0UDSS0D6 gr rr$s r(r#BltLocalDecoderConfig.__init__Qr*r+r,)r-Tr.r/r0r1N r3r4r5r6Nr7r8r9r:rCs@r(rErEJsK%J"  %#(>(>r+rEcL^\rSrSrSrSrSU4SjjrSrU=r$)BltGlobalTransformerConfig|z? Configuration class for the Blt Global Transformer component. blt_global_transformerc  :>XlX lU=(d UUlX-UlU =(d [ SU-S- 5UlX@lXPlX`lXpl Xl Xl Xl Xl U RSS5 [TU]@"S0U DSS0D6 gr )rrrrrrrrrrrrrr r!r"r#)r%rrrrrrrrrrrr r&r's r(r##BltGlobalTransformerConfig.__init__s '#6 #6#M:M #: !2!Nc!k/A:M6N!2( '>$$($!2  ($/ =6=u=r+) rrrrr rrrrrrrr) r/r1Nr3r4r6Nr7ir9r:rCs@r(rLrL|s?*J  $ > >r+rLcL^\rSrSrSrSrSU4SjjrSrU=r$)BltPatcherConfiga7 Configuration class for the Blt Patcher/Entropy model component. Args: vocab_size (`int`, *optional*, defaults to 260): Vocabulary size of the Blt patcher model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling the patcher model. hidden_size (`int`, *optional*, defaults to 768): Dimension of the hidden representations. num_hidden_layers (`int`, *optional*, defaults to 14): Number of hidden layers in the Transformer decoder. num_attention_heads (`int`, *optional*, defaults to 12): Number of attention heads for each attention layer in the Transformer decoder. num_key_value_heads (`int`, *optional*): This is the number of key_value heads that should be used to implement Grouped Query Attention. If `num_key_value_heads=num_attention_heads`, the model will use Multi Head Attention (MHA), if `num_key_value_heads=1` the model will use Multi Query Attention (MQA) otherwise GQA is used. When converting a multi-head checkpoint to a GQA checkpoint, each group key and value head should be constructed by meanpooling all the original heads within that group. For more details, check out [this paper](https://huggingface.co/papers/2305.13245). If it is not specified, will default to `num_attention_heads`. max_position_embeddings (`int`, *optional*, defaults to 8192): The maximum sequence length that this model might ever be used with. rms_norm_eps (`float`, *optional*, defaults to 1e-05): The epsilon used by the rms normalization layers. dropout (`float`, *optional*, defaults to 0.0): The dropout ratio for the attention probabilities. rope_theta (`float`, *optional*, defaults to 10000.0): The base period of the RoPE embeddings. intermediate_size (`int`, *optional*, defaults to 2048): Dimension of the MLP representations. rope_scaling (`dict`, *optional*): Dictionary containing the RoPE scaling configuration. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. blt_patcherc  T>XlX lX0lX@lX$-UlUbUOUUlX`lXplXlXl SUl U =(d [SUR-S- 5Ul Xl XlU RSS5 [ TU]D"S0U DSS0D6 g)Nr7r rr Fr )rrrrrrrrrrrrrrr r!r"r#)r%rrrrrrrrrrrr r&r's r(r#BltPatcherConfig.__init__s %&!2#6 #: :M:Y#6_r '>$( $ !2!Sc!d>N>N:NQR:R6S(!2  ($/ =6=u=r+)rrrrr rrrrrrrrr) r-i Ni r3r4g@r/Nr9r:rCs@r(rTrTs@#JJ $!>!>r+rTcp^\rSrSrSrSrS/r\\\ \ S.r SU4Sjjr Sr U=r$) BltConfiga This is the configuration class to store the configuration of a [`BltModel`]. It is used to instantiate a Blt model according to the specified arguments, defining the model architecture. Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the documentation from [`PretrainedConfig`] for more information. Args: vocab_size (`int`, *optional*, defaults to 260): Vocabulary size of the Blt model. Defines the number of different tokens that can be represented by the `inputs_ids` passed when calling [`BltModel`]. max_position_embeddings (`int`, *optional*, defaults to 4096): The maximum sequence length that this model might ever be used with. patch_in_forward (`bool`, *optional*, defaults to `True`): Whether to perform patching during the forward pass. patch_size (`int`, *optional*, defaults to 4): Size of the patches used in the patching mechanism. patching_mode (`str`, *optional*, defaults to `"entropy"`): The mode used for patching, such as entropy-based patching. patching_threshold (`float`, *optional*, defaults to 1.34): Threshold value used for determining when to apply patches. patching_batch_size (`int`, *optional*, defaults to 1): Batch size used during the patching process. max_patch_length (`int`, *optional*): Maximum length of patches that can be generated. cross_attn_k (`int`, *optional*, defaults to 2): Number of cross-attention heads used in the model. encoder_hash_byte_group_size (`list`, *optional*): List of byte group sizes used in the encoder hash function. encoder_hash_byte_group_vocab (`int`, *optional*, defaults to 500002): Vocabulary size for the encoder hash byte groups. encoder_hash_byte_group_nb_functions (`int`, *optional*, defaults to 1): Number of hash functions used in the encoder byte grouping. patcher_config (`BltPatcherConfig`, *optional*): Configuration for the patcher component of the model. encoder_config (`BltLocalEncoderConfig`, *optional*): Configuration for the local encoder component of the model. decoder_config (`BltLocalDecoderConfig`, *optional*): Configuration for the local decoder component of the model. global_config (`BltGlobalTransformerConfig`, *optional*): Configuration for the global transformer component of the model. tie_word_embeddings (`bool`, *optional*, defaults to `False`): Whether to tie weight embeddings. initializer_range (`float`, *optional*, defaults to 0.02): The standard deviation of the truncated_normal_initializer for initializing all weight matrices. rope_theta (`float`, *optional*, defaults to 500000.0): The base period of the RoPE embeddings. rope_scaling (`dict`, *optional*): Dictionary containing the RoPE scaling configuration. ```python >>> from transformers import BltModel, BltConfig >>> # Initializing a Blt configuration >>> configuration = BltConfig() >>> # Initializing a model from the configuration >>> model = BltModel(configuration) >>> # Accessing the model configuration >>> configuration = model.config ``` Checkpoint: [facebook/blt](https://huggingface.co/facebook/blt) bltpast_key_values)patcher_configencoder_configdecoder_config global_configc H>XlX lUUlUUlUUlX0lX@lXPlX`lXpl Xl URSS5Ul URSS5Ul URS5UlURSS5UlXlU =(d /SQUlXlXlU c$[)US 9Ul[,R/S 5 OS[1U [25(a#U R5S U5 [)S0U D6UlO [(5(aXlUc$[7US 9Ul[,R/S 5 OS[1U[25(a#UR5S U5 [7S0UD6UlO[65(aXlUc$[;US 9Ul[,R/S 5 OS[1U[25(a#UR5S U5 [;S0UD6UlO[:5(aXlUc$[?US 9Ul [,R/S5 OT[1U[25(a#UR5S U5 [?S0UD6Ul O[1U[>5(aUUl UR8RBUR -nUUR@RB:waUOSUR@l"URGSS5 [HTU]"SSU0UD6 g)Npatching_devicecudarealtime_patchingTpatching_threshold_add monotonicityF)rr )r z8patcher_config is None, using default Blt patcher configr z8encoder_config is None, using default Blt encoder configz8decoder_config is None, using default Blt decoder configz6global_config is None, using default Blt global configr r )&rrr rrpatch_in_forward patch_size patching_modepatching_thresholdpatching_batch_sizemax_patch_lengthgetrergrhrirencoder_hash_byte_group_sizeencoder_hash_byte_group_vocab$encoder_hash_byte_group_nb_functionsrTr`loggerinfo isinstancedict setdefaultrrarErbrLrcrencoder_cross_output_sizer!r"r#)r%rrrnrorprqrrrsrrurvrwr`rarbrcr r rrr&r}r's r(r#BltConfig.__init__>s2%'>$!2$(!1$*"4#6 0%zz*;VD!',?!F&,jj1I&J#"JJ~u=)-I,^L^)-J*4X1  !"2EV"WD  KKR S  - -  % %&9;L M"2"D^"DD  (8 9 9"0   !"7J["\D  KKR S  - -  % %&9;L M"7"I."ID  (= > >"0   !"7J["\D  KKR S  - -  % %&9;L M"7"I."ID  (= > >"0   !;N_!`D  KKP Q  t , ,  $ $%8:K L!;!Lm!LD   'A B B!.D %)$7$7$C$CdFWFW$W!)BdFXFXFdFd)d %jn 4  ($/ K-@KFKr+)rrbrarwrurvrcr rsrrirnror`rrrerprqrhrgrrr)r-rRTrjentropyg]?r2Nr.Ni"r2NNNNFr9r6N)r;r<r=r>r?r@keys_to_ignore_at_inferencerTrrErL sub_configsr#rArBrCs@r(r\r\sy@DJ#4"5*//3 K $,%)&,-.!+`L`Lr+r\)r\rTrrErLN)r?configuration_utilsrutilsr get_loggerr;rxrrErLrTr\__all__r r+r(rsu3   H %/>,/>d/>,/>d'>!1'>TI>'I>XlL lL^ r+