C1igS)0SSKrSSKrSSKrSSKrSSKrSSKJrJrJrJ r J r J r SSK J r JrJrJrJrJr SrS6S\S\\ \S\ \\ \S\S \S \\4 S jjrS7S\S\\ \S \\ \S\S \S \\4 S jjrS8S\S\\ \S\S \S \\ 4 SjjrS7S\S\\ \S\S \S \\4 SjjrS9SSSSSSSSSSSSSS. S\S\ \S\ \ \S\S\S\S\S\S\S\S\S \S!\ \ \ \S"\ \ \\ \\ \4S#\S$\S \\4"S%jjjrS:S&SSSSSSSSSSSS'SS(.S\S)\ \ \\ \ \4S\S \S*\S+\S\S\S\S\S\S\S \S!\ \ \ \S"\ \ \\ \\ \4S,\ \ \S-\S.\\/\4S \\4&S/jjjrS:S&SSSSSSSSSSSS'SS(.S\S)\ \ \\ \ \4S\S \S*\S+\S\S\S\S\S\S\S \S!\ \ \ \S"\ \ \\ \\ \4S,\ \ \S-\S.\\/\4S \\4&S0jjjr"S1S25rS3r S4r!S5r"g);N) AsyncIterableCallableIterableListOptionalUnion)GenerationResultGenerationStepResult Generator ScoringResultTranslationResult Translatorc*[[S[5 [[S[5 [[S[5 [[ S[ 5 [[ S[5 [[ S[5 [[ S[5 g)z4Registers additional attributes to compiled modules.translate_iterablescore_iterablegenerate_tokensgenerate_iterableasync_generate_tokensN) setattrrtranslator_translate_iterabletranslator_score_iterabletranslator_generate_tokensr generator_generate_iterablegenerator_score_iterablegenerator_generate_tokensgenerator_async_generate_tokensP/home/james-whalen/.local/lib/python3.13/site-packages/ctranslate2/extensions.pyregister_extensionsr sd J,.KL J(*CD J)+EF I*,GH I')AB I(*CD I.0OPr translatorsource target_prefixmax_batch_size batch_typereturnc+# U/nUbURU5 [URUUU40UD6ShvN gN7f)aTranslates an iterable of tokenized examples. This method is built on top of :meth:`ctranslate2.Translator.translate_batch` to efficiently translate an arbitrarily large stream of data. It enables the following optimizations: * stream processing (the iterable is not fully materialized in memory) * parallel translations (if the translator has multiple workers) * asynchronous batch prefetching * local sorting by length Arguments: source: An iterable of tokenized source examples. target_prefix: An optional iterable of tokenized target prefixes. max_batch_size: The maximum batch size. batch_type: Whether :obj:`max_batch_size` is the number of "examples" or "tokens". **kwargs: Any translation options accepted by :meth:`ctranslate2.Translator.translate_batch`. Returns: A generator iterator over :class:`ctranslate2.TranslationResult` instances. Example: This method can be used to efficiently translate text files: .. code-block:: python # Replace by your own tokenization and detokenization functions. tokenize_fn = lambda line: line.strip().split() detokenize_fn = lambda tokens: " ".join(tokens) with open("input.txt") as input_file: source = map(tokenize_fn, input_file) results = translator.translate_iterable(source, max_batch_size=64) for result in results: tokens = result.hypotheses[0] target = detokenize_fn(tokens) print(target) N)append_process_iterabletranslate_batch)r!r"r#r$r%kwargs iterabless rrrsQ`I ' ""   s 4><>targetc+T# [URX/UU40UD6ShvN gN7f)a9Scores an iterable of tokenized examples. This method is built on top of :meth:`ctranslate2.Translator.score_batch` to efficiently score an arbitrarily large stream of data. It enables the following optimizations: * stream processing (the iterable is not fully materialized in memory) * parallel scoring (if the translator has multiple workers) * asynchronous batch prefetching * local sorting by length Arguments: source: An iterable of tokenized source examples. target: An iterable of tokenized target examples. max_batch_size: The maximum batch size. batch_type: Whether :obj:`max_batch_size` is the number of "examples" or "tokens". **kwargs: Any scoring options accepted by :meth:`ctranslate2.Translator.score_batch`. Returns: A generator iterator over :class:`ctranslate2.ScoringResult` instances. Nr) score_batch)r!r"r-r$r%r+s rrr[s8<!     (&( generator start_tokensc+T# [URU/UU40UD6ShvN gN7f)a#Generates from an iterable of tokenized prompts. This method is built on top of :meth:`ctranslate2.Generator.generate_batch` to efficiently run generation on an arbitrarily large stream of data. It enables the following optimizations: * stream processing (the iterable is not fully materialized in memory) * parallel generations (if the generator has multiple workers) * asynchronous batch prefetching * local sorting by length Arguments: start_tokens: An iterable of tokenized prompts. max_batch_size: The maximum batch size. batch_type: Whether :obj:`max_batch_size` is the number of "examples" or "tokens". **kwargs: Any generation options accepted by :meth:`ctranslate2.Generator.generate_batch`. Returns: A generator iterator over :class:`ctranslate2.GenerationResult` instances. N)r)generate_batch)r2r3r$r%r+s rrrs78!     r1tokensc+T# [URU/UU40UD6ShvN gN7f)aScores an iterable of tokenized examples. This method is built on top of :meth:`ctranslate2.Generator.score_batch` to efficiently score an arbitrarily large stream of data. It enables the following optimizations: * stream processing (the iterable is not fully materialized in memory) * parallel scoring (if the generator has multiple workers) * asynchronous batch prefetching * local sorting by length Arguments: tokens: An iterable of tokenized examples. max_batch_size: The maximum batch size. batch_type: Whether :obj:`max_batch_size` is the number of "examples" or "tokens". **kwargs: Any score options accepted by :meth:`ctranslate2.Generator.score_batch`. Returns: A generator iterator over :class:`ctranslate2.ScoringResult` instances. Nr/)r2r6r$r%r+s rrrs78!    r1Fi) max_decoding_lengthmin_decoding_length sampling_topk sampling_toppsampling_temperaturereturn_log_probrepetition_penaltyno_repeat_ngram_size disable_unksuppress_sequences end_tokenmax_input_lengthuse_vmapr:r;r<r=r>r?r@rArBrCrDrErFc #r# [URU/UbU/OSU U U U U UUUUUUUUS9ShvN gN7f)aYields tokens as they are generated by the model. Arguments: source: Source tokens. target_prefix: Optional target prefix tokens. max_decoding_length: Maximum prediction length. min_decoding_length: Minimum prediction length. sampling_topk: Randomly sample predictions from the top K candidates. sampling_topp: Keep the most probable tokens whose cumulative probability exceeds this value. sampling_temperature: Sampling temperature to generate more random samples. return_log_prob: Include the token log probability in the result. repetition_penalty: Penalty applied to the score of previously generated tokens (set > 1 to penalize). no_repeat_ngram_size: Prevent repetitions of ngrams with this size (set 0 to disable). disable_unk: Disable the generation of the unknown token. suppress_sequences: Disable the generation of some sequences of tokens. end_token: Stop the decoding on one of these tokens (defaults to the model EOS token). max_input_length: Truncate inputs after this many tokens (set 0 to disable). use_vmap: Use the vocabulary mapping file saved in this model Returns: A generator iterator over :class:`ctranslate2.GenerationStepResult` instances. Note: This generation method is not compatible with beam search which requires a complete decoding. N) r@rArBrCrDr:r;r<r=r> return_scoresrErF)_generate_tokensr*)r!r"r#r:r;r<r=r>r?r@rArBrCrDrErFs rrrs[\ "" (4$-1-//##1%)!s -757iT) max_length min_lengthr<r=r>r?r@rArBrCrD static_promptcache_static_promptcallbackpromptrJrKrLrMrNc## [U5S:a[US[5(aU/n[URU40SU_SU_SU _SU _SU _SU _SU_S U_S U_S U_S U_S U_SU _SU_SU_SS_SU_6ShvN gN7f)a]Yields tokens as they are generated by the model. Arguments: prompt: Batch of start tokens. If the decoder starts from a special start token like , this token should be added to this input. max_batch_size: The maximum batch size. batch_type: Whether :obj:`max_batch_size` is the number of "examples" or "tokens". max_length: Maximum generation length. min_length: Minimum generation length. sampling_topk: Randomly sample predictions from the top K candidates. sampling_topp: Keep the most probable tokens whose cumulative probability exceeds this value. sampling_temperature: Sampling temperature to generate more random samples. return_log_prob: Include the token log probability in the result. repetition_penalty: Penalty applied to the score of previously generated tokens (set > 1 to penalize). no_repeat_ngram_size: Prevent repetitions of ngrams with this size (set 0 to disable). disable_unk: Disable the generation of the unknown token. suppress_sequences: Disable the generation of some sequences of tokens. end_token: Stop the decoding on one these tokens (defaults to the model EOS token). static_prompt: If the model expects a static prompt (a.k.a. system prompt) it can be set here to simplify the inputs and optionally cache the model state for this prompt to accelerate future generations. cache_static_prompt: Cache the model state after the static prompt and reuse it for future generations using the same static prompt. callback: Optional function that is called for each generated token when obj:`beam_size` is 1. If the callback function returns ``True``, the decoding will stop for this batch index. Returns: A generator iterator over :class:`ctranslate2.GenerationStepResult` instances. Note: This generation method is not compatible with beam search which requires a complete decoding. rr$r%r@rArBrCrDrJrKr<r=r>rHrLrMinclude_prompt_in_resultFrNN)len isinstancestrrIr5)r2rOr$r%rJrKr<r=r>r?r@rArBrCrDrLrMrNs rrrsp 6{Q:fQi55  &  .  2  .$$2& $!"0#$"'%&'sA6B8A>9Bc# [U5S:a[US[5(aU/n[URU40SU_SU_SU _SU _SU _SU _SU_S U_S U_S U_S U_S U_SU _SU_SU_SS_SU_6ShvN nU7v MN g7f)avYields tokens asynchronously as they are generated by the model. Arguments: prompt: Batch of start tokens. If the decoder starts from a special start token like , this token should be added to this input. max_batch_size: The maximum batch size. batch_type: Whether :obj:`max_batch_size` is the number of "examples" or "tokens". max_length: Maximum generation length. min_length: Minimum generation length. sampling_topk: Randomly sample predictions from the top K candidates. sampling_topp: Keep the most probable tokens whose cumulative probability exceeds this value. sampling_temperature: Sampling temperature to generate more random samples. return_log_prob: Include the token log probability in the result. repetition_penalty: Penalty applied to the score of previously generated tokens (set > 1 to penalize). no_repeat_ngram_size: Prevent repetitions of ngrams with this size (set 0 to disable). disable_unk: Disable the generation of the unknown token. suppress_sequences: Disable the generation of some sequences of tokens. end_token: Stop the decoding on one of these tokens (defaults to the model EOS token). static_prompt: If the model expects a static prompt (a.k.a. system prompt) it can be set here to simplify the inputs and optionally cache the model state for this prompt to accelerate future generations. cache_static_prompt: Cache the model state after the static prompt and reuse it for future generations using the same static prompt. callback: Optional function that is called for each generated token when obj:`beam_size` is 1. If the callback function returns ``True``, the decoding will stop for this batch index. Returns: An async generator iterator over :class:`ctranslate2.GenerationStepResult` instances. Note: This generation method is not compatible with beam search which requires a complete decoding. rr$r%r@rArBrCrDrJrKr<r=r>rHrLrMrQFrNN)rRrSrTAsyncGeneratorr5)r2rOr$r%rJrKr<r=r>r?r@rArBrCrDrLrMrN step_results rrr_sp 6{Q:fQi55+  &  .  2  .$$2& $!"0#$"'%&'k*+s*A3B 5B9B:B=B BB c,\rSrSrSrSrSrSrSrg)rVic[R"5Ul[R"5UlSUlXlX lX0l gN) asyncioQueuequeue threadingEventshutdown_event iterator_task process_funcargsr+)selfrbrcr+s r__init__AsyncGenerator.__init__s6]]_ 'oo/!(  rc# [UR/URQ70URD6HenURR U5IShvN [ R"S5IShvN URR5(dMe O URR S5IShvN gNkNON 7f)Ng-C6?) rIrbrcr+r]putr[sleepr`is_set)rdrWs rproducerAsyncGenerator.producers+    $  -1[[ K**..- - ---' ' '""))++ jjnnT""" . ' #s<ACB=C/B?0!C"C7C8C?CCcX[R"UR55UlU$rZ)r[ create_taskrkra)rds r __aiter__AsyncGenerator.__aiter__s $00A rcX# URR5(a[eURR 5IShvN nUc URR 5 [eU$N)![ Ra! URR 5 [ef=f7frZ)r`rjStopAsyncIterationr]getsetr[CancelledError)rditems r __anext__AsyncGenerator.__anext__s    % % ' '$ $ %))D|##'')((K * %% %    # # %$ $ %s.&B*A2A0(A2/B*0A225B''B*)rcrar+rbr]r`N) __name__ __module__ __qualname____firstlineno__rerkrorw__static_attributes__rrrrVrVs # %rrVc/^^^ ^ # [R"5m [R"5mUR SS5m T cSm UU U 4SjnUR SSUS.5 U"U0UD6mUU 4Sjn[R "USS9nUR5 T R 5nUcO[U[5(aUeUv M2UR5 g![a TR5 M.f=f7f) NrNcg)NFr)rWs r"_generate_tokens..sErch>T"U5nTRU5 TR5=(d U$rZ)rhrj)rWuser_callback_resultgenerator_closed step_results user_callbacks r _callback#_generate_tokens.._callbacks1,[9%&&(@,@@rTr9) asynchronous beam_sizerNc>THnUR5 M TRS5 g![anTRU5 SnAN2SnAff=frZ)result Exceptionrh)re async_resultsrs r_catch_exception*_generate_tokens.._catch_exceptionsN ' (    Q   s. AAA)r-daemon) r]r\r^r_rsupdateThreadstartrSr GeneratorExitrtjoin) rbrcr+rrthreadrWrrrrs @@@@rrIrIs;;=L (JJz40M1 A  MM ! !$1&1M  %5d CF LLN "&&(    k9 - -      KKM    "  s*CC?C C?C<8C?;C<<C?c+t# US:a [S5e[U5S:XaUSnO[R"U6nUR UUSS.5 US:aUS-OUn[ R "5n[XVU5HnUR"U"U0UD65 U(dM$USR5(dM>UR"5R5v U(dMhUSR5(aMDM U(a+UR"5R5v U(aM*gg7f)Nr9zmax_batch_size must be >= 1rT)r$r%r) ValueErrorrR itertools zip_longestr collectionsdeque_batch_iteratorextenddonepopleftr) rbr,r$r%r+iterableread_batch_sizer]streamss rr)r)s677 9~Q<(()4 MM,$  ."_batch_iterator..Fs01Bsz+Input iterables do not have the same length)rStuplerRmaxrzipr()r batch_sizer%rrJexamplebatchelements rrr-sGJ'5))jG  #3wqz?j8  8 #ZWQZ9JC Oa/:= J  _ 4zAB B ?000G!'3NE3wrs KKQ48 :: T#Y :HT#Y/0: :  : :B $$ T#Y $ T#Y $ $  $m$T ""49%"" "  "P "" T#Y "" " m "P*.? # "#! ! !48<@ #?? I?DI&?  ?  ??? ?????!d3i1?c49d3i789? !?"#?$"#%?J N "#! ! !48<@)- $7;'NN $s)T$s)_, -NN N  NNNN NNNNN!d3i1N c49d3i789!N"DI&#N$%N&,-t34'N("#)Nh O "#! ! !48<@)- $7;'OO $s)T$s)_, -OO O  OOOO OOOOO!d3i1O c49d3i789!O"DI&#O$%O&,-t34'O('()Od%%%%P3l'< r