hX`SSKJrJrJrJr SSKrSSKJrJrJ r J r SSK J r J r "SS5rg))AnyCallableOptionalUnionN)GenerationConfigPreTrainedTokenizerPreTrainedTokenizerFastset_seed)SUPPORTED_ARCHITECTURESPreTrainedModelWrapperc \rSrSrSrSS\S\\\4S\ \ \ /\ \ 4S\ S\S \\S \S \\S S4S jjrSS\\ \\R&\ \R&\ \ \4S\S\\\ \R*4S \ \ \ 4SjjrSrg)BestOfNSamplera Sampler for best-of-n generation. Args: model ([`PreTrainedModelWrapper`]): The pretrained model to use for generation. tokenizer ([`~transformers.PreTrainedTokenizer`] or [`~transformers.PreTrainedTokenizerFast`]): Tokenizer associated with the pretrained model. queries_to_scores (`Callable[[list[str]], list[float]]`): Callable that takes a list of generated texts and returns the associated reward scores. length_sampler (`Any`): Sampler used to sample the length of the generated text. sample_size (`int`, *optional*, defaults to `4`): Number of samples to generate for each query. seed (`int`, *optional*): Random seed used to control generation. n_candidates (`int`, *optional*, defaults to `1`): Number of candidates to return for each query. generation_config ([`~transformers.GenerationConfig`], *optional*): Generation config passed to the underlying model's `generate` method. See [`~transformers.GenerationConfig`] for more details. Nmodel tokenizerqueries_to_scoreslength_sampler sample_sizeseed n_candidatesgeneration_configreturnc >Ub [U5 [U[[45(d[ S[ U535e[U[ 5(d[ S[ U5S[ 35eXlX lX0l X@l Xl XPl Xpl g)NzHtokenizer must be a PreTrainedTokenizer or PreTrainedTokenizerFast, got z,model must be a PreTrainedModelWrapper, got z - supported architectures are: )r isinstancerr ValueErrortyper rrrr gen_configrr) selfrrrrrrrrs V/home/james-whalen/.local/lib/python3.13/site-packages/trl/extras/best_of_n_sampler.py__init__BestOfNSampler.__init__/s   TN)&9;R%STTZ[_`i[jZkl %"9;;>tE{mKkmDlEF  "!2,+&(tokenized_queryskip_special_tokensdevicec &Sn[U[R5(a"URS:XaUR S5nO[U[ 5(a[ US5nU[La&[R"U5R S5nOiU[RLa!UVs/sHowRS5PM nnO5UVs/sH(n[R"U5RS5PM* nn/n UHnURURS45nURR"URU54UR5UR S.UD6R#5n UR$R'XS9n [R"UR)U 55n U R+UR,5R.V s/sHoU PM n n U R1U 5 M U $s snfs snfs sn f)a! Generate the best of n samples for input queries. Args: tokenized_query (`list[int]` or `torch.Tensor` or `list[torch.Tensor]` or `list[list[int]]`): Either a single tokenized query (a single tensor or a list of integers) or a batch of tokenized queries (a list of tensors or a list of lists of integers). skip_special_tokens (`bool`, *optional*, defaults to `True`): Whether to remove the special tokens from the output. device (`str` or `torch.device`, *optional*): The device on which the model will be loaded. **generation_kwargs: Additional keyword arguments passed along to the underlying model's `generate` method. This is used to override generation config. Returns: `list[list[str]]`: A list of lists of generated texts. Nr)r()max_new_tokensr)r%)rtorchTensorndim unsqueezelistrinttensorreshaperepeatrrgeneratetorrsqueezer batch_decodertopkrindicesappend) rr$r%r&generation_kwargsqueries element_typer1queryresultoutputscoresis r r4BestOfNSampler.generateOs2 ou|| 4 49M9MQR9R%//2G  . . 23Ls",,7AA!D-APQv>>'2QM\]_E5<<.66w?_]EllD$4$4a#89GZZ(( 6"#224"&//$  gi  ^^000aF\\$"8"8"@AF)/T5F5F)G)O)OP)OAQi)OFP MM& ! 'R]Qs-H/H H)rrrrrrr)Nr(N)TN)__name__ __module__ __qualname____firstlineno____doc__r rrr rr/strfloatrr0rrr!r+r,boolr&r4__static_attributes__r#r rrs:"8<)%),.EEF)$T#YKe$<= )  )  )sm))$$45) )F%)59 5tCy%,,U\\8JDQUVYQZO[\5"5sELL012 5 d3i 55r#r)typingrrrrr+ transformersrrr r modelsr r rrNr#r rRs%21 aaDmmr#