cCi+ S SK JrJrJr SSKJr SSKJr SSKJ r \" 5 ( a S SK
r
SSK J r SS K
Jr S
r " S S \ 5 rg)
)AnyUnionoverload )GenerationConfig)is_torch_available )PipelineN)%MODEL_FOR_TEXT_TO_SPECTROGRAM_MAPPING)SpeechT5HifiGanzmicrosoft/speecht5_hifiganc
* ^ \ rS rSrSrSrSrSrSrSrSr \
" SS9r SSSS.U 4S jjr S
r
S r\S \S
\S\\\4 4S j5 r\S \\ S
\S\\\\4 4S j5 rS \\\\ 4 S\\\\4 \\\\4 4 4U 4S jjr SS jrS rSrU =r$ )TextToAudioPipeline a|
Text-to-audio generation pipeline using any `AutoModelForTextToWaveform` or `AutoModelForTextToSpectrogram`. This
pipeline generates an audio file from an input text and optional other conditional inputs.
Unless the model you're using explicitly sets these generation parameters in its configuration files
(`generation_config.json`), the following default values will be used:
- max_new_tokens: 256
Example:
```python
>>> from transformers import pipeline
>>> pipe = pipeline(model="suno/bark-small")
>>> output = pipe("Hey it's HuggingFace on the phone!")
>>> audio = output["audio"]
>>> sampling_rate = output["sampling_rate"]
```
Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial)
You can specify parameters passed to the model by using [`TextToAudioPipeline.__call__.forward_params`] or
[`TextToAudioPipeline.__call__.generate_kwargs`].
Example:
```python
>>> from transformers import pipeline
>>> music_generator = pipeline(task="text-to-audio", model="facebook/musicgen-small", framework="pt")
>>> # diversify the music generation by adding randomness with a high temperature and set a maximum music length
>>> generate_kwargs = {
... "do_sample": True,
... "temperature": 0.7,
... "max_new_tokens": 35,
... }
>>> outputs = music_generator("Techno music with high melodic riffs", generate_kwargs=generate_kwargs)
```
This pipeline can currently be loaded from [`pipeline`] using the following task identifiers: `"text-to-speech"` or
`"text-to-audio"`.
See the list of available models on [huggingface.co/models](https://huggingface.co/models?filter=text-to-speech).
TF )max_new_tokensN)vocoder
sampling_rate no_processorc
> [ T U ] " U0 UD6 X0l U R S:X a [ S5 eS U l U R R [ R " 5 ; aG Uc= [ R " [ 5 R U R R 5 OUU l X l U R
b% U R
R R U l U R c U R R nU R R" R% SS 5 nUb UR' UR) 5 5 S HL n[+ XhS 5 nUb X l M [+ USS 5 c M* [+ UR, US 5 nUc MF X l MN U R cT U R ( dB [/ U R0 S5 ( a& U R0 R2 R U l g g g g )Ntfz5The TextToAudioPipeline is only available in PyTorch.generation_config) sample_rater codec_configfeature_extractor)super__init__r framework
ValueErrorr model __class__r valuesr from_pretrainedDEFAULT_VOCODER_IDtodevicer config__dict__getupdateto_dictgetattrr hasattr processorr )
selfr r r argskwargsr&
gen_configsampling_rate_namer s
^/home/james-whalen/.local/lib/python3.13/site-packages/transformers/pipelines/text_to_audio.pyr TextToAudioPipeline.__init__a s
$)&) ) >>T ! TU U :: #H#O#O#Q Q ? //0BCFFtzzGXGXY
L + << #!% !4!4!B!BD % ZZ&&F ,,001DdKJ%
j 0 0 23&F" 'D I
,)6& V^T:F$+F,?,?ASUY$ZM$0-:* 'G %d.?.?GDNN\oDpDp!% !A!A!O!OD Eq.? % c h [ U[ 5 ( a U/nU R R R S:X a? U R
R R SS5 SSSSS.nUR U5 UnU R ( a U R O U R nU" U40 UDSS 0D6nU$ )
Nbarkmax_input_semantic_lengthr FT
max_length)r9 add_special_tokensreturn_attention_maskreturn_token_type_idspaddingreturn_tensorspt)
isinstancestrr r&
model_typer semantic_configr( r) r tokenizerr- )r. textr0
new_kwargs preprocessoroutputs r3
preprocess TextToAudioPipeline.preprocess s dC 6D :: ' '6 1 # 4 4 D D H HIdfi j&+)-).' J
f % F)-):):t~~ dBfBTB
r5 c U R X R S9nUS nUS nU R R 5 ( a^ U R X@R S9nSU; a U R US' UR U5 U R R " S0 UDUD6nOC[ U5 ( a [ SUR 5 35 eU R " S0 UDUD6S nU R b U R U5 nU$ )N)r% forward_paramsgenerate_kwargsr zYou're using the `TextToAudioPipeline` with a forward-only model, but `generate_kwargs` is non empty. For forward-only TTA models, please use `forward_params` instead of `generate_kwargs`. For reference, the `generate_kwargs` used here are: r ) _ensure_tensor_on_devicer% r can_generater r) generatelenr keysr )r. model_inputsr0 rL rM rH s r3 _forward TextToAudioPipeline._forward s ..vkk.J 01 !23 :: " " $ $ " ; ;OT_T_ ; `O #/97;7M7M 34
! !/ 2ZZ((J<J>JF?## KKZK_K_KaJbd
ZZA,A.A!DF << #\\&)F
r5 text_inputsrL returnc g NrN r. rW rL s r3 __call__ TextToAudioPipeline.__call__ s SVr5 c g rZ rN r[ s r3 r\ r] s _br5 c & > [ TU ] " U40 UD6$ )a
Generates speech/audio from the inputs. See the [`TextToAudioPipeline`] documentation for more information.
Args:
text_inputs (`str` or `list[str]`):
The text(s) to generate.
forward_params (`dict`, *optional*):
Parameters passed to the model generation/forward method. `forward_params` are always passed to the
underlying model.
generate_kwargs (`dict`, *optional*):
The dictionary of ad-hoc parametrization of `generate_config` to be used for the generation call. For a
complete overview of generate, check the [following
guide](https://huggingface.co/docs/transformers/en/main_classes/text_generation). `generate_kwargs` are
only passed to the underlying model if the latter is a generative model.
Return:
A `dict` or a list of `dict`: The dictionaries have two keys:
- **audio** (`np.ndarray` of shape `(nb_channels, audio_length)`) -- The generated audio waveform.
- **sampling_rate** (`int`) -- The sampling rate of the generated audio waveform.
)r r\ )r. rW rL r s r3 r\ r] s 0 w >~>>r5 c [ U SS 5 b U R US' [ U SS 5 b U R US' U R US' U( a UO0 U( a UO0 S.nUc 0 n0 nXU4$ )Nassistant_modelassistant_tokenizerrD )rL rM )r+ ra rD rb )r. preprocess_paramsrL rM paramspostprocess_paramss r3 _sanitize_parameters(TextToAudioPipeline._sanitize_parameters s 4*D 1 =151E1EO - . 4. 5 A+/>>OK (595M5MO 1 2 1?nB2Ar
$ " *<<V V
bDIbbdSVX[S[nI]b b? d3i 0? tCH~tDcN 33 4?8 =. r5 r )typingr r r
generationr utilsr baser
rr models.auto.modeling_autor !models.speecht5.modeling_speecht5r r# r rN r5 r3 r s> ( ' ) & QC1 w( wr5