cCit S SK Jr S SKJrJr S SKrS SKrSSKJ r J
r
SSK J r SSK
JrJr \
R " \5 r\ " \" S S S
95 " S S \5 5 rg)
)UserDict)AnyUnionN )add_end_docstringslogging ) ffmpeg_read)Pipelinebuild_pipeline_init_argsT)has_feature_extractor
has_tokenizerc ^ \ rS rSrSrSrSrSrSrU 4S jr S\
\ R \
\\4 S\S\\\\4 4U 4S jjrS
rSS jrS rS
rSrU =r$ )#ZeroShotAudioClassificationPipeline a
Zero shot audio classification pipeline using `ClapModel`. This pipeline predicts the class of an audio when you
provide an audio and a set of `candidate_labels`.
The default `hypothesis_template` is : `"This is a sound of {}."`. Make sure you update it for your usage.
Example:
```python
>>> from transformers import pipeline
>>> from datasets import load_dataset
>>> dataset = load_dataset("ashraq/esc50")
>>> audio = next(iter(dataset["train"]["audio"]))["array"]
>>> classifier = pipeline(task="zero-shot-audio-classification", model="laion/clap-htsat-unfused")
>>> classifier(audio, candidate_labels=["Sound of a dog", "Sound of vacuum cleaner"])
[{'score': 0.9996, 'label': 'Sound of a dog'}, {'score': 0.0004, 'label': 'Sound of vacuum cleaner'}]
```
Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This audio
classification pipeline can currently be loaded from [`pipeline`] using the following task identifier:
`"zero-shot-audio-classification"`. See the list of available models on
[huggingface.co/models](https://huggingface.co/models?filter=zero-shot-audio-classification).
FTc x > [ TU ] " S0 UD6 U R S:w a [ SU R S35 eg )NptzThe z is only available in PyTorch. )super__init__ framework
ValueError __class__)selfkwargsr s o/home/james-whalen/.local/lib/python3.13/site-packages/transformers/pipelines/zero_shot_audio_classification.pyr ,ZeroShotAudioClassificationPipeline.__init__D s>
"6" >>T ! tDNN#33Q RS S " audiosr returnc & > [ TU ] " U40 UD6$ )ah
Assign labels to the audio(s) passed as inputs.
Args:
audios (`str`, `list[str]`, `np.array` or `list[np.array]`):
The pipeline handles three types of inputs:
- A string containing a http link pointing to an audio
- A string containing a local path to an audio
- An audio loaded in numpy
candidate_labels (`list[str]`):
The candidate labels for this audio. They will be formatted using *hypothesis_template*.
hypothesis_template (`str`, *optional*, defaults to `"This is a sound of {}"`):
The format used in conjunction with *candidate_labels* to attempt the audio classification by
replacing the placeholder with the candidate_labels. Pass "{}" if *candidate_labels* are
already formatted.
Return:
A list of dictionaries containing one entry per proposed label. Each dictionary contains the
following keys:
- **label** (`str`) -- One of the suggested *candidate_labels*.
- **score** (`float`) -- The score attributed by the model to that label. It is a value between
0 and 1, computed as the `softmax` of `logits_per_audio`.
)r __call__)r r r r s r r" ,ZeroShotAudioClassificationPipeline.__call__K s . w 1&11r c H 0 nSU; a US US' SU; a US US' U0 0 4$ )Ncandidate_labelshypothesis_templater )r r preprocess_paramss r _sanitize_parameters8ZeroShotAudioClassificationPipeline._sanitize_parametersd sI '4:;M4N 0 1 F *7=>S7T 3 4 "b((r c p [ U[ 5 ( ar UR S5 ( d UR S5 ( a! [ R " U5 R
nO%[
US5 nUR 5 nS S S 5 [ U[ 5 ( a [ XR R 5 n[ U[ R 5 ( d [ S5 e[ UR 5 S:w a [# S5 eU R U/U R R SS9nU R$ S:X a UR' U R( 5 nX%S ' U Vs/ s H ocR+ U5 PM nnU R- XpR$ S
S 9nU/US ' U$ ! , ( d f GN(= fs snf )
Nzhttp://zhttps://rbz"We expect a numpy ndarray as inputr zNWe expect a single channel audio input for ZeroShotAudioClassificationPipeliner )
sampling_ratereturn_tensorsr% T)r- padding text_inputs)
isinstancestr
startswithrequestsgetcontentopenreadbytesr
feature_extractorr, npndarray TypeErrorlenshaper r todtypeformat tokenizer) r audior% r& finputsx sequencesr/ s r
preprocess.ZeroShotAudioClassificationPipeline.preprocessm sj eS ! ! **e.>.>z.J.J ! U+33%&! FFHE ' eU # # '='='K'KLE%,, @A A u{{ q mn n''
G4#9#9#G#GX\ (
>>T !YYtzz*F%5!"?"&&}5 k!nh / /%a.K &a.+K**;{;l; !1 ..
r c UR S5 nUS S nU R S:X a UR SS9nUR 5 nO [ S5 e[ [
XR5 S S9 VVs/ s H u pgXgS .PM nnnU$ s snnf )
Nr% rK r r )dimz `tf` framework not supported.c U S * $ )Nr r )rF s r AZeroShotAudioClassificationPipeline.postprocess.. s _`ab_c^cr )key)scorelabel)rL r softmaxtolistr sortedzip) r rQ r% rK probsscoresrZ candidate_labelresults r postprocess/ZeroShotAudioClassificationPipeline.postprocess s (,,-?@ x(+ >>T !NNqN)E\\^F <= = +1V1NTc*d
*d& 6*d
s ,Br )NzThis is a sound of {}.)__name__
__module__ __qualname____firstlineno____doc___load_processor_load_image_processor_load_feature_extractor_load_tokenizerr r r: r; r8 r1 dictr listr" r( rH rR rd __static_attributes__
__classcell__)r s @r r r s~ : O ! "OT2uRZZT%A B 2c 2VZ[_`ceh`h[iVj 22): " r r ) collectionsr typingr r numpyr: r3 utilsr r audio_classificationr
baser r
get_loggerrf loggerr r r r r{ s_ ! . 4
H % ,4W[\]H( H ^Hr