bCiSrSSKJr SSKJrJrJrJr SSKrSSKJ r SSK J r SSK J r Jr SS KJr SS KJrJrJr SS KJrJr SS KJrJrJrJrJrJr S SKJ r J!r!J"r" \RF"\$5r%S\RLS\RL4Sjr'S\RLS\RL4Sjr(S\RLS\RL4Sjr)\\"SS9"SS\555r*\\"SS9"SS\555r+\\"SS\555r,"SS \ RZ5r."S!S"\ RZ5r/SJS#\ RZS$\RLS%\RLS&\RLS'\\RLS(\0S)\0S*\14S+jjr2"S,S-\ RZ5r3"S.S/\ RZ5r4"S0S1\5r5\"S2S3\55r6"S4S5\ RZ5r7"S6S7\ RZ5r8\"S8S9"S9S:\655r9"S;S<\ RZ5r:\"S=S9"S>S?\655r;\"S@SA\655r<\"SBSC\655r=\"SDSE\655r>\"SFS9"SGSH\655r?/SIQr@g)KzPyTorch CLIP model.) dataclass)AnyCallableOptionalUnionN)nn)ACT2FN) _create_4d_causal_attention_mask_prepare_4d_attention_mask)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPoolingImageClassifierOutput)ALL_ATTENTION_FUNCTIONSPreTrainedModel) ModelOutputauto_docstringcan_return_tuplefilter_out_non_signature_kwargslogging torch_int) CLIPConfigCLIPTextConfigCLIPVisionConfiglogitsreturnc[RRU[R"[ U5UR S95$)Ndevice)r functional cross_entropytorcharangelenr!)rs `/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/clip/modeling_clip.pycontrastive_lossr(%s/ == & &vu||CKPVP]P]/^ __ similaritycX[U5n[UR55nX-S- $)Ng@)r(t)r* caption_loss image_losss r' clip_lossr/)s*#J/L!*,,.1J  % ,,r)tensorc[R"US5n[R"USSS9n[R"US5nU$)z This method is equivalent to tensor.norm(p=2, dim=-1, keepdim=True) and used to make model `executorch` exportable. See issue https://github.com/pytorch/executorch/issues/3566 T)dimkeepdim?)r$powsum)r0 square_tensor sum_tensor normed_tensors r'_get_vector_normr</s< IIfa(M=b$?JIIj#.M r)z} Base class for vision model's outputs that also contains image embeddings of the pooling of the last hidden states. ) custom_introc\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \RS4\ S'Sr \\ \RS4\ S'S rg) CLIPVisionModelOutput:z image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)` *optional* returned when model is initialized with `with_projection=True`): The image embeddings obtained by applying the projection layer to the pooler_output. N image_embedslast_hidden_state. hidden_states attentions)__name__ __module__ __qualname____firstlineno____doc__rArr$ FloatTensor__annotations__rBrCtuplerD__static_attributes__rEr)r'r?r?:sr 15L(5,,-459x 1 129=AM8E%"3"3S"89:A:>Ju00#567>r)r?ze Base class for text model's outputs that also contains a pooling of the last hidden states. c\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \RS4\ S'Sr \\ \RS4\ S'S rg) CLIPTextModelOutputLz text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim)` *optional* returned when model is initialized with `with_projection=True`): The text embeddings obtained by applying the projection layer to the pooler_output. N text_embedsrB.rCrDrE)rFrGrHrIrJrRrr$rKrLrBrCrMrDrNrEr)r'rPrPLsr 04K%++,359x 1 129=AM8E%"3"3S"89:A:>Ju00#567>r)rPc\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr\\ S 'Sr\\ S 'S \\4S jrS rg) CLIPOutput^ae loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `return_loss` is `True`): Contrastive loss for image-text similarity. logits_per_image (`torch.FloatTensor` of shape `(image_batch_size, text_batch_size)`): The scaled dot product scores between `image_embeds` and `text_embeds`. This represents the image-text similarity scores. logits_per_text (`torch.FloatTensor` of shape `(text_batch_size, image_batch_size)`): The scaled dot product scores between `text_embeds` and `image_embeds`. This represents the text-image similarity scores. text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`CLIPTextModel`]. image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`CLIPVisionModel`]. text_model_output (`BaseModelOutputWithPooling`): The output of the [`CLIPTextModel`]. vision_model_output (`BaseModelOutputWithPooling`): The output of the [`CLIPVisionModel`]. Nlosslogits_per_imagelogits_per_textrRrAtext_model_outputvision_model_outputrcJ^[U4SjTR555$)Nc3n># UH*nUS;aTUO[TU5R5v M, g7f))rYrZN)getattrto_tuple).0kselfs r' &CLIPOutput.to_tuple..}s<   LLDGRYZ^`aRbRkRkRm m s25)rMkeysras`r'r^CLIPOutput.to_tuple|s# YY[   r)rE)rFrGrHrIrJrVrr$rKrLrWrXrRrArYrrZrMrr^rNrEr)r'rTrT^s&)-D(5$$ %,48hu001837OXe//07/3K%++,304L(5,,-448186:3: %* r)rTc^\rSrSrS\4U4SjjrS\RS\S\S\R4Sjr S S \RS\R4S jjr S r U=r $) CLIPVisionEmbeddingsconfigc>[TU]5 XlURUlUR UlUR Ul[R"[R"UR55Ul [R"URURUR UR SS9UlUR UR -S-UlURS-Ul[R""UR UR5UlUR'S[R("UR 5R+S5SS9 g)NF) in_channels out_channels kernel_sizestridebiasr2r position_idsrr3 persistent)super__init__rj hidden_size embed_dim image_size patch_sizer Parameterr$randnclass_embeddingConv2d num_channelspatch_embedding num_patches num_positions Embeddingposition_embeddingregister_bufferr%expandrarj __class__s r'rvCLIPVisionEmbeddings.__init__s  ++ ++ ++!||EKK,GH!yy++??  !OOt>1D!--1"$,,t/A/A4>>"R ^U\\$:L:L-M-T-TU\-]jopr) embeddingsheightwidthrcURSS- nURRRS5nURSS- n[R R 5(d%XF:Xa X#:XaURUR5$USS2SS24nUSS2SS24nURSn X R-n X0R-n [US-5n URSXU 5nURSSSS5n[RRUX4SS S 9nURSSSS5RSSU 5n[R "Xx4SS 9$) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rrNr3r6r r2bicubicF)sizemode align_cornersr4)shaperweight unsqueezer$jit is_tracingrqrzrreshapepermuterr" interpolateviewcat) rarrrrrrclass_pos_embedpatch_pos_embedr4 new_height new_widthsqrt_num_positionss r'interpolate_pos_encoding-CLIPVisionEmbeddings.interpolate_pos_encodingsi!&&q)A- !44;;EEaH*003a7 yy##%%+*F6?**4+<+<= =,QU3,QU3r". __, &}c'9:)11!5G]`a)11!Q1=--33 ( 4 *11!Q1=BB1b#Nyy/;CCr) pixel_valuesc ^URup4pVU(dJXPR:wdX`R:wa,[SUSUSURSURS3 5eURRR nURUR US95nURS5RSS5nURRUSS5n [R"X/SS 9n U(aXRXU5-n U $XRUR5-n U $) NzInput image size (*z) doesn't match model ().)dtyper2rr3r)rry ValueErrorrrrtoflatten transposer}rr$rrrrq) rarr batch_size_rr target_dtype patch_embeds class_embedsrs r'forwardCLIPVisionEmbeddings.forwards$'3'9'9$ v'V-F%SbSbJb$VHAeW4KDOOK\\]^b^m^m]nnpq ++2288 ++LOO,O,OP #++A.88A> ++22:q"E YY ;C ##&C&CJX]&^^J$&=&=d>O>O&PPJr)) r}rjrxryrrrrzrF)rFrGrHrIrrvr$TensorintrrKrrN __classcell__rs@r'rhrhsjq/q,'D5<<'D'DUX'D]b]i]i'DRE$5$5Z_ZfZfr)rhc ^\rSrSrS\4U4SjjrS S\\RS\\RS\\RS\R4Sjjr S r U=r $) CLIPTextEmbeddingsrjcN>[TU]5 URn[R"UR U5Ul[R"URU5UlURS[R"UR5RS5SS9 g)NrqrrFrs) rurvrwrr vocab_sizetoken_embeddingmax_position_embeddingsrrr$r%rrarjrxrs r'rvCLIPTextEmbeddings.__init__s && !||F,=,=yI"$,,v/M/My"Y  ELL)G)GHOOPWXej  r) input_idsrq inputs_embedsrc<UbURSOURSnURRRSnXE:a[SUSU35eUcURSS2SU24nUcUR U5nURU5nX6-nU$)Nr3rzRSequence length must be less than max_position_embeddings (got `sequence length`: z and max_position_embeddings: )rrrrrqr)rarrqr seq_lengthmax_position_embeddingposition_embeddingsrs r'rCLIPTextEmbeddings.forwards -6,AY__R(}GZGZ[]G^ !%!8!8!?!?!E!Ea!H  .d,<=S[TU]5 XlURUlUR UlURUR -UlURUR -UR:wa&[SURSUR S35eURS-Ul URUl SUl [R"URUR5Ul[R"URUR5Ul[R"URUR5Ul[R"URUR5Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: rF)rurvrjrwrxnum_attention_heads num_headshead_dimrscaleattention_dropoutr is_causalrLineark_projv_projq_projout_projrs r'rvCLIPAttention.__init__s  ++33$..8 ==4>> )T^^ ;MdnnM]^NN#2' ]]D( // ii? ii? ii?  $..$..A r)rCrcausal_attention_maskrrc rURupVnURU5nURU5n URU5n UR XVSUR 5R SS5nU R XVSUR 5R SS5n U R XVSUR 5R SS5n URRS:Xa USLUl OUbUbX#-nOUbUn[n URRS:wa[URRn U "UUU U UURURUR(dSO URUS9 upU RXVU5R!5n UR#U 5n U(dSn X4$) z#Input shape: Batch x Time x Channelr3rr2flash_attention_2Neager)rrrr)rrrrrrrrj_attn_implementationrrrrrrrrr)rarCrrrrrrxqueriesrdvaluesattention_interfacerrs r'rCLIPAttention.forward-s-:,?,?) ++m,{{=)]+,,zr4==ISSTUWXYyyT]]CMMaQRSZRGQQRSUVW ;; + +/B B2$>DN).C.O!/!G&2!6(? ;; + +w 6"9$++:Z:Z"[ $7     nnJJ#}}C$,,/ % ! "))*)LWWY mmK0  L((r)) rjrrxrrrrrrrr)NNF)rFrGrHrIrJrrrrvr$rrboolrMrrNrrs@r'rrsGBu%5~%EFB.268<,1 1)||1)!.1) ( 5 1) $D> 1) u||Xell33 4 1)1)r)rcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)CLIPMLPiac>[TU]5 Xl[URUl[ R"URUR5Ul [ R"URUR5Ul gN) rurvrjr hidden_act activation_fnrrrwintermediate_sizefc1fc2rs r'rvCLIPMLP.__init__bsb  #F$5$5699V//1I1IJ99V55v7I7IJr)rCrclURU5nURU5nURU5nU$r)rrr)rarCs r'rCLIPMLP.forwardis4/ **=9 / r))rrjrr) rFrGrHrIrvr$rrrNrrs@r'rras)KU\\ellr)rc ^\rSrSrS\\\44U4SjjrS S\RS\RS\RS\ \ S\ \R4 S jjrS rU=r$) CLIPEncoderLayeriprjc<>[TU]5 URUl[ U5Ul[ R"URURS9Ul [U5Ul [ R"URURS9Ul gN)eps) rurvrwrxr self_attnr LayerNormlayer_norm_eps layer_norm1rmlp layer_norm2rs r'rvCLIPEncoderLayer.__init__qsl ++&v.<<F & u  ! &&r)rc>\rSrSr%\\S'SrSrSrSr Sr Sr Sr Sr g)CLIPPreTrainedModelirjclipTc" URRn[U[5(adURR R RSUS-S9 URR R RSUS-S9 GOF[U[5(aURRn[RRURSURS-U-S9 [RRURR URRU-S9 [RRURR URRU-S9 GON[U[ 5(Ga!URRnURS-SURR"-S--U-nURS-U-n[RRUR$R US9 [RRUR&R US9 [RRUR(R US9 [RRUR*R US9 GO[U[,5(aURRnURR.S-SURR"-S--U-nSURR.-S-U-n[RRUR0R US9 [RRUR2R US9 GO.[U[45(a[RRUR6R UR8S-URR-S9 [RRUR:R UR<S-URR-S9 GOk[U[>5(aa[RRUR:R URR.S-URR-S9 O[U[@5(aa[RRUR6R URR.S-URR-S9 O[U[B5(aj[RRURDR URRFR.S-URR-S9 [U[RH5(aIURJR RM5 UR R ROS5 [U[RP5(a3URJb%URJR RM5 ggg) zInitialize the weightsrg{Gz?)meanstdr)rr2g?N))rjinitializer_factor isinstancerrrdatanormal_rrhrinitr}rxrinitializer_rangernum_hidden_layersrrrrrrwrr CLIPModeltext_projectiontext_embed_dimvisual_projectionvision_embed_dimCLIPVisionModelWithProjectionCLIPTextModelWithProjectionCLIPForImageClassification classifier vision_configr rpzero_fill_r)rarfactor in_proj_std out_proj_stdfc_stds r' _init_weights!CLIPPreTrainedModel._init_weightss// f0 1 1  " " ) ) . . 6 6CVd] 6 S  % % , , 1 1 9 9sQU 9 V  4 5 5[[33F GGOOF22&BRBRTXBX[aBaO b GGOOF2299v}}?^?^ag?gO h GGOOF55<<&--BaBadjBjO k  . .[[33F!++T1q6==;Z;Z7Z_c6cdgmmK",,d2f > GGOO((//KK++T1DKK4R4RR   ; < < GGOO&&--KK++T1DKK4R4RR   : ; ; GGOO!!((KK--994?$++B`B``   fbll + + KK   " " $ MM   $ $S ) fbii ( (V[[-D KK   " " $.E (r)rEN)rFrGrHrIrrLbase_model_prefixsupports_gradient_checkpointing_supports_sdpa_supports_flash_attn_supports_flex_attn_supports_attention_backendr2rNrEr)r'rrs0 &*#N"&6%r)rc ^\rSrSrSrS\4U4SjjrS S\\RS\\RS\\ S\\ S \ 4 S jjr S r U=r$) CLIPEncoderiz Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`CLIPEncoderLayer`]. Args: config: CLIPConfig rjc>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snf)NF) rurvrjr ModuleListranger!rlayersgradient_checkpointing)rarjrrs r'rvCLIPEncoder.__init__sS  mmuVMeMeGf$gGf!%5f%=Gf$gh &+#%hsA&rrroutput_hidden_statesrc^UbUOURRnUbUOURRnU(aSOSnU(aSOSnUn[UR5H0upU(aXh4-nU "UUUUS9n U SnU(dM(X{S4-nM2 U(aXh4-n[ UUUS9$)a Args: inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) causal_attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Causal mask for the text model. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NrE)rrr)rBrCrD)rjrrB enumerater?r) rarrrrrBencoder_statesall_attentionsrCidx encoder_layer layer_outputss r'rCLIPEncoder.forwardsJ2C1N-TXT_T_TqTq$8$D $++JjJj  40d% "+DKK"8 C#!/2B!B)%"3 M*!,M  !/3C2E!E#9 +.>>N+(%  r))rjr@r?NNNN)rFrGrHrIrJrrvrr$rrrrrNrrs@r'r;r;s,z,268<,0/3 D !.D  ( 5 D $D> D 'tn D  D D r)r;c^\rSrSrS\4U4Sjjr\S S\\RS\\RS\\RS\\ S\\ S \ 4 S jj5r S r U=r$) CLIPTextTransformeri;rjc>[TU]5 XlURn[ U5Ul[ U5Ul[R"X!RS9Ul URUl gr) rurvrjrwrrr;encoderrr r final_layer_norm eos_token_idrs r'rvCLIPTextTransformer.__init__<s]  && ,V4"6* " Y F 'tn F  $F F r)rMzI The text model from CLIP without any head or projection on top. c^\rSrSr%\\S'SS/rSrS\4U4SjjrS\ R4Sjr S r \ \SS \\R"S \\R"S \\R"S \\S\\S\4 Sjj55rSrU=r$) CLIPTextModelirjrrFcd>[TU]U5 [U5UlUR 5 gr)rurvrM text_model post_initrs r'rvCLIPTextModel.__init__s&  -f5 r)rcBURRR$rr_rrres r'get_input_embeddings"CLIPTextModel.get_input_embeddings))999r)c8XRRlgrrcrars r'set_input_embeddings"CLIPTextModel.set_input_embeddings5:""2r)rrrqrrBc(URUUUUUS9$)a Examples: ```python >>> from transformers import AutoTokenizer, CLIPTextModel >>> model = CLIPTextModel.from_pretrained("openai/clip-vit-base-patch32") >>> tokenizer = AutoTokenizer.from_pretrained("openai/clip-vit-base-patch32") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled (EOS token) states ```rrrqrrBr_)rarrrqrrBs r'rCLIPTextModel.forwards)4)%/!5   r)rnr[)rFrGrHrIrrL_no_split_modulesr7rvrModulerdrirrrr$rrrrrNrrs@r'r]r]s -/AB ~ :bii:;-115/3,0/3  ELL) !. u||,  $D>  'tn   $  r)r]c ^\rSrSrS\4U4Sjjr\S S\\RS\\ S\\ S\\ S\ 4 S jj5r S r U=r$) CLIPVisionTransformerirjc>[TU]5 XlURn[ U5Ul[ R"X!RS9Ul [U5Ul [ R"X!RS9Ul gr) rurvrjrwrhrrr r  pre_layrnormr;rOpost_layernormrs r'rvCLIPVisionTransformer.__init__sd  && .v6LL8M8MN"6*  ll9:O:OPr)rrrBrrc~UbUOURRnUbUOURRnUc [S5eUR XS9nUR U5nUR UUUS9nURnUSS2SSS24nURU5n[UUURURS9$)Nz You have to specify pixel_values)r)rrrBrrT) rjrrBrrrurOrBrvrrCrD) rarrrBrrCrXrBrYs r'rCLIPVisionTransformer.forwards2C1N-TXT_T_TqTq$8$D $++JjJj   ?@ @ h ))-8 +/<<'/!5,8,  ,==)!Q'2 ++M: )/')77&11   r))rjrrOrvruNNNF)rFrGrHrIrrvrrr$rKrrrrNrrs@r'rsrss{Q/Q59,0/338 ! u001! $D>! 'tn ! #+4. ! $ ! ! r)rszK The vision model from CLIP without any head or projection on top. c ^\rSrSr%\\S'SrS/rS\4U4SjjrS\ R4Sjr \ \ S S\\R S\\S \\S \S\4 S jj55rS rU=r$)CLIPVisionModelirjrrcd>[TU]U5 [U5UlUR 5 gr)rurvrs vision_modelr`rs r'rvCLIPVisionModel.__init__s'  1&9 r)rcBURRR$rr~rrres r'rd$CLIPVisionModel.get_input_embeddings   ++;;;r)rrBrc&URUUUUS9$)ag Example: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, CLIPVisionModel >>> model = CLIPVisionModel.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled CLS states ```rrrBrr~)rarrrBrs r'rCLIPVisionModel.forwards(<  %/!5%= !  r)rrz)rFrGrHrIrrLmain_input_namerprvrrqrdrrrr$rKrrrrNrrs@r'r|r|s $O+,/ ! 'tn ! #' ! $ ! ! r)r|c^\rSrSr%\\S'/SQrSrS\4U4Sjjr\ "5\ SS\ RS\ \ RS\ \ RS \ R4S jj55r\ "5\ SS \ RS \S \ R4S jj55r\\ SS\ \ R&S \ \ RS\ \ RS\ \ R&S\ \S\ \S\ \S \S \4Sjj55rSrU=r$)r"i4rj)rrrhFc>[TU]U5 [UR[5(d"[ S[ UR5S35e[UR[5(d"[ S[ UR5S35eURnURnURUl URUl URUl [RU5nURUl[ RU5nUR"Ul[$R&"URURSS9Ul[$R&"URURSS9Ul[$R,"[.R0"UR2R455UlUR95 g)NzKconfig.text_config is expected to be of type CLIPTextConfig but is of type .zOconfig.vision_config is expected to be of type CLIPVisionConfig but is of type Frp)rurvr text_configr TypeErrortyper+rprojection_dimrwr$r&r] _from_configr_r|r~rrr%r#r{r$r0rjlogit_scale_init_value logit_scaler`)rarjrr+r_r~rs r'rvCLIPModel.__init__:s~  &,,n==++,-Q0  &..0@AA--./q2  (( ,, $33)55 - 9 9"// < $//&33MB (55!#4+@+@$BUBU\a!b!yy)<)Q>QX]^<< T[[5W5W(XY r)rrrqrcbURUUUS9nURnURU5nU$)ay Returns: text_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`CLIPTextModel`]. Examples: ```python >>> import torch >>> from transformers import AutoTokenizer, CLIPModel >>> model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32") >>> tokenizer = AutoTokenizer.from_pretrained("openai/clip-vit-base-patch32") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> with torch.inference_mode(): ... text_features = model.get_text_features(**inputs) ```)rrrq)r_rUr#)rarrrq text_outputsrY text_featuress r'get_text_featuresCLIPModel.get_text_features]sD648??)%4C4 %22 ,,]; r)rrc`URUUS9nURnURU5nU$)a Returns: image_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`CLIPVisionModel`]. Examples: ```python >>> import torch >>> from transformers import AutoProcessor, CLIPModel >>> from transformers.image_utils import load_image >>> model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = load_image(url) >>> inputs = processor(images=image, return_tensors="pt") >>> with torch.inference_mode(): ... image_features = model.get_image_features(**inputs) ```)rr)r~rUr%)rarrvision_outputsrYimage_featuress r'get_image_featuresCLIPModel.get_image_featuressC<6:5F5F%%=6G6 '44 // >r) return_lossrrBc UbUOURRnUbUOURRnURUUUUS9n UR UUUUUS9n U R n UR U 5n U R n URU 5n U [U 5- n U [U 5- n [R"XR5RU R55n XRR5RU R5-n U R5nSnU(a [!U 5n[#UUU U U U U S9$)aj return_loss (`bool`, *optional*): Whether or not to return the contrastive loss. Examples: ```python >>> import torch >>> from transformers import AutoProcessor, CLIPModel >>> from transformers.image_utils import load_image >>> model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = load_image(url) >>> inputs = processor( ... text=["a photo of a cat", "a photo of a dog"], images=image, return_tensors="pt", padding=True ... ) >>> with torch.inference_mode(): ... outputs = model(**inputs) >>> logits_per_image = outputs.logits_per_image # this is the image-text similarity score >>> probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities ```Nrrm)rVrWrXrRrArYrZ)rjrrBr~r_rUr%r#r<r$rr,rr!rexpr/rT)rarrrrqrrrBrrrrArRrXrWrVs r'rCLIPModel.forwardsqP2C1N-TXT_T_TqTq$8$D $++JjJj 6:5F5F%/!5%= 6G6 48??)%/!5 4C4 &33 --l; "00 **;7 $&6|&DD !$4[$AA  ,,{NN4D4G4G HZHZ4[\),<,<,@,@,B,E,EkFXFX,YY*,,. _-D-+#%* .  r))rrr$r_r#r&r~r%)NNr)NNNNNNNF)rFrGrHrIrrLrpr7rvrrr$rrrKrrrrrrTrrNrrs@r'r"r"4s Z !z!F%&26/3 !<<!!.!u||, !    !'!F%&*/#''##'#    #'#J15481537&*,0/3).V E,,-V u001V !. V u//0 V d^ V $D>V 'tnV #'V  V V r)r"c^\rSrSr%\\S'SrSS/rS\4U4SjjrS\ R4Sjr S r \ \SS \\R"S \\R"S \\R"S \\S\\S\4 Sjj55rSrU=r$)r(irjFrrc>[TU]U5 [RU5nURUl[ R "URURSS9Ul UR5 gNFr) rurvr]rr_rrrwrr#r`)rarjr_rs r'rv$CLIPTextModelWithProjection.__init__ s[  "//7 $//!yy););V=R=RY^_ r)rcBURRR$rrcres r'rd0CLIPTextModelWithProjection.get_input_embeddingsrfr)c8XRRlgrrcrhs r'ri0CLIPTextModelWithProjection.set_input_embeddingsrkr)rrrqrrBcURUUUUUS9nURnURU5n[UURUR UR S9$)a Examples: ```python >>> import torch >>> from transformers import AutoTokenizer, CLIPTextModelWithProjection >>> model = CLIPTextModelWithProjection.from_pretrained("openai/clip-vit-base-patch32") >>> tokenizer = AutoTokenizer.from_pretrained("openai/clip-vit-base-patch32") >>> inputs = tokenizer(["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> with torch.inference_mode(): ... outputs = model(**inputs) >>> text_embeds = outputs.text_embeds ```rm)rRrBrCrD)r_rUr#rPrBrCrD) rarrrqrrBrrYrRs r'r#CLIPTextModelWithProjection.forwardso648??)%/!5 4C4 %22 **=9 "#*<<&44#..   r))r_r#r[)rFrGrHrIrrLr7rprvrrqrdrirrrr$rrrPrrNrrs@r'r(r(s  -/AB ~ :bii:;-115/3,0/3 ( ELL)( !.( u||, ( $D> ( 'tn (  ( ( r)r(c ^\rSrSr%\\S'SrS\4U4SjjrS\R4Sjr \ \ S S\ \RS\ \S\ \S \S\4 S jj55rS rU=r$) r'iIrjrc>[TU]U5 [RU5nURUl[ R "URURSS9Ul UR5 gr) rurvr|rr~rrrwrr%r`rarjr~rs r'rv&CLIPVisionModelWithProjection.__init__Ns\  &33F; (55!#6+=+=v?T?T[`!a r)rcBURRR$rrres r'rd2CLIPVisionModelWithProjection.get_input_embeddingsYrr)rrBrcURUUUUS9nURnURU5n[UURUR UR S9$)aX Examples: ```python >>> import torch >>> from transformers import AutoProcessor, CLIPVisionModelWithProjection >>> from transformers.image_utils import load_image >>> model = CLIPVisionModelWithProjection.from_pretrained("openai/clip-vit-base-patch32") >>> processor = AutoProcessor.from_pretrained("openai/clip-vit-base-patch32") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = load_image(url) >>> inputs = processor(images=image, return_tensors="pt") >>> with torch.inference_mode(): ... outputs = model(**inputs) >>> image_embeds = outputs.image_embeds ```r)rArBrCrD)r~rUr%r?rBrCrD)rarrrBrrrYrAs r'r%CLIPVisionModelWithProjection.forward\sn<6:5F5F%/!5%= 6G6  '44 --m< $%,>>(66%00   r))r~r%rz)rFrGrHrIrrLrrvrrqrdrrrr$rKrr?rrNrrs@r'r'r'Is $O / * 'tn * #' *  * * r)r'z CLIP vision encoder with an image classification head on top (a linear layer on top of the pooled final hidden states of the patch tokens) e.g. for ImageNet. c ^\rSrSrSrS\SS4U4Sjjr\\S S\ \ RS\ \ RS\ \ S \ \ S\ 4 S jj55rS rU=r$) r)irrjrNc>[TU]U5 URUl[R UR 5nUR UlURS:a5[R"UR RUR5O[R"5Ul UR5 g)Nr) rurv num_labelsr|rr+r~rrrwIdentityr*r`rs r'rv#CLIPForImageClassification.__init__s   ++&33F4H4HI (55OUN_N_bcNcBIIf**668I8I Jikititiv  r)labelsrrBcUbUOURRnUbUOURRnURUUUS9nURn[ R "USS2SS2SS24SS9nURU5nSnUbURX'UR5n[UUURURS9$)ab labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the image classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). N)rrBrr)rVrrCrD) rjrrBr~rBr$rr* loss_functionrrCrD) rarrrrBrsequence_outputrrVs r'r"CLIPForImageClassification.forwards2C1N-TXT_T_TqTq$8$D $++JjJj /3.?.? /!5/@/  "33 **_QAX%>AF1  %%fdkkBD$!//))   r))r*rr~rK)rFrGrHrIrrrvrrrr$rrrrrNrrs@r'r)r)s%O z d 04)-,0/3 ( u||,( &( $D> ( 'tn (  ( ( r)r))r"rr]r(r|r'r))rT)ArJ dataclassesrtypingrrrrr$r activationsr modeling_attn_mask_utilsr r modeling_layersr modeling_outputsrrrmodeling_utilsrrutilsrrrrrrconfiguration_cliprrr get_loggerrFloggerrr(r/r<r?rPrTrqrhrfloatrrrrrrr;rMr]rsr|r"r(r'r)__all__rEr)r'rs!11 !d9bbFwwLL   H % `U\\`ell`-%,,-5<<- U\\ell  ?K ?  ?  ?+ ?  ?        FP299Pf%%^"% II% <<% % << % U\\* %  %%%0H)BIIH)V bii /1/d?%/?%?%DS "))S lS "))S l 2 '2  2 j- BII- ` 1 )1  1 hL #L L ^A "5A A H> $7> > B < !4<  < ~ r)