bCiSrSSKrSSKJr SSKJrJrJrJr SSK r SSK J r SSK J r SSKJr SSKJrJrJrJr SS KJrJr SS KJrJrJr SS KJrJrJrJ r J!r!J"r" S S K#J$r$J%r%J&r& \!RN"\(5r)S\ RTS\ RT4Sjr+S\ RTS\ RT4Sjr,\\"SS\555r-"SS\ R\5r/"SS\ R\5r0"SS\ R\5r1S\00r2"SS\ R\5r3"SS\ R\5r4"S S!\ R\5r5"S"S#\5r6"S$S%\ R\5r7"S&S'\ R\5r8SJS(\ R\S)\ RTS*\ RTS+\ RTS,\\ RTS-\9S.\94S/jjr:"S0S1\ R\5r;"S2S3\ R\5r<"S4S5\5r="S6S7\ R\5r>"S8S9\ R\5r?\"S:S;\55r@"S<S=\ R\5rA"S>S?\@5rB\"S@SA9"SBSC\@55rC"SDSE\@5rD"SFSG\@5rESKSHjrF/SIQrGg)LzPyTorch AltCLIP model.N) dataclass)AnyCallableOptionalUnion)ACT2FN)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPooling,BaseModelOutputWithPoolingAndCrossAttentions'BaseModelOutputWithPoolingAndProjection)ALL_ATTENTION_FUNCTIONSPreTrainedModel)apply_chunking_to_forward find_pruneable_heads_and_indicesprune_linear_layer) ModelOutputauto_docstringcan_return_tuplefilter_out_non_signature_kwargslogging torch_int) AltCLIPConfigAltCLIPTextConfigAltCLIPVisionConfiglogitsreturnc[RRU[R"[ U5UR S95$)Ndevice)nn functional cross_entropytorcharangelenr")rs f/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/altclip/modeling_altclip.pycontrastive_lossr*+s/ == & &vu||CKPVP]P]/^ __ similaritycX[U5n[UR55nX-S- $)Ng@)r*t)r, caption_loss image_losss r) clip_lossr1/s*#J/L!*,,.1J  % ,,r+c\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr \\R\ S'Sr\\ S 'Sr\\ S 'S \\4S jrS rg) AltCLIPOutput5aq loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `return_loss` is `True`): Contrastive loss for image-text similarity. logits_per_image (`torch.FloatTensor` of shape `(image_batch_size, text_batch_size)`): The scaled dot product scores between `image_embeds` and `text_embeds`. This represents the image-text similarity scores. logits_per_text (`torch.FloatTensor` of shape `(text_batch_size, image_batch_size)`): The scaled dot product scores between `text_embeds` and `image_embeds`. This represents the text-image similarity scores. text_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`AltCLIPTextModel`]. image_embeds (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`AltCLIPVisionModel`]. text_model_output (`BaseModelOutputWithPooling`): The output of the [`AltCLIPTextModel`]. vision_model_output (`BaseModelOutputWithPooling`): The output of the [`AltCLIPVisionModel`]. Nlosslogits_per_imagelogits_per_text text_embeds image_embedstext_model_outputvision_model_outputrcJ^[U4SjTR555$)Nc3n># UH*nUS;aTUO[TU5R5v M, g7f))r:r;N)getattrto_tuple).0kselfs r) )AltCLIPOutput.to_tuple..Us<   LLDGRYZ^`aRbRkRkRm m s25)tuplekeysrBs`r)r?AltCLIPOutput.to_tupleTs# YY[   r+)__name__ __module__ __qualname____firstlineno____doc__r5rr& FloatTensor__annotations__r6r7r8r9r:r r;rErr?__static_attributes__rIr+r)r3r35s&)-D(5$$ %,48hu001837OXe//07/3K%++,304L(5,,-448186:3: %* r+r3c>^\rSrSrSrU4SjrSSjrSrSrU=r $)AltRobertaEmbeddings\zN Same as BertEmbeddings with a tiny tweak for positional embeddings indexing. c>[TU]5 [R"URUR UR S9Ul[R"URUR 5Ul [R"URUR 5Ul [R"UR URS9Ul [R"UR5Ul[#USS5UlUR'S[(R*"UR5R-S5SS9 UR'S [(R."UR0R35[(R4S 9SS9 UR Ul[R"URUR UR6S9Ul g) N) padding_idxepsposition_embedding_typeabsolute position_idsrF persistenttoken_type_idsdtype)super__init__r# Embedding vocab_size hidden_size pad_token_idword_embeddingsmax_position_embeddingsposition_embeddingstype_vocab_sizetoken_type_embeddings LayerNormlayer_norm_epsDropouthidden_dropout_probdropoutr>rYregister_bufferr&r'expandzerosr[sizelongrVrBconfig __class__s r)rdAltRobertaEmbeddings.__init__bsi !||F,=,=v?Q?Q_e_r_rs#%<<0N0NPVPbPb#c %'\\&2H2H&J\J\%]"f&8&8f>S>STzz&"<"<= '.v7PR\']$  ELL)G)GHOOPWXej    ekk$*;*;*@*@*B%**Ubg  "..#%<<  * *F,>,>DL\L\$  r+cUc+Ub[XRU5nOURU5nUbUR5nOUR5SSnUSnUcv[ US5(a-UR SS2SU24nUR USU5n U nO8[R"U[RURRS9nUcURU5nURU5n XJ-n URS:XaURU5n X- n UR!U 5n UR#U 5n U $)Nr]rr`rrbr"rZ)"create_position_ids_from_input_idsrV&create_position_ids_from_inputs_embedsrvhasattrr`rtr&rurwr[r"rirmrYrkrnrr) rB input_idsr`r[ inputs_embedspast_key_values_length input_shape seq_lengthbuffered_token_type_ids buffered_token_type_ids_expandedrm embeddingsrks r)forwardAltRobertaEmbeddings.forward{sM  $A)M]M]_uv #JJ=Y  #..*K',,.s3K ^  !t-..*.*=*=a*n*M'3J3Q3QR]^_R`bl3m0!A!&[ SWSdSdSkSk!l   00;M $ : :> J":  ' ': 5"&":":<"H   -J^^J/ \\*- r+cUR5SSnUSn[R"URS-X0R-S-[RUR S9nUR S5RU5$)z We are provided embeddings directly. We cannot infer which are padded so just generate sequential position ids. Args: inputs_embeds: torch.Tensor Returns: torch.Tensor Nr]rr}r)rvr&r'rVrwr" unsqueezert)rBrrsequence_lengthr[s r)r;AltRobertaEmbeddings.create_position_ids_from_inputs_embedss~$((*3B/ %a.||   q /4D4D"Dq"HPUPZPZcpcwcw %%a(// <[TU]5 URUR-S:wa7[ US5(d&[ SURSURS35eURUl[ URUR- 5UlURUR-Ul[R"URUR5Ul [R"URUR5Ul [R"URUR5Ul [R"UR5UlU=(d [#USS5UlUR$S:XdUR$S :XaHUR&Ul[R("S UR&-S - UR5Ulgg) Nrembedding_sizezThe hidden size (z6) is not a multiple of the number of attention heads ()rYrZ relative_keyrelative_key_queryr)rcrdrgnum_attention_headsr ValueErrorintattention_head_size all_head_sizer#Linearquerykeyvaluerpattention_probs_dropout_probrrr>rYrjredistance_embeddingrBryrYrzs r)rd AltRobertaSelfAttention.__init__s     : : :a ?PVXhHiHi#F$6$6#78 445Q8  $*#=#= #&v'9'9F( ' -zC $  ' '> 9T=Y=Y]q=q+1+I+ID (&(ll1v7U7U3UXY3Y[_[s[s&tD #>rr+ hidden_statesattention_mask head_maskoutput_attentionsrcHURSSn/UQSPURP7nURU5RU5R SS5nUR U5RU5R SS5nUR U5RU5R SS5n [R"XxR SS55n URS:XdURS:XGaLURSURSp[R"U [RURS9RSS5n [R"U [RURS9RSS5nX- nURXR-S- 5nURUR S9nURS:Xa[R""S UU5nU U-n OHURS:Xa8[R""S UU5n[R""S UU5nU U-U-n U [$R&"UR5- n UbX-n [(R*R-U SS 9nUR/U5nUbUU-n[R"UU 5nUR1S SSS 5R35nUR55SSUR64-nURU5nU(aUU4nU$U4nU$)Nr]rrrrr}razbhld,lrd->bhlrzbhrd,lrd->bhlrdimrr)shaperrview transposerrr&matmulrYr'rwr"rrjtorbeinsummathsqrtr#r$softmaxrrpermute contiguousrvr)rBrrrrr hidden_shape query_layer key_layer value_layerattention_scores query_length key_lengthposition_ids_lposition_ids_rdistancepositional_embeddingrelative_position_scoresrelative_position_scores_queryrelative_position_scores_keyattention_probs context_layernew_context_layer_shapeoutputss r)rAltRobertaSelfAttention.forwards$))#2. CCbC$*B*BC jj/44\BLLQPQR HH]+00>HHAN jj/44\BLLQPQR !<< 5H5HR5PQ  ' '> 9T=Y=Y]q=q'2'8'8';Y__Q=O*"\\,ejjQ^QeQefkklnpqrN"\\*EJJ}OcOcdiijkmopN%6H#'#:#:8FbFb;bef;f#g #7#:#:ARAR#:#S ++~=+0<<8H+Wk+l(#36N#N --1EE16>NP[]q1r./4||S"S%**+BC 6G=/2O\M]r+) rrrrrrrjrrYrrNNNF)rJrKrLrMrdr&TensorrrOboolrErrQrrs@r)rrsuu67;15,1 :||:!!2!23:E--. : $D> : u||  ::r+rcz^\rSrSrU4SjrS\R S\R S\R 4SjrSrU=r $)AltRobertaSelfOutputi c(>[TU]5 [R"URUR5Ul[R "URURS9Ul[R"UR5Ul gNrW) rcrdr#rrgdensernrorprqrrrxs r)rdAltRobertaSelfOutput.__init__ s` YYv1163E3EF f&8&8f>S>STzz&"<"<= r+r input_tensorrcpURU5nURU5nURX-5nU$rrrrrnrBrrs r)rAltRobertaSelfOutput.forward5 =1  ]3 }'CD r+rnrrr rJrKrLrMrdr&rrrQrrs@r)rr 6> U\\RWR^R^r+reagerc ^\rSrSrS U4SjjrSrS S\RS\\RS\\RS\\ S\ \R4 S jjr S r U=r$) AltRobertaAttentionic>[TU]5 [UR"XS9Ul[ U5Ul[5Ulg)N)rY) rcrd"ALT_ROBERTA_SELF_ATTENTION_CLASSES_attn_implementationrBroutputset pruned_headsrs r)rdAltRobertaAttention.__init__ s@ 6v7R7RS   +62 Er+c6[U5S:Xag[XRRURRUR 5up[ URRU5URl[ URRU5URl[ URRU5URl [ URRUSS9URl URR[U5- URlURRURR-URl UR RU5Ulg)Nrrr)r(rrBrrrrrrrrrrunion)rBheadsindexs r) prune_headsAltRobertaAttention.prune_heads(s  u:? 7 9900$))2O2OQUQbQb  -TYY__eD *499==%@ ,TYY__eD .t{{/@/@%QO )- (E(EE (R %"&))"?"?$))B_B_"_  --33E:r+rrrrrcfURUUUUS9nURUSU5nU4USS-nU$N)rrrrr)rBr)rBrrrr self_outputsattention_outputrs r)rAltRobertaAttention.forward:sSyy )/ !  ;;|A F#% QR(88r+)rrrBrr)rJrKrLrMrdrr&rrrOrrErrQrrs@r)rrsy";*7;15,1 ||!!2!23E--.  $D>  u||  r+rcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)AltRobertaIntermediateiMc>[TU]5 [R"URUR 5Ul[UR[5(a[URUl gURUl gr) rcrdr#rrgintermediate_sizer isinstance hidden_actstrr intermediate_act_fnrxs r)rdAltRobertaIntermediate.__init__Ns` YYv1163K3KL f'' - -'-f.?.?'@D $'-'8'8D $r+rrcJURU5nURU5nU$rrrrBrs r)rAltRobertaIntermediate.forwardVs& =1 00? r+rrrs@r)rrMs(9U\\ellr+rcz^\rSrSrU4SjrS\R S\R S\R 4SjrSrU=r $)AltRobertaOutputi]c(>[TU]5 [R"URUR 5Ul[R"UR URS9Ul[R"UR5Ul gr) rcrdr#rrrgrrnrorprqrrrxs r)rdAltRobertaOutput.__init__^s` YYv779K9KL f&8&8f>S>STzz&"<"<= r+rrrcpURU5nURU5nURX-5nU$rrrs r)rAltRobertaOutput.forwarddrr+rrrs@r)rr]rr+rc ^\rSrSrU4SjrS S\R S\\RS\\RS\\ S\ \R 4 Sjjr S r S r U=r$) AltRobertaLayerilc>[TU]5 URUlSUl[ U5Ul[ U5Ul[U5Ul g)Nr) rcrdchunk_size_feed_forward seq_len_dimr attentionr intermediaterrrxs r)rdAltRobertaLayer.__init__msI '-'E'E$,V426:&v. r+rrrrrc UR"U4UUUS.UD6nUSnUSSn[URURURU5n U 4U-nU$r)r rfeed_forward_chunkr r ) rBrrrrkwargsself_attention_outputsrr layer_outputs r)rAltRobertaLayer.forwardus"& " )/ "  " 2!4(,0  # #T%A%A4CSCSUe  /G+r+cJURU5nURX!5nU$r)rr)rBrintermediate_outputrs r)r"AltRobertaLayer.feed_forward_chunks)"//0@A{{#6I r+)r r rrr r)rJrKrLrMrdr&rrrOrrErrrQrrs@r)r r lsy/7;15,1 ||!!2!23E--.  $D>  u|| 2r+r c^\rSrSrU4Sjr\S S\RS\\RS\\RS\\ S\\ S\\ S \ \ \R\ 44S jj5rS rU=r$) AltRobertaEncoderic>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snfNF) rcrdryr# ModuleListrangenum_hidden_layersr layergradient_checkpointing)rBryirzs r)rdAltRobertaEncoder.__init__sR  ]]U6KcKcEd#eEdOF$;Ed#ef &+#$fA&rrrroutput_hidden_states return_dictrc U(aSOSnU(aSOSn [UR5H=upU(aX4-nUbX:OSn U "SUUU US.UD6n U SnU(dM5XS4-n M? U(aX4-n[UUU S9$)NrI)rrrrrrlast_hidden_stater attentions) enumerater r )rBrrrrr%r&rall_hidden_statesall_self_attentionsr" layer_modulelayer_head_mask layer_outputss r)rAltRobertaEncoder.forwards#7BD$5b4(4OA#$58H$H!.7.CilO(+-)"3   M*!,M  &91=M & 'tn & d^&  uU\\"O3 4& & r+rcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)AltRobertaPooleric>[TU]5 [R"URUR5Ul[R "5Ulgr)rcrdr#rrgrTanh activationrxs r)rdAltRobertaPooler.__init__s9 YYv1163E3EF '')r+rrc\USS2S4nURU5nURU5nU$)Nr)rr6)rBrfirst_token_tensor pooled_outputs r)rAltRobertaPooler.forwards6+1a40 #56  6 r+)r6rrrs@r)r3r3s($ U\\ellr+r3modulerrrrscalingrrc [R"XRSS55U-nUbX-n[RR US[R S9RUR5n[RRXURS9n[R"X5n U RSS5R5n X4$)Nr]r)rrb)ptrainingrr) r&rrr#r$rfloat32rrbrrr@r) r<rrrrr=rrr attn_weights attn_outputs r)eager_attention_forwardrDs<<}}R'<=GL!#4 ==((2U]](SVVW\WbWbcL==((6??([L,,|3K''1-88:K  $$r+c^\rSrSrSrU4SjrS S\RS\\RS\\RS\\ S\ \R\\R44 S jjr S r U=r $) AltCLIPAttentioniz=Multi-headed attention from 'Attention Is All You Need' paperc >[TU]5 XlURUlUR UlURUR -UlURUR -UR:wa&[SURSUR S35eURS-Ul URUl SUl [R"URUR5Ul[R"URUR5Ul[R"URUR5Ul[R"URUR5Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: ).F)rcrdryrg embed_dimr num_headshead_dimrscaleattention_dropoutrr is_causalr#rk_projv_projq_projout_projrxs r)rdAltCLIPAttention.__init__s  ++33$..8 ==4>> )T^^ ;MdnnM]^NN#2' ]]D( // ii? ii? ii?  $..$..A r+rrcausal_attention_maskrrc URupVnURU5nURU5n URU5n UR XVUR UR 5RSS5nU R XVUR UR 5RSS5n U R XVUR UR 5RSS5n URRS:waUbUbX#-nOUbUnO USLUl [n URRS:wa[URRn U "UUU U UURURUR(dSO URS9upU R!XVU5R#5n UR%U 5n U(dSn X4$)z#Input shape: Batch x Time x Channelrrflash_attention_2Nr)rOr=rr)rrRrPrQrrKrLrryrrOrDrrMr@rrreshaperrS)rBrrrUr batch_sizerrJqueriesrFvaluesattention_interfacerCrBs r)rAltCLIPAttention.forwards-:,?,?) ++m,{{=)]+,,zt~~t}}U__`acdeyyOYYZ[]^_ZT^^T]]S]]^_abc ;; + +/B B).C.O!/!G&2!62$>DN(? ;; + +w 6"9$++:Z:Z"[ $7     nnJJ#}}C$,, % ! "))*)LWWY mmK0  L((r+) ryrrrJrLrOrPrKrSrRrMrQr)rJrKrLrMrNrdr&rrrrErrQrrs@r)rFrFsGB.268<,1 /)||/)!./) ( 5 /) $D> /) u||Xell33 4 /)/)r+rFcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $) AltCLIPMLPi8c>[TU]5 Xl[URUl[ R"URUR5Ul [ R"URUR5Ul gr) rcrdryr r activation_fnr#rrgrfc1fc2rxs r)rdAltCLIPMLP.__init__9sb  #F$5$5699V//1I1IJ99V55v7I7IJr+rrclURU5nURU5nURU5nU$r)rcrbrdrs r)rAltCLIPMLP.forward@s4/ **=9 / r+)rbryrcrdrrs@r)r`r`8s)KU\\ellr+r`c ^\rSrSrS\4U4SjjrS S\RS\RS\RS\\ S\ \R4 S jjr S r U=r$) AltCLIPEncoderLayeriGryc<>[TU]5 URUl[ U5Ul[ R"URURS9Ul [U5Ul [ R"URURS9Ul gr) rcrdrgrJrF self_attnr#rnro layer_norm1r`mlp layer_norm2rxs r)rdAltCLIPEncoderLayer.__init__Hsm ++)&1<<F & u  ! &&r+ric^\rSrSrSrS\4U4Sjjr\S S\\ RS\\ RS\\ S\\ S \\ S \ \ \44 S jj5rS rU=r$)AltCLIPEncoderiyz Transformer encoder consisting of `config.num_hidden_layers` self attention layers. Each layer is a [`AltCLIPEncoderLayer`]. Args: config: AltCLIPConfig ryc>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snfr) rcrdryr#rrrrilayersr!)rBry_rzs r)rdAltCLIPEncoder.__init__sT  mm%PVPhPhJi$jJiQ%8%@Ji$jk &+#%kr$rrUrr%r&rcUbUOURRnUbUOURRnUbUOURRnU(aSOSnU(aSOSnUn [ UR 5H0upU(aXy4-nU "U UUUS9n U Sn U(dM(XS4-nM2 U(aXy4-n[ XUS9$)a Args: inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) causal_attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Causal mask for the text model. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NrI)rrrr()ryrr%use_return_dictr+rwr ) rBrrrUrr%r&encoder_statesall_attentionsridx encoder_layerr0s r)rAltCLIPEncoder.forwardsN2C1N-TXT_T_TqTq$8$D $++JjJj &1%>N+Vd  r+)ryr!rw)NNNNN)rJrKrLrMrNrrdrrr&rrrrEr rrQrrs@r)ruruys,}, 268<,0/3&*D !.D  ( 5 D $D> D 'tn D d^D  uo% &D D r+ruc^\rSrSrS\4U4SjjrS\RS\S\S\R4Sjr S S \RS\R4S jjr S r U=r $) AltCLIPVisionEmbeddingsiryc>[TU]5 XlURUlUR UlUR Ul[R"[R"UR55Ul [R"URURUR UR SS9UlUR UR -S-UlURS-Ul[R""UR UR5UlUR'S[R("UR 5R+S5SS9 g)NF) in_channels out_channels kernel_sizestridebiasrrr[r\r^)rcrdryrgrJ image_size patch_sizer# Parameterr&randnclass_embeddingConv2d num_channelspatch_embedding num_patches num_positionsreposition_embeddingrsr'rtrxs r)rd AltCLIPVisionEmbeddings.__init__s  ++ ++ ++!||EKK,GH!yy++??  !OOt>1D!--1"$,,t/A/A4>>"R ^U\\$:L:L-M-T-TU\-]jopr+rheightwidthrcURSS- nURRRS5nURSS- n[R R 5(d%XF:Xa X#:XaURUR5$USS2SS24nUSS2SS24nURSn X R-n X0R-n [US-5n URSXU 5nURSSSS5n[RRUX4SS S 9nURSSSS5RSSU 5n[R "Xx4SS 9$) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rrNr]g?rrbicubicF)rvmode align_cornersr)rrweightrr&jit is_tracingr[rrrYrr#r$ interpolatercat) rBrrrrrrclass_pos_embedpatch_pos_embedr new_height new_widthsqrt_num_positionss r)interpolate_pos_encoding0AltCLIPVisionEmbeddings.interpolate_pos_encodingsi!&&q)A- !44;;EEaH*003a7 yy##%%+*F6?**4+<+<= =,QU3,QU3r". __, &}c'9:)11!5G]`a)11!Q1=--33 ( 4 *11!Q1=BB1b#Nyy/;CCr+ pixel_valuesc ^URup4pVU(dJXPR:wdX`R:wa,[SUSUSURSURS3 5eURRR nURUR US95nURS5RSS5nURRUSS5n [R"X/SS 9n U(aXRXU5-n U $XRUR5-n U $) NzInput image size (*z) doesn't match model (rHrarrr]r)rrrrrrbrflattenrrrtr&rrrr[) rBrrrZrxrr target_dtype patch_embeds class_embedsrs r)rAltCLIPVisionEmbeddings.forwards$'3'9'9$ v'V-F%SbSbJb$VHAeW4KDOOK\\]^b^m^m]nnpq ++2288 ++LOO,O,OP #++A.88A> ++22:q"E YY ;C ##&C&CJX]&^^J$&=&=d>O>O&PPJr+) rryrJrrrrrrrs)rJrKrLrMrrdr&rrrrOrrQrrs@r)rrsjq2q,'D5<<'D'DUX'D]b]i]i'DRE$5$5Z_ZfZfr+rc2\rSrSr%\\S'SrSr/rSr Sr g)AltCLIPPreTrainedModeli$ryaltclipTc6 URRn[U[5(aURRn[R R URSURS-U-S9 [R R URRURRU-S9 [R R URRURRU-S9 g[U[5(Ga URRnURS-SURR-S--U-nURS-U-n[R R URRUS9 [R R UR RUS9 [R R UR"RUS9 [R R UR$RUS9 g[U[&5(aURRnURR(S-SURR-S--U-nSURR(-S-U-n[R R UR*RUS9 [R R UR,RUS9 g[U[.5(a[R R UR0RUR2S-URR-S9 SUR0l[R R UR6RUR8S-URR-S9 SUR6lg[U[R:5(aJUR<R>RA5 URR>RCS5 g[U[RD5(akURR>R SURRS9 UR<b%UR<R>RA5 gg[U[RF5(axURR>R SURRS9 URHb2URR>URHRA5 ggg) zInitialize the weightsrXrI)meanstd)rrTg?N)%ryinitializer_factorrrr#initnormal_rrJrrinitializer_rangerrFrrRrPrQrSr`rgrcrd AltCLIPModeltext_projectiontext_embed_dim_is_hf_initializedvisual_projectionvision_embed_dimrnrdatazero_fill_rrerV)rBr<factor in_proj_std out_proj_stdfc_stds r) _init_weights$AltCLIPPreTrainedModel._init_weights+s// f5 6 6[[33F GGOOF22&BRBRTXBX[aBaO b GGOOF2299v}}?^?^ag?gO h GGOOF55<<&--BaBadjBjO k  0 1 1[[33F!++T1q6==;Z;Z7Z_c6cdgmmK",,d2f..r+rIN) rJrKrLrMrrPbase_model_prefixsupports_gradient_checkpointing_no_split_modulerrQrIr+r)rr$s !&*#+?r+rc^\rSrSrS\4U4Sjjr\\S S\\ RS\\ S\\ S\\ S\\ S \ \ \44 S jj55rS rU=r$) AltCLIPVisionTransformeriYryc>[TU]5 XlURn[ U5Ul[ R"X!RS9Ul [U5Ul [ R"X!RS9Ul gr) rcrdryrgrrr#rnro pre_layrnormruencoderpost_layernorm)rBryrJrzs r)rd!AltCLIPVisionTransformer.__init__Zsd  && 1&9LL8M8MN%f-  ll9:O:OPr+rrr%r&rrcUbUOURRnUbUOURRnUbUOURRnUc [ S5eUR XS9nUR U5nURUUUSS9nUSnUSS2SSS24n URU 5n [UU URURS9$)Nz You have to specify pixel_values)rT)rrr%r&rr) pooler_outputrr*) ryrr%r{rrrrrr rr*) rBrrr%r&rrencoder_outputsr)r:s r)r AltCLIPVisionTransformer.forwardds2C1N-TXT_T_TqTq$8$D $++JjJj &1%$ 'tn $ d^ $ #+4. $  u00 1$ $ r+rc^\rSrSr%\\S'SrS\4U4SjjrS\R4Sjr \ S S\ \ RS\ \S\ \S \S \ \S\\\44 S jj5rS rU=r$)AltCLIPVisionModeliryrcd>[TU]U5 [U5UlUR 5 gr)rcrdr vision_model post_initrxs r)rdAltCLIPVisionModel.__init__s'  4V< r+rcBURRR$r)rrrrGs r)get_input_embeddings'AltCLIPVisionModel.get_input_embeddingss  ++;;;r+rr%rr&c^UbUOURRnURUUUUUS9$)aN Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, AltCLIPVisionModel >>> model = AltCLIPVisionModel.from_pretrained("BAAI/AltCLIP") >>> processor = AutoProcessor.from_pretrained("BAAI/AltCLIP") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor(images=image, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled CLS states ```rrr%rr&)ryr{r)rBrrr%rr&s r)rAltCLIPVisionModel.forwardsA:&1%$ 'tn $ #' $ d^ $  u00 1$ $ r+raE The model behaves as an encoder following the architecture described in *Attention is all you need*_ by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin. .. _*Attention is all you need*: https://huggingface.co/papers/1706.03762 ) custom_introcn^\rSrSr%\\S'SU4SjjrSrSrSr \ SS\ \ RS\ \ RS \ \ RS \ \ RS \ \ RS \ \ RS \ \S\ \S\ \S\\\ R\44Sjj5rSrU=r$)AltRobertaModeliryc>[TU]U5 Xl[U5Ul[ U5UlU(a [U5OSUlUR5 g)z^ add_pooling_layer (bool, *optional*, defaults to `True`): Whether to add a pooling layer N) rcrdryrSrrrr3poolerr)rBryadd_pooling_layerrzs r)rdAltRobertaModel.__init__sL   .v6(0 2C&v.  r+c.URR$rrrirGs r)r$AltRobertaModel.get_input_embeddingss...r+c$XRlgrrrBrs r)set_input_embeddings$AltRobertaModel.set_input_embeddingss */'r+cUR5H7up#URRURR U5 M9 g)z Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsrr r r)rBheads_to_pruner rs r) _prune_headsAltRobertaModel._prune_headss< +002LE LL  u % / / ; ;E B3r+rrr`r[rrrr%r&rc UbUOURRnUbUOURRnU bU OURRn UbUb [ S5eUb"UR X5 UR 5n O"UbUR 5SSn O [ S5eU upUb URO URn Uc[R"X4U S9nUcr[URS5(a3URRSS2SU 24nURX5nUnO$[R"U [RU S9nUR!X*5nUR#XPRR$5nURUUUUS9nUR'UUUUUSS 9nUS nUR(bUR)U5OSn[+UUUR,UR.S 9$) NzDYou cannot specify both input_ids and inputs_embeds at the same timer]z5You have to specify either input_ids or inputs_embedsr!r`r})rr[r`rT)rrrr%r&rr)ryrr%r{r%warn_if_padding_and_no_attention_maskrvr"r&onesrrr`rtrurwget_extended_attention_mask get_head_maskrrrr rr*)rBrrr`r[rrrr%r&rrZrr"rrextended_attention_maskembedding_outputrsequence_outputr:s r)rAltRobertaModel.forwards2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y Q#..*K  &',,.s3KTU U!, %.%:!! @T@T  !"ZZ*)A6RN  !t(899*.//*H*HKZK*X'3J3Q3QR\3i0!A!&[ SY!Z150P0PQ_0m&&y++2O2OP ??%)' +  ,, 2/!5 ' *!,8< 8O O4UY )-')77&11   r+)ryrrr)T NNNNNNNNN)rJrKrLrMrrPrdrrrrrr&rrrrEr rrQrrs@r)rrs  /0C-11515/3,004,0/3&*G ELL)G !.G !. G u||, G ELL) G   -G $D>G 'tnG d^G  uU\\"$PP QG G r+rc^\rSrSr%\\S'U4SjrS\R4Sjr S\RSS4Sjr SS \ \ S\R4U4S jjjr\\SS \ \R$S \ \R$S \ \R$S\ \R$S\ \R$S\ \R$S\ \S\ \S\ \S\\\44Sjj55rSrU=r$)AltCLIPTextModeli9ryc>[TU]U5 [USS9Ul[R "UR UR5Ul[R"UR URS9Ul UR5 g)NF)rrW) rcrdrrobertar#rrg project_dimtransformationrnropre_LNrrxs r)rdAltCLIPTextModel.__init__<se  &vG  ii(:(:F[TU]U5$r)rcresize_token_embeddings)rBrrzs r)r(AltCLIPTextModel.resize_token_embeddingsIsw.~>>r+rrr`r[rrrr&r%c UbUOURRnURUUUUUUUU SS9 n U Sn URU 5n UR U 5n U SS2S4n [ U U U R U RS9$)a Examples: ```python >>> from transformers import AutoProcessor, AltCLIPTextModel >>> model = AltCLIPTextModel.from_pretrained("BAAI/AltCLIP") >>> processor = AutoProcessor.from_pretrained("BAAI/AltCLIP") >>> texts = ["it's a cat", "it's a dog"] >>> inputs = processor(text=texts, padding=True, return_tensors="pt") >>> outputs = model(**inputs) >>> last_hidden_state = outputs.last_hidden_state >>> pooled_output = outputs.pooler_output # pooled CLS states ```NT) rrr`r[rrrr%r&rr)ryr{rrrrrr*)rBrrr`r[rrrr&r%rrprojection_staters r)rAltCLIPTextModel.forwardLs@&1%; d^; 'tn;  u== >; ; r+rcV^\rSrSr%\\S'S\4U4Sjjr\"5\SS\ RS\ \ RS\ \ RS\ \ RS\ R4 S jj55r \"5\SS \ RS \S\ R4S jj55r\SS\ \ R S \ \ RS\ \ RS\ \ R S\ \ RS \ \S\ \S\ \S \S\ \S\\\44Sjj5rSrU=r$)riryc>[TU]U5 [UR[5(d"[ S[ UR5S35e[UR[5(d"[ S[ UR5S35eURnURnURUl URUl URUl URUl[U5Ul[#U5Ul[&R("URURSS9Ul[&R("URURSS9Ul[&R."[0R2"UR4R655UlUR;5 g)NzRconfig.vision_config is expected to be of type AltCLIPVisionConfig but is of type .zNconfig.text_config is expected to be of type AltCLIPTextConfig but is of type F)r)rcrdr vision_configr TypeErrortype text_configrrprojection_dimrrrgrr text_modelrrr#rrrrr&tensorrylogit_scale_init_value logit_scaler)rBryrrrzs r)rdAltCLIPModel.__init__sk  &..0CDD--./q2 &,,.?@@++,-Q0  (( ,, -3-H-H *$33)55 - 9 9*;74]C!#4+@+@$BUBU\a!b!yy)<)Q>QX]^<< T[[5W5W(XY r+rrr[r`rcdURUUUUS9nURnURU5nU$)af Returns: text_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The text embeddings obtained by applying the projection layer to the pooled output of [`AltCLIPTextModel`]. Examples: ```python >>> import torch >>> from transformers import AutoProcessor, AltCLIPModel >>> model = AltCLIPModel.from_pretrained("BAAI/AltCLIP") >>> processor = AutoProcessor.from_pretrained("BAAI/AltCLIP") >>> inputs = processor(text=["a photo of a cat", "a photo of a dog"], padding=True, return_tensors="pt") >>> with torch.inference_mode(): ... text_features = model.get_text_features(**inputs) ```)rrr[r`)rrr)rBrrr[r` text_outputsr: text_featuress r)get_text_featuresAltCLIPModel.get_text_featuressF6)%) ' %22 ,,]; r+rrc`URUUS9nURnURU5nU$)a Returns: image_features (`torch.FloatTensor` of shape `(batch_size, output_dim`): The image embeddings obtained by applying the projection layer to the pooled output of [`AltCLIPVisionModel`]. Examples: ```python >>> import torch >>> from transformers import AutoProcessor, AltCLIPModel >>> from transformers.image_utils import load_image >>> model = AltCLIPModel.from_pretrained("BAAI/AltCLIP") >>> processor = AutoProcessor.from_pretrained("BAAI/AltCLIP") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = load_image(url) >>> inputs = processor(images=image, return_tensors="pt") >>> with torch.inference_mode(): ... image_features = model.get_image_features(**inputs) ```)rr)rrr)rBrrvision_outputsr:image_featuress r)get_image_featuresAltCLIPModel.get_image_featuressB:**%%=+ '44 // >r+ return_lossrr%r&c UbUOURRnUbUOURRnU bU OURRn UR UUUUUUU S9n UR UUUU U S9n U Sn UR U 5n U SnURU5nXRSSSS9- n XRSSSS9- nURR5n[R"XR55U-nURnSnU(a [U5nU (dUUXX4nUbU4U-$U$[!UUUUU U U S 9$) a0 return_loss (`bool`, *optional*): Whether or not to return the contrastive loss. Examples: ```python >>> from PIL import Image >>> import requests >>> from transformers import AutoProcessor, AltCLIPModel >>> model = AltCLIPModel.from_pretrained("BAAI/AltCLIP") >>> processor = AutoProcessor.from_pretrained("BAAI/AltCLIP") >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg" >>> image = Image.open(requests.get(url, stream=True).raw) >>> inputs = processor( ... text=["a photo of a cat", "a photo of a dog"], images=image, return_tensors="pt", padding=True ... ) >>> outputs = model(**inputs) >>> logits_per_image = outputs.logits_per_image # this is the image-text similarity score >>> probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities ```N)rrr`r[rr%r&rrrr]T)r?rkeepdim)r5r6r7r8r9r:r;)ryrr%r{rrrrnormr!expr&rr.Tr1r3)rBrrrr[r`r-rr%rr&r$r)r9r8r!r7r6r5rs r)rAltCLIPModel.forwardsJ2C1N-TXT_T_TqTq$8$D $++JjJj &1%[ 'tn[ #'[ d^[  um# $[ [ r+rcURU5R5n[R"USS9R U5U-U-nUR 5U-$)z Replace non-padding symbols with their position numbers. Position numbers begin at padding_idx+1. Padding symbols are ignored. This is modified from fairseq's `utils.make_positions`. Args: x: torch.Tensor x: Returns: torch.Tensor rr)nerr&cumsumtype_asrw)rrVrmaskincremental_indicess r)r~r~\sW << $ ( ( *D <<!4<rIs !11 !9 GllwwXX   H % `U\\`ell`-%,,-5<<-    K     HV=299V=rSbiiSn 299  $&" *"))*\ RYY  ryy %0%R.  . d ryy .% II% <<% % << % U\\* %  %%.F)ryyF)T  /4/dT RYYT pPbiiPf1?_1?1?h1 ryy1 h2 /2 jk ,k k \P -P fL )L ` 4 _r+