cCiFSSKrSSKJr SSKJrJrJr SSKrSSK J r SSK J r SSK Jr SSKJr SSKJr SS KJr SS KJrJrJr SS KJrJr SS KJr SS KJrJ r J!r!J"r"J#r# SSK$J%r% SSK&J'r' SSK(J)r)J*r* \"S5"SS\ RV55r,SDS\ RVS\RZS\RZS\RZS\\RZS\.S\.4Sjjr/"SS\ RV5r0\\!"S S!9"S"S#\555r1"S$S%\ RV5r2"S&S'\ RV5r3"S(S)\ RV5r4\ Rj\,S*.r6"S+S,\5r7"S-S.\ RV5r8\!"S/S0\55r9\!"S1S2\955r:\!"S3S4\55r;"S5S6\ RV5r<\\!"S7S!9"S8S9\555r=\!"S:S!9"S;S<\;55r>\\!"S=S!9"S>S?\555r?\!"S@S!9"SASB\;\55r@/SCQrAg)EN) dataclass)CallableOptionalUnion)ACT2FN)Cache)GenerationMixin)use_kernel_forward_from_hub)GradientCheckpointingLayer)BaseModelOutputBaseModelOutputWithPastBaseModelOutputWithPooling)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) ModelOutputTransformersKwargsauto_docstringcan_return_tuple torch_int)check_model_inputs) AutoModel)InternVLConfigInternVLVisionConfigRMSNormc8^\rSrSrSU4SjjrSrSrSrU=r$)InternVLVisionRMSNorm,c>[TU]5 [R"[R "U55UlX lg)z4 InternVLVisionRMSNorm is equivalent to T5LayerNorm N)super__init__nn Parametertorchonesweightvariance_epsilon)self hidden_sizeeps __class__s h/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/internvl/modeling_internvl.pyr$InternVLVisionRMSNorm.__init__.s/ ll5::k#:; #cURnUR[R5nUR S5R SSS9nU[R "X0R-5-nURURU5-$)NrT)keepdim) dtypetor'float32powmeanrsqrtr*r))r+ hidden_states input_dtypevariances r/forwardInternVLVisionRMSNorm.forward6sw#)) %((7  $$Q',,R,>% H?T?T4T(UU {{]--k:::r1c^[URR5SUR3$)Nz, eps=)tupler)shaper*r+s r/ extra_repr InternVLVisionRMSNorm.extra_repr=s* ))*+6$2G2G1HIIr1)r*r))gư>) __name__ __module__ __qualname____firstlineno__r$r>rD__static_attributes__ __classcell__r.s@r/r r ,s$;JJr1r modulequerykeyvalueattention_maskscalingdropoutc UnUn [R"XRSS55U-n Ub"USS2SS2SS2SURS24n X-n [R R U SS9n [R RXURS9n [R"X5n U RSS5R5n X4$)Nrrr3dim)ptrainingr) r'matmul transposerBr% functionalsoftmaxrSrY contiguous) rMrNrOrPrQrRrSkwargs key_states value_states attn_weights causal_mask attn_outputs r/eager_attention_forwardreAsJL<<';';Aq'ABWLL!$Q1.D 0@0@0D.D%DE #1 ==((2(>L==((6??([L,,|:K''1-88:K  $$r1c^\rSrSrSrS\4U4SjjrS S\RS\ \RS\ \ 4Sjjr S r U=r$) InternVLVisionAttention\z+Attention Class for InternVL Vision Encoderconfigc$>[TU]5 XlURUlUR UlURUR -UlURUR -UR:wa&[SURSUR S35eURS-Ul URUl URnURnSUl [R"URUR UR-UR S9Ul[R"URUR UR-UR S9Ul[R"URUR UR-UR S9Ul[R"URUR5UlUS:a[R*"U5O[R,"5Ul U(a[/UR5O[R,"5UlU(a[/UR5Ulg[R,"5Ulg)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).gFbiasr)r#r$rir, embed_dimnum_attention_heads num_headshead_dim ValueErrorscaleattention_dropoutprojection_dropout use_qk_norm is_causalr%Linearattention_biasq_projk_projv_projprojection_layerDropoutIdentityr q_normk_norm)r+ri proj_dropoutqk_normr.s r/r$ InternVLVisionAttention.__init___s  ++33$..8 ==4>> )T^^ ;MdnnM]^NN#2' ]]D( !'!9!900 $$ii0NU[UjUjk ii0NU[UjUjk ii0NU[UjUjk " $..$.. I>JQ>N"**\":TVT_T_Ta?F+DNN;BKKM ?F+DNN; BKKM r1r;rQr_c UR5upEnURU5nURU5nURU5n UR U5nUR U5nUR XEURUR5RSS5nUR XEURUR5RSS5nU RXEURUR5RSS5n [n URRS:wa[URRn U "UUUU U4UR(dSO UR UR"SS.UD6upU R XEUR$5n UR'U 5n UR)U 5n X4$)NrreagerF)rSrRrv)sizeryrzr{rrreshaperorpr[viewreri_attn_implementationrrYrsrrrmr|rt)r+r;rQr_ batch_sizeseq_len_ query_statesr`raattention_interfacerdrboutputs r/r>InternVLVisionAttention.forward{s "/!3!3!5 Q{{=1 [[/ {{=1 {{<0 [[, #++JQUQ^Q^_iijkmno '' T^^T]][eefgijk #((dnndmm\ffghjkl (? ;; + +w 6"9$++:Z:Z"[ $7      % $}}C$2H2HJJ %  % ! "))*t~~N &&{3((0##r1)rsrirmrprvrrzrortr|rryrrr{N)rFrGrHrI__doc__rr$r'Tensorrrrr>rJrKrLs@r/rgrg\sS5Z3Z>26'$||'$!.'$+, '$'$r1rgz7 Class for outputs of [`InternVLVisionModel`]. ) custom_introc\rSrSrSrSrg)$InternVLVisionModelOutputWithPoolinga2 pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`): Average of the last layer hidden states of the patch tokens (excluding the *[CLS]* token) if *config.use_mean_pooling* is set to True. If set to False, then the final hidden state of the *[CLS]* token will be returned. N)rFrGrHrIrrJrr1r/rrsr1rcf^\rSrSrSrU4SjrS\RS\R4SjrSr U=r $)InternVLVisionPatchEmbeddingsz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) of shape `(batch_size, seq_length, hidden_size)` to be consumed by a Transformer. cH>[TU]5 URURp2URUR pTUSUS-USUS--nUSUS-USUS-4nX lX0lX@lX`lXpl[R"XEX3S9Ul g)Nrr) kernel_sizestride) r#r$ image_size patch_size num_channelsr, num_patches patch_shaper%Conv2d projection) r+rirrrr,rrr.s r/r$&InternVLVisionPatchEmbeddings.__init__s !'!2!2F4E4EJ$*$7$79K9Kk!!} 1 5*Q-:VW=:XY !!} 1 5z!} ST 7UV $$(&&))L:ir1 pixel_valuesreturncURup#pEX0R:wa [S5eURU5nURSURSpUR S5R SS5nXgU44$)NzeMake sure that the channel dimension of the pixel values match with the one set in the configuration.rrr)rBrrqrflattenr[) r+rrrheightwidth embeddings patch_height patch_widths r/r>%InternVLVisionPatchEmbeddings.forwards2>2D2D/ & ,, ,w __\2 $.$4$4Q$79I9I!9Lk''*44Q: +666r1)rrrrrr) rFrGrHrIrr$r'rr>rJrKrLs@r/rrs. j 7ELL 7U\\ 7 7r1rc^\rSrSrSrS\SS4U4SjjrS\RS\ S \ S\R4S jr SS \RS \ \RS\R4S jjr SrU=r$)InternVLVisionEmbeddingsz[ Construct the CLS token, position and patch embeddings. Optionally, also the mask token. rirNc^>[TU]5 [R"[R "SSUR 55UlUR(a<[R"[R "SSUR 55Ul OSUl [U5Ul URUl [UR[R R"5(a UROURUR4UlURR$nUR&(a?[R"[R "SUS-UR 55UlOSUl[R*"UR,5Ulg)Nr)r#r$r%r&r'zerosr, cls_tokenuse_mask_token mask_tokenrpatch_embeddingsr isinstancer collectionsabcIterabler use_absolute_position_embeddingsposition_embeddingsr}hidden_dropout_probrS)r+rirr.s r/r$!InternVLVisionEmbeddings.__init__s' ekk!Q8J8J&KL   ll5;;q!V=O=O+PQDO"DO =f E ++&++[__-E-EFF   ##V%6%67  ++77  2 2')||EKK;QR?TZTfTf4g'hD $'+D $zz&"<"<= r1rrrcURSS- nURRSS- n[RR 5(dXE:XaX#:Xa UR$URSS2SS24nURSS2SS24nURSnX R S-n X0R S-n [ US-5n URSXU5nURSSSS5n[RRUX4SS S 9nURSSSS5RSSU5n[R"Xg4SS 9$) a This method allows to interpolate the pre-trained position encodings, to be able to use the model on higher resolution images. This method is also adapted to support torch.jit tracing. Adapted from: - https://github.com/facebookresearch/dino/blob/de9ee3df6cf39fac952ab558447af1fa1365362a/vision_transformer.py#L174-L194, and - https://github.com/facebookresearch/dinov2/blob/e1277af2ba9496fbadf7aec6eba56e8d882d1e35/dinov2/models/vision_transformer.py#L179-L211 rNr3r?rrbicubicF)rmode align_cornersrV)rBrr'jit is_tracingrrrpermuter%r\ interpolatercat) r+rrrr num_positionsclass_pos_embedpatch_pos_embedrW new_height new_widthsqrt_num_positionss r/interpolate_pos_encoding1InternVLVisionEmbeddings.interpolate_pos_encodings]!&&q)A- 0066q9A= yy##%%+*F6?++ +221bqb59221ab59r"q11 __Q// &}c'9:)11!5G]`a)11!Q1=--33 ( 4 *11!Q1=BB1b#Nyy/;CCr1rbool_masked_poscURu p4nURU5unupxUR5upnUbIURR XS5n UR S5R U 5n USU - -X--nURR U SS5n [R"X4SS9nURbX`RXdU5-nURU5nXgU44$)Nr3rrV) rBrrrexpand unsqueezetype_asrr'rrrrS)r+rrrrrrrrrr mask_tokensw cls_tokenss r/r> InternVLVisionEmbeddings.forwards +001e262G2G 2U/ /\!+!2 Q  &//00bIK))"-55kBA#q1u- ?J^^**:r2> YY 7Q?  # # /#&C&CJX]&^^J\\*- +666r1)rrSrrrrrr)rFrGrHrIrrr$r'rintrr BoolTensorr>rJrKrLs@r/rrs >3>>,&D5<<&D&DUX&D]b]i]i&DV7;7ll7"%"2"237  77r1rcb^\rSrSrU4SjrS\R S\R 4SjrSrU=r $)InternVLVisionMLPi8c>[TU]5 Xl[URUl[ R"URUR5Ul [ R"URUR5Ul gr) r#r$rir hidden_act activation_fnr%rwr,intermediate_sizefc1fc2r+rir.s r/r$InternVLVisionMLP.__init__9sb  #F$5$5699V//1I1IJ99V55v7I7IJr1r;rclURU5nURU5nURU5nU$r)rrr)r+r;s r/r>InternVLVisionMLP.forward@s4/ **=9 / r1)rrirr) rFrGrHrIr$r'rr>rJrKrLs@r/rr8s)KU\\ellr1r) layer_normrms_normc^\rSrSrSrS\SS4U4SjjrS\RS\ \ \R\ \R\R444Sjr S r U=r $) InternVLVisionLayeriJz?This corresponds to the Block class in the timm implementation.rirNc>[TU]5 URUlSUl[ U5Ul[ U5Ul[UR"URURS9Ul [UR"URURS9Ul URn[R "U["R$"UR5-SS9Ul[R "U["R$"UR5-SS9Ul[R*"UR,5Ulg)Nrr-T) requires_grad)r#r$chunk_size_feed_forward seq_len_dimrg attentionrmlpNORM2FN norm_typer,layer_norm_epslayernorm_beforelayernorm_afterlayer_scale_init_valuer%r&r'r(lambda_1lambda_2r}rrS)r+ri init_valuesr.s r/r$InternVLVisionLayer.__init__Ms '-'E'E$08$V, '(8(8 9&:L:LRXRgRg h&v'7'789K9KQWQfQfg33  [5::f>P>P3Q%Qaef  [5::f>P>P3Q%Qaef zz&"<"<= r1r;cURURU55up#URU-nX!-nURU5nUR U5nUR U5nUR bUR U-nXA-nU$r)rrrrrrSr)r+r;attention_outputr layer_outputs r/r>InternVLVisionLayer.forward\s#nn  ! !- 0  ==+;;)8 ++M: xx - ||L1 == $==<7L$3 r1) rrrSrrrrrr)rFrGrHrIrrr$r'rrrAr>rJrKrLs@r/rrJs_I >3 > >|| uU\\"E%,, *D$EE Fr1rcd^\rSrSrS\SS4U4SjjrS\RS\\ \ 44Sjr Sr U=r $) InternVLVisionEncoderixrirNc>[TU]5 Xl[R"[ UR 5Vs/sHn[U5PM sn5UlSUl gs snf)NF) r#r$rir% ModuleListrangenum_hidden_layersrlayergradient_checkpointing)r+riir.s r/r$InternVLVisionEncoder.__init__ysS  ]]vOgOgIh#iIhA$7$?Ih#ij &+#$jsA&r;cJURH nU"U5nM [US9$)N)last_hidden_state)r r )r+r; layer_modules r/r>InternVLVisionEncoder.forwards.!JJL(7M'+  r1)rir r )rFrGrHrIrr$r'rrrAr r>rJrKrLs@r/rrxsA,3,,  ||   uo% &   r1rc`^\rSrSr%\\S'SrSrSrS/r Sr Sr Sr Sr \\S.rU4SjrS rU=r$) InternVLVisionPreTrainedModeliriinternvl_visionrTr)r; attentionscn>[TU]U5 [U[5(aURR R 5 URb$URR R 5 URb%URR R 5 gg[U[5(asURR RURR5 URR RURR5 gg)zInitialize the weightsN)r# _init_weightsrrrdatazero_rrrrfill_rirr)r+rMr.s r/r+InternVLVisionPreTrainedModel._init_weightss f% f6 7 7    ! ! ' ' )  ,!!&&,,.))5**//5576  3 4 4 OO & &t{{'I'I J OO & &t{{'I'I J5r1r)rFrGrHrIr__annotations__base_model_prefixmain_input_namesupports_gradient_checkpointing_no_split_modules_supports_sdpa_supports_flash_attn_supports_flex_attn_supports_attention_backendrrg_can_record_outputsrrJrKrLs@r/rrsV  )$O&*#./N"&-- K Kr1rc ^\rSrSrS\SS4U4SjjrSr\"SS9\S S \ RS \ \ RS\ \\44S jj55rS rU=r$)InternVLVisionModelirirNc8>[TU]U5 Xl[U5Ul[ U5UlUR(a[R"5O([R"URURS9Ul UR5 g)Nr)r#r$rirrrencoderuse_mean_poolingr%r~ LayerNormr,r layernorm post_initrs r/r$InternVLVisionModel.__init__sm   26:,V4 $44BKKM",,vGYGY_e_t_t:u  r1c.URR$r)rrrCs r/get_input_embeddings(InternVLVisionModel.get_input_embeddingss///r1F)tie_last_hidden_statesrrcURXS9up4URU5nUSnURU5n[UURUR S9$)z bool_masked_pos (`torch.BoolTensor` of shape `(batch_size, num_patches)`, *optional*): Boolean masked positions. Indicates which patches are masked (1) and which aren't (0). )rr)rr;r)rr*r-rr;r)r+rrembedding_outputrencoder_outputssequence_outputs r/r>InternVLVisionModel.forwardsa#oolo\,,'78)!,..93-)77&11  r1)rirr*r-r)rFrGrHrIrr$r1rrr'rrrrrArr>rJrKrLs@r/r(r(sx 3  0u57; ll "%"2"23  u:: ;  6 r1r(c@\rSrSr%\\S'SrSrSrSr Sr Sr Sr Sr Srg)InternVLPreTrainedModeliriTpast_key_valuesrN)rFrGrHrIrrrr _skip_keys_device_placementr#r"_can_compile_fullgraphr$r%rJrr1r/r:r:s7 &*#"3N!"&r1r:c6^\rSrSrS\4U4SjjrSrSrU=r$)InternVLMultiModalProjectoriric0>[TU]5 [R"URR [ SUR- 5S--5Ul[R"URR [ SUR- 5S--URR 5Ul [URUl[R"URR URR 5Ulg)Nrr)r#r$r%r, vision_configr,rdownsample_ratiorrw text_configlinear_1rprojector_hidden_actactlinear_2rs r/r$$InternVLMultiModalProjector.__init__s ,,v';';'G'G#aRXRiRiNiJjnoJo'op  , ,s1v7N7N3N/OST/T TV\VhVhVtVt  &556 &"4"4"@"@&BTBTB`B`a r1cURU5nURU5nURU5nURU5nU$r)rrErGrH)r+image_featuresr;s r/r>#InternVLMultiModalProjector.forwards@7  m4 /  m4 r1)rGrrErH) rFrGrHrIrr$r>rJrKrLs@r/r@r@sb~br1r@zM Base class for InternVL outputs, with hidden states and attentions. cB\rSrSr%SrSr\\R\ S'Sr g)InternVLModelOutputWithPastia past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nimage_hidden_statesr) rFrGrHrIrrOrr' FloatTensorrrJrr1r/rNrNs 8<%"3"34;r1rNzx The InternVL model which consists of a vision backbone and a language model, without a language modeling head. cH^\rSrSrSS0rS\4U4SjjrSrSrSr S r SS \ RS \ \\\\4S \ \4S jjrS\ R&S\ RS\ R4Sjr\\SS\ \ R&S \ \ RS\ \ R.S\ \ R&S\ \S\ \ RS \ \\\\4S \ \S\ \ R&S\\S\\\44Sjj55rSS\ R.S\4SjjrSr U=r!$) InternVLModeli zlanguage_model.modellanguage_modelric>[TU]U5 [R"UR5Ul[ U5Ul[R"UR5Ul UR5 gr) r#r$r from_configrB vision_towerr@multi_modal_projectorrDrSr.rs r/r$InternVLModel.__init__sY  %11&2F2FG%@%H"'33F4F4FG r1c6URR5$r)rSr1rCs r/r1"InternVLModel.get_input_embeddingss""7799r1c:URRU5 gr)rSset_input_embeddingsr+rPs r/r\"InternVLModel.set_input_embeddingss 007r1cXlgrrSr+decoders r/ set_decoderInternVLModel.set_decoder!s%r1cUR$rr`rCs r/ get_decoderInternVLModel.get_decoder$s"""r1rvision_feature_layervision_feature_select_strategyc nUbUOURRnUbUOURRnURURS9nURR nUS:XaUR US9RnOURUS9RUnUS:XaUSS2SS2SS24nURSn[US-5nURSn URXUS5nURXeS 9nURU SURS5nURU5nU$) a Obtains image last hidden states from the vision tower and apply multimodal projection. Args: pixel_values (`torch.FloatTensor]` of shape `(batch_size, channels, height, width)`) The tensors corresponding to the input images. vision_feature_layer (`int` or `list[int]`): Layer index or list of layer indices to extract features from. Returns: vision_features (`torch.Tensor`): Image feature tensor of shape `(num_images, image_length, embed_dim)`. N)r5r3)rdefaultrrr) scale_factor)rirhrir6r5rCrVr vision_modelr;rBrr pixel_shufflerW) r+rrhrir_rCvision_featureschannels feature_sizers r/get_image_features InternVLModel.get_image_features'sS&%9$D $++JjJj  .9 +;; ' $TZZ8 ;;77 2 %"//\/J\\O"//\/JXXYmnO )Y 6-aQh7O#((+8S=) $**1- *11*LZ\],,_,\*11*b/BWBWXZB[\44_Er1 input_ids inputs_embedsrKc JUcjX R5"[R"URR[R UR S95:HnURS5nOXRR:HnUR5nURS5RU5RUR 5nURSURS-nX$R5UR5:wa[SUSU35eU$)z Obtains multimodal placeholder mask from `input_ids` or `inputs_embeds`, and checks that the placeholder token count is equal to the length of multimodal features. If the lengths are different, an error is raised. )r5devicer3rrz6Image features and image tokens do not match: tokens: z , features )r1r'tensorriimage_token_idlongrwallsumr expand_asr6rBnumelrq)r+rtrurKspecial_image_maskn_image_tokensn_image_featuress r/get_placeholder_mask"InternVLModel.get_placeholder_mask]s  !.2K2K2M T[[77uzzR_RfRfg3" "4!7!7!; !*kk.H.H!H +//1/99"=GG VYYZgZnZno)//2^5I5I!5LL  , 2 2 48L8L8N NHHXXcdtcuv "!r1rQ position_idsr<cache_positionr_rc 0UbUOURRnUbUOURRnUSLUSL- (a [S5eUcUR 5"U5nUbXUR UUUS9n U R URUR5n URXU S9n URX5nUR"SUUUUU S.U D6n [U RU RU RU R UbW S9$SS9$)Nz:You must specify exactly one of input_ids or inputs_embedsrrhri)rurK)rQrr<rur)rr<r;rrOr)rirhrirqr1rrr6rwr5rmasked_scatterrSrNrr<r;r)r+rtrrQrr<rurhrirr_rKroutputss r/r>InternVLModel.forwardusb %9$D $++JjJj  .9 +;; '  -t"; <YZ Z   557 BM  #!44)%9/M5N ,..}/C/C]EXEXYN!%!:!:~";" *889K\M%% )%+')    +%77#33!//))2>2J   QU   r1rorlc UR5up4pVXR-S:wdXB-S:wa [S5eURX4[XR-5[Xb- 55nUR SSSS5R 5nURU[XR-5[XB-5[XbS-- 55nUR SSSS5R 5nU$)aPerform pixel shuffle downsampling on vision features. Args: vision_features (`torch.Tensor`): Input tensor of shape (batch_size, width, height, channels). scale_factor (`float`, *optional*, defaults to `0.5`): Factor by which to downsample. Default is 0.5, which halves the dimensions. Returns: vision_features (`torch.Tensor`): Downsampled tensor of shape (batch_size, height*scale_factor, width*scale_factor, channels/(scale_factor^2)). rzKHeight and width must be divisible by scale_factor for proper downsampling.rrr)rrqrrrr^)r+rorlrrrrps r/rnInternVLModel.pixel_shuffles/>.B.B.D+ 6  A %)=)Bjk k*.. s6#893x?V;W *11!Q1=HHJ*.. F12C8L4MsS[mn_nSoOp  *11!Q1=HHJr1)rSrWrVNN) NNNNNNNNN)r)"rFrGrHrI_checkpoint_conversion_mappingrr$r1r\rcrfr'rPrrrliststrrr LongTensorrrrrr rrrArNr>floatrnrJrKrLs@r/rRrR s '=>N%O"~:8&# AE8< 4''4'uS$s)^'<=4)1 4l"))":?:K:K"]b]n]n"015481537+/59@D8<597 E,,-7 u0017 !. 7 u//0 7 "% 7   1 127 'uS$s)^'<=7 )1 7 !!1!127 +,7  u11 27 7 r!U\\!!!r1rRzT Base class for InternVL causal language model (or autoregressive) outputs. c\rSrSr%SrSr\\R\ S'Sr \\R\ S'Sr \\ \ S'Sr \\\R\ S'Sr\\\R\ S'Sr\\R\ S 'S rg) InternVLCausalLMOutputWithPastia loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided): Language modeling loss (for next-token prediction). logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`): Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax). past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache). Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. image_hidden_states (`torch.FloatTensor`, *optional*): A `torch.FloatTensor` of size `(batch_size, num_images, sequence_length, hidden_size)`. image_hidden_states of the model produced by the vision encoder and after projecting the last hidden state. Nlosslogitsr<r;rrOr)rFrGrHrIrrrr'rPrrr<r r;rArrOrJrr1r/rrs )-D(5$$ %,*.FHU&& '.'+OXe_+8[TU]U5 [U5Ul[R "UR RUR RSS9Ul UR5 g)NFrk) r#r$rRmodelr%rwrDr, vocab_sizerr.rs r/r$)InternVLForConditionalGeneration.__init__sS  "6* yy!3!3!?!?ASASA^A^ejk  r1c6URR5$r)rr1rCs r/r15InternVLForConditionalGeneration.get_input_embeddingsszz..00r1c:URRU5 gr)rr\r]s r/r\5InternVLForConditionalGeneration.set_input_embeddings s ''.r1rcUR$r)rrCs r/get_output_embeddings6InternVLForConditionalGeneration.get_output_embeddings s ||r1c:URRU5 gr)rrcras r/rc,InternVLForConditionalGeneration.set_decoders w'r1c6URR5$r)rrfrCs r/rf,InternVLForConditionalGeneration.get_decoderszz%%''r1rrhric BURR"SUUUS.UD6$)Nrr)rrr)r+rrhrir_s r/rr3InternVLForConditionalGeneration.get_image_featuress3zz,, %!5+I    r1c.URR$r)rrSrCs r/rS/InternVLForConditionalGeneration.language_model$szz(((r1c.URR$r)rrVrCs r/rV-InternVLForConditionalGeneration.vision_tower(szz&&&r1c.URR$r)rrWrCs r/rW6InternVLForConditionalGeneration.multi_modal_projector,szz///r1rtrQrr<rulabelsrlogits_to_keep image_sizesr_c  UbUOURRnUbUOURRnUR"SUUUUUUUUU U S. U D6nUSn[ U [ 5(a [ U *S5OU nURUSS2USS245nSnU b3UR"SUXRRRS.U D6n[UUURURURURS9$)as Example: ```python >>> import torch >>> from transformers import AutoProcessor, AutoModelForImageTextToText >>> torch_device = "cuda" >>> processor = AutoProcessor.from_pretrained("OpenGVLab/InternVL3-1B-hf") >>> model = AutoModelForImageTextToText.from_pretrained( ... "OpenGVLab/InternVL3-1B-hf", dtype=torch.bfloat16, device_map=torch_device ... ) >>> messages = [ ... { ... "role": "user", ... "content": [ ... { ... "type": "image", ... "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg", ... }, ... { ... "type": "image", ... "url": "https://thumbs.dreamstime.com/b/golden-gate-bridge-san-francisco-purple-flowers-california-echium-candicans-36805947.jpg", ... }, ... {"type": "text", "text": "These images depict two different landmarks. Can you identify them?"}, ... ], ... }, ... ] >>> inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt").to(torch_device) >>> generate_ids = model.generate(**inputs, max_new_tokens=200) >>> print(processor.decode(generate_ids[0, inputs["input_ids"].shape[1] :], skip_special_tokens=True)) The images depict the Statue of Liberty and the Golden Gate Bridge. ```N) rtrrQrr<rurhrirrr)rrr)rrr<r;rrOr)rirhrirrrslicer loss_functionrDrrr<r;rrO)r+rtrrQrr<rurhrirrrrr_rr; slice_indicesrrs r/r>(InternVLForConditionalGeneration.forward0s4l%9$D $++JjJj  .9 +;; ' **  %)%+'!5+I)#      8B>SV8W8W~ot4]k mA}a,?@A  %%f9P9P9[9[_eD.#33!//)) ' ; ;   r1c P>[T U]"U4UUUUUS.UD6n USS:XaXIS'U $)N)r<rurQrrrr)r#prepare_inputs_for_generation) r+rtr<rurrQrrr_ model_inputsr.s r/r>InternVLForConditionalGeneration.prepare_inputs_for_generationsTw<  +')))    !  !,8 (r1)rrr) NNNNNNNNNNrN)NNNNNN)(rFrGrHrIr_tied_weights_keysrr$r1r\r%Modulerrcrfr'rPrrrrrrrpropertyrSrVrWrrrrr rrrArr>rrJrKrLs@r/rrs_"8-"?#, &" ++~ 1/ryy(( AE8<  ''  'uS$s)^'<=  )1  ))''0015481537+/59@D8<-15934.2\ E,,-\ u001\ !. \ u//0 \ "% \   1 12\ 'uS$s)^'<=\ )1 \ ))*\ !!1!12\ c5<</0\ ell+\ +,\  u44 5\ \ Br1r)rr(r:rRr)r)Bcollections.abcr dataclassesrtypingrrrr'torch.nnr% activationsr cache_utilsr generationr integrationsr modeling_layersr modeling_outputsr rrmodeling_utilsrrprocessing_utilsrutilsrrrrr utils.genericrautorconfiguration_internvlrrrr rrrergrrrrr,rrrrr(r:r@rNrRrr__all__rr1r/rs.!,, ! )79ddF&aa/HY'JBIIJ(J6% II% <<% % << % U\\* %  %%6F$biiF$R  +E  !7BII!7L[7ryy[7|  3H I+4+\ BII &KOKK<' 7' ' T 'o ' '"))$  <"9 <  < A+A  AH  <[<  <0 u'>u  up r1