cCi{VSrSSKrSSKJrJr SSKrSSKJrJ r J r SSK J r J r Jr SSKJrJrJrJrJrJrJrJrJrJr SSKJrJrJrJrJrJ r J!r! \"5(aSSK"r"\ RF"\$5r%S \\&\&\\&\\4S \&\&\4S jr'"S S \ 5r("SS\5r)S/r*g)zImage processor class for Fuyu.N)OptionalUnion)BaseImageProcessor BatchFeature get_size_dict)padresizeto_channel_dimension_format) ChannelDimension ImageInputPILImageResamplingget_image_sizeinfer_channel_dimension_formatis_scaled_imageis_valid_imagemake_list_of_imagesto_numpy_arrayvalidate_preprocess_arguments) TensorTypefilter_out_non_signature_kwargsis_torch_availableis_torch_deviceis_torch_dtypeloggingrequires_backendsimagesreturnc[U5(aU//$[U[5(a[SU55(aU$[U[5(aUVs/sHn[ U5PM sn$[ S5es snf)Nc3B# UHn[U[5v M g7fN) isinstancelist).0images h/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/fuyu/image_processing_fuyu.py .make_list_of_list_of_images..@s'TVE 5$(?(?VszHimages must be a list of list of images or a list of images or an image.)rr"r#allr ValueError)rr%s r&make_list_of_list_of_imagesr+:sufz&$C'TV'T$T$T &$8>?u#E*?? _ ``@sA?cD\rSrSrSrSS\\\\44Sjjr S Sjr Sr g) FuyuBatchFeatureIz BatchFeature class for Fuyu image processor and processor. The outputs dictionary from the processors contains a mix of tensors and lists of tensors. N tensor_typec ^^^^ UcU$URUS9ummUU4SjmUU 4SjnUR5Hum n[U[5(aG[US[5(a/UVVs/sHoDVs/sH oR"U5PM snPM snnUT 'Mb[U[5(aUVs/sH oR"U5PM snUT 'MU"U5UT 'M U$s snfs snnfs snf)a Convert the inner content to tensors. Args: tensor_type (`str` or [`~utils.TensorType`], *optional*): The type of tensors to use. If `str`, should be one of the values of the enum [`~utils.TensorType`]. If `None`, no modification is done. )r/c2>T"U5(aU$T"U5$r!)elem as_tensor is_tensors r&_convert_tensor._convert_tensor^s T? "cX>T"U5$! TS:Xa [S5e[S5e=f)Noverflowing_valueszKUnable to create tensor returning overflowing values of different lengths. zUnable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length.)r*)r3r6keys r&_safe_convert_tensorAFuyuBatchFeature.convert_to_tensors.._safe_convert_tensorcsB &t,, ..$%rss Xs )r)_get_is_as_tensor_fnsitemsr"r#) selfr/r<valueelemsr3r6r4r5r;s @@@@r&convert_to_tensors#FuyuBatchFeature.convert_to_tensorsPs  K#99k9R 9 # **,JC%&&:eAh+E+EY^_Y^PUUKUT248UKY^_S E4((DIJED1$7EJS 17S ' L_Ks2 C$;C C$5C*C$c ^^^ ^ [US/5 SSKm 0nTRS5m T cz[T5S:akTSn[ U5(aOU[ U[ 5(d%[U5(d[ U[5(aUm O[S[ U5S35eUU UU 4SjnUR5Hupg[ U[5(aP[ US[5(a8/nUH*n URU V s/sH o"U 5PM sn 5 M, XU'Mj[ U[5(aUV s/sH o"U 5PM sn X6'MU"U5X6'M X0l U$s sn fs sn f)a Send all values to device by calling `v.to(*args, **kwargs)` (PyTorch only). This should support casting in different `dtypes` and sending the `BatchFeature` to a different `device`. Args: args (`Tuple`): Will be passed to the `to(...)` function of the tensors. kwargs (`Dict`, *optional*): Will be passed to the `to(...)` function of the tensors. Returns: [`BatchFeature`]: The same instance after modification. torchrNdevicez*Attempting to cast a BatchFeature to type z. This is not supported.c~>TR"U5(aUR"T0TD6$TbURTS9$U$)N)rG)is_floating_pointto)r3argsrGkwargsrFs r&_to FuyuBatchFeature.to.._tosD&&t,,ww///!wwfw--Kr8)rrFgetlenrr"strrintr*r?r#appenddata) r@rKrLnew_dataargrMkvnew_vrBr3rGrFs `` @@r&rJFuyuBatchFeature.to{sK $ *H% >c$i!mq'Cc""C%%)=)=CQTAUAU!#McRUhZWo!pqq  JJLDA!T""z!A$'='=ELL!>#d)!>?# At$$567QTs4yQ7 !!f !  "?8s E0 E5rTr!)rr) __name__ __module__ __qualname____firstlineno____doc__rrrQrrCrJ__static_attributes__r2r8r&r-r-Is' )huS*_7M.N)V8r8r-c!^\rSrSrSr/SQrSS\RSSSSSSSS S4 S \S \ \ \ \ 4S \S \S\ S\ S\S\\ \\ 4S\\ \\ 4S\S\ S\ \ \ \ 44U4Sjjjr\RSS4S\R$S \ \ \ 4S \S\ \\ \4S\ \\ \4S\R$4 SjjrS/S\R$S \ \ \ 4S\ S\ S\ \\ \4S\ \\ \4S\R$4Sjjr\"5SSSSSSSSSSSS\R.SS4S \ \S \ \ \ \ 4S \ \S \ \S\ \ S\ \ S\ \S\ \ S\ \ S\ \S\ \ S\ \ \ \ 4S\ \\ \4S\ \\ \4S\ \4S jj5rS0S!\ S"\ S\ \ \ \ 4S\ 4S#jjrS0SS$S\ \ \ \ 4SS$4S%jjrS0S&S$S'S$S(S$S)S$S*\ S+\ S,\S\ \ \ \ 4S\4S-jjrS.rU=r$)1FuyuImageProcessora This class should handle the image processing part before the main FuyuForCausalLM. In particular, it should handle: - Processing Images: Taking a batch of images as input. If the images are variable-sized, it resizes them based on the desired patch dimensions. The image output is always img_h, img_w of (1080, 1920) Then, it patches up these images using the patchify_image function. - Creating Image Input IDs: For each patch, a placeholder ID is given to identify where these patches belong in a token sequence. For variable-sized images, each line of patches is terminated with a newline ID. - Image Patch Indices: For each image patch, the code maintains an index where these patches should be inserted in a token stream. Args: do_resize (`bool`, *optional*, defaults to `True`): Whether to resize the image to `size`. size (`dict[str, int]`, *optional*, defaults to `{"height": 1080, "width": 1920}`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. resample (`PILImageResampling`, *optional*, defaults to `Resampling.BILINEAR`): `PILImageResampling` filter to use when resizing the image e.g. `PILImageResampling.BILINEAR`. do_pad (`bool`, *optional*, defaults to `True`): Whether to pad the image to `size`. padding_value (`float`, *optional*, defaults to 1.0): The value to pad the image with. padding_mode (`str`, *optional*, defaults to `"constant"`): The padding mode to use when padding the image. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. image_mean (`float`, *optional*, defaults to 0.5): The mean to use when normalizing the image. image_std (`float`, *optional*, defaults to 0.5): The standard deviation to use when normalizing the image. do_rescale (`bool`, *optional*, defaults to `True`): Whether to rescale the image. rescale_factor (`float`, *optional*, defaults to `1 / 255`): The factor to use when rescaling the image. patch_size (`dict[str, int]`, *optional*, defaults to `{"height": 30, "width": 30}`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the patches. rimage_input_ids image_patchesimage_patch_indices_per_batch#image_patch_indices_per_subsequenceTN?constantg?gp? do_resizesizeresampledo_pad padding_value padding_mode do_normalize image_mean image_std do_rescalerescale_factor patch_sizec  >[TU]"S0U D6 XlUbUOSSS.UlX0lX@lXPlX`lXplXl Xl Xl Xl U bXl gSSS.Ul g)Ni8i)heightwidthr2)super__init__rlrmrnrorprqrrrsrtrurvrw)r@rlrmrnrorprqrrrsrtrurvrwrL __class__s r&r}FuyuImageProcessor.__init__st "6"" ,DTD2Q   *(($"$,(2(>*r\^D_r8r% data_formatinput_data_formatrc [X5upxUSUSpX::aXy::aU$X- n X- n [X5n [X}-5n[X-5n[SUX4UUUS.UD6nU$)a Resize an image to `(size["height"], size["width"])`. Args: image (`np.ndarray`): Image to resize. size (`dict[str, int]`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. resample (`PILImageResampling`, *optional*, defaults to `PILImageResampling.BILINEAR`): `PILImageResampling` filter to use when resizing the image e.g. `PILImageResampling.BILINEAR`. data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. Returns: `np.ndarray`: The resized image. ryrz)r%rmrnrrr2)rminrRr )r@r%rmrnrrrL image_height image_width target_height target_widthheight_scale_factorwidth_scale_factoroptimal_scale_factor new_height new_width scaled_images r&r FuyuImageProcessor.resize sF%35$L! &*8nd7m|  &<+HL+:)7"#6K<=  :;  (#/    r8modeconstant_valuesc n[X5upxUSUSpSn Sn X- n X- n[UX4X44UUUUS9nU$)a0 Pad an image to `(size["height"], size["width"])`. Args: image (`np.ndarray`): Image to pad. size (`dict[str, int]`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. data_format (`ChannelDimension` or `str`, *optional*): The data format of the output image. If unset, the same format as the input image is used. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred. ryrzr)paddingrrrr)rr )r@r%rmrrrrrrrr padding_top padding_leftpadding_bottom padding_right padded_images r& pad_imageFuyuImageProcessor.pad_imageDsi,%35$L! &*8nd7m|  &5$2  !2\4QR+#/  r8return_tensorsc UbUO URnUbUO URnUbUO URnUbUO URnU bU O URn U bU O UR n UbUO UR nU bU O URn U bU O URn UbUO URnUbUO URnU bU O URn U bU O UR n U bU O URn [U[5(a"[SU55(a [S5e[!U5n[#U U UU U UUUS9 UVVs/sHoVs/sHn[%U5PM snPM nnnUVs/sH o(dM UPM snSSnU (a%['U5(a[(R+S5 Uc [-U5nUVs/sHo(dM [/USUS9PM nn[1U5nU(a3UVVs/sH$nUVs/sHnUR3UX?S9PM snPM& nnnUVs/sHo(dM [/USUS9PM nnUVs/sH nUS/PM nnUVs/sH nUS/PM nn[5UU5VVs/sHunnUSUS- /PM nnnU(a6UVVs/sH'nUVs/sHnUR7UUUUUS 9PM snPM) nnnU (a3UVVs/sH$nUVs/sHnUR9UXS 9PM snPM& nnnU(a4UVVs/sH%nUVs/sHnUR;UXUS 9PM snPM' nnnUb/UVVs/sH nUVs/sHn[=UX5PM snPM" nnnUUUUS .n[?UUS 9$s snfs snnfs snfs snfs snfs snnfs snfs snfs snfs snnfs snfs snnfs snfs snnfs snfs snnfs snfs snnf)a Utility function to preprocess the images and extract necessary information about original formats. Args: images (`ImageInput`): Images to preprocess. Expects a single image, a list or images or a list of lists of images. Pixel values range from 0 to 255, or between 0 and 1 if `do_rescale` is `False`. do_resize (`bool`, *optional*, defaults to `self.do_resize`): Whether to resize the image to `size`. size (`dict[str, int]`, *optional*, defaults to `self.size`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the output image. resample (`PILImageResampling`, *optional*, defaults to `self.resample`): `PILImageResampling` filter to use when resizing the image e.g. `PILImageResampling.BILINEAR`. do_pad (`bool`, *optional*, defaults to `self.do_pad`): Whether to pad the image to `size`. padding_value (`float`, *optional*, defaults to `self.padding_value`): The value to pad the image with. padding_mode (`str`, *optional*, defaults to `self.padding_mode`): The padding mode to use when padding the image. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. image_mean (`float`, *optional*, defaults to `self.image_mean`): The mean to use when normalizing the image. image_std (`float`, *optional*, defaults to `self.image_std`): The standard deviation to use when normalizing the image. do_rescale (`bool`, *optional*, defaults to `self.do_rescale`): Whether to rescale the image. rescale_factor (`float`, *optional*, defaults to `self.rescale_factor`): The factor to use when rescaling the image. patch_size (`dict[str, int]`, *optional*, defaults to `self.patch_size`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the patches. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format of the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. c3l# UH*n[U[5=(a [U5S:v M, g7f)N)r"r#rP)r$r3s r&r'0FuyuImageProcessor.preprocess..s++ibhZ^JtT,B,Us4yTU~,Ubhs24z:Multiple images for a single sample are not yet supported.)rurvrrrsrtrlrmrnrzIt looks like you are trying to rescale already rescaled images. If the input images have pixel values between 0 and 1, set `do_rescale=False` to avoid rescaling them again.) channel_dim)rmr)rmrrr)scaler)meanstdr)rimage_unpadded_heightsimage_unpadded_widthsimage_scale_factors)rTr/) rlrmrnrorurvrrrsrtrprqrwr"r#anyr*r+rrrlogger warning_oncerrrr ziprrescale normalizer r-)r@rrlrmrnrorprqrrrsrtrurvrwrrr batch_imagesr%first_image_in_listoriginal_image_sizes image_sizes image_sizerr original_size resized_sizerrTs r& preprocessFuyuImageProcessor.preprocessjsL"+!6IDNN 'tTYY'38!-4;;#-#9Zt +9+E4K^K^'3'?|TEVEV #-#9Zt !*!6IDNN )6)B HZHZ '3'?|TEVEV #-#9Zt +9+E4K^K^#-#9Zt fd # #+ibh+i(i(iYZ Z26: %!)%! S__R^FCF5.FCR^ _5AKL&FvLKANqQ /*=>>   s   $ >?R S T` S_ci DN6!92C DS_  T" +*FbhhagX]USagh*  _ku^jTZntO~fQi=NO^j uDO!PKj:a=/K!PCN O;Z*Q-; O 033G/U /U+ |!_}Q/ / 0/U  + +F"( "(NN!)(5*; #"( +   +*Fnttmsdie>_mst*   +  +F"(!'NN5z\mNn!'+   "+*FbhhagX],UKSagh*  #&<%:#6    T~FF_D_L i v!P O    uis N)N$ $N)2 N/N/ N4N4 N>N9 )N>7 OOO 2OO; OO "O8 O)O$ O)2 O4<O/ O4) O?3O: O?$N)9N>O$O)/O4:O?rrcUbUO URnURSURSpTX-S:wa[SU<SU35eX%-S:wa[SU<SU35eX-nX%-nXg-nU$)a` Calculate number of patches required to encode an image. Args: image_height (`int`): Height of the image. image_width (`int`): Width of the image. patch_size (`dict[str, int]`, *optional*, defaults to `self.patch_size`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the patches. ryrzrz image_height=z must be divisible by z image_width=)rwr*) r@rrrw patch_height patch_widthnum_patches_per_dim_hnum_patches_per_dim_w num_patchess r&get_num_patches"FuyuImageProcessor.get_num_patches s$.#9Zt $(OOH$=tw?Wk  &! + .D\NST T  $ ) ~-CK=QR R , < + :+C r8z torch.Tensorc\[US/5 UbUO URnUSUSpCURupV nURSX35nURSXD5n U R 5n U R XVSX45n U R SSSSS 5n U RUSXc-U-5n U $) aD Convert an image into a tensor of patches. Args: image (`torch.Tensor`): Image to convert. Shape: [batch, channels, height, width] patch_size (`dict[str, int]`, *optional*, defaults to `self.patch_size`): Dictionary in the format `{"height": int, "width": int}` specifying the size of the patches. rFryrzrrrr)rrwshapeunfold contiguousviewpermutereshape) r@r%rwrr batch_sizechannels_unfolded_along_heightpatchess r&patchify_image!FuyuImageProcessor.patchify_image9s $ *#-#9Zt $.x$8*W:Mk &+[[" a % Q K'..q+K$$&,,zRS//!Q1a0//*b(2IK2WXr8 image_input image_presentimage_unpadded_himage_unpadded_wimage_placeholder_idimage_newline_idvariable_sizedc [US/5 UbUO URnUSUSp/n /n /n [URS5GH`n/n/n[URS5GHnX.U4(GaXU4nURSURSnnU(af[ U[ R "X>U4U - 5U -5n[ U[ R "XNU4U - 5U -5nUSS2SU2SU24nUUnnURUUS9n[R"U/U[RURS 9nURURS5S 9RS5nUURS:XdeU(a{URS UU -5n[R"URSS/U[RURS 9n[R "UU/SS 9nURS 5nU R#U/5 UR#U5 UR#U5 GMUR#[R$"/[RURS 95 GM U R#U5 U R#U5 GMc /n/nU GHnSn/n/n UHn!U!U:Hn"[R&"U"5n[R("U[R*U!RS 9R-U!5n#[R."U!S 5n$[R."U!S 5n%[R0"U"S S9Sn&U#U-U$U&'U#U%U&'UR#U$5 U R#U%5 UU- nM UR#U5 UR#U 5 GM [3U U U UUS.S9$)a Process images for model input. In particular, variable-sized images are handled here. Args: image_input (`torch.Tensor` of shape [batch_size, subsequence_size, num_channels, height, width]): Tensor of images padded to model input size. image_present (`torch.Tensor` of shape [batch_size, subsequence_size, num_images]): Tensor of 1s and 0s indicating whether an image is present. image_unpadded_h (`torch.Tensor` of shape [batch_size, subsequence_size]): Tensor of unpadded image heights. image_unpadded_w (`torch.Tensor` of shape [batch_size, subsequence_size]): Tensor of unpadded image widths. image_placeholder_id (int): The id of the image placeholder token. Comes from an associated tokenizer. image_newline_id (int): The id of the image newline token. Comes from an associated tokenizer. variable_sized (bool): Whether to process images as variable-sized. patch_size (`dict[str, int]`, *optional*, defaults to `self.patch_size`): Size of the patches. rFNryrzrrr)rr)dtyperG)r%r)dimT)as_tuplerer[)rrwrangerrmathceilrrFfullint32rGr unsqueezesqueezercatrStensor count_nonzeroarangeint64type_as full_likenonzeror-)'r@rrrrrrrrwrrrbatch_image_patchesbatch_image_input_ids batch_indexrfrg subseq_indexr%rrnew_hnew_wrtensor_of_image_idsr newline_idsrhrisample_image_input_ids index_offsetper_batch_indicesper_subsequence_indicessubseq_image_input_ids patches_maskindicesindices_in_stream_per_batch!indices_in_stream_per_subsequence patches_indss' r&preprocess_with_tokenizer_info1FuyuImageProcessor.preprocess_with_tokenizer_infoSs> $ *#-#9Zt $.x$8*W:Mk,.8::< !2!21!56K OM %k&7&7&: ;  l!:;;'\(ABE05 A A+L%!$( II&6L7P&QT`&`adpp!!$' II&6L7P&QT_&_`cnn!!&a%%&7 8495k "&"6"6L^i"6"jK*/**$ ';5;;WbWiWi+'#118J1KSSTUVG&'--*::::%.A.I.I"k]hNh.i+&+jj066q91=,"'++#.#5#5 ' /4ii9Lk8Z`a.b+.A.I.I".M+MM5'*#**+>?!((1#**5<<%++VaVhVh+ijS!TVX.Y+49OODZ\^4_1$}}\DI!L BDH 8zz838n8% 8 eC)9$9:; 8 $E#/?*?$@A 8 8|!$>BDH$zz$38n$ $  $ eC)9$9:; $$E#/?*?$@A$ $L%&%))-15!%)-&*'+&*%)%)*./3>N>T>TDH/3#sGD>sGtCH~& sG -. sG  sG sGsmsGtnsGUOsGE?sGTNsG!sGT#s(^,sGeC)9$9:;sG $E#/?*?$@A!sG"!,#sG'sGjCcxX\]`be]eXfOgsv2NcSVh@XdrF04B #B &B ) B ) B " B B B T#s(^,B  B B r8rc)+r`rtypingrrnumpyrimage_processing_utilsrrrimage_transformsr r r image_utilsr r rrrrrrrrutilsrrrrrrrrF get_loggerr\rr#r+r-rc__all__r2r8r&r s& "UU      H % a $tJ'($z*:JF G a $z  aj|jZ_ +_ D r8