cCiWMSrSSKrSSKrSSKJrJr SSKrSSKJ r SSK J r J r SSK JrJrJrJr SSKJrJrJrJrJrJrJr SS KJrJrJrJr SS KJ r \"5(aSSK!r!SS K"J#r#J$r$J%r% \"5(aSSK&r&\RN"\(5r)S r*S r+S S\,S\-S\,S\,S\-S\-S\-S\-S\\.S\\,S\#RF4Sjjr/S!S\R`S\,S\\\,\144Sjjr2"SS\ 5r3S/r4g)"z%Image processor class for Pix2Struct.N)OptionalUnion)hf_hub_download)BaseImageProcessor BatchFeature)convert_to_rgb normalizeto_channel_dimension_format to_pil_image)ChannelDimension ImageInputget_image_sizeinfer_channel_dimension_formatmake_flat_list_of_imagesto_numpy_array valid_images) TensorTypeis_torch_availableis_vision_availablelogging)requires_backends)Image ImageDraw ImageFontzybelkada/fontsc[[S/5 URS5n[RR R XU4X4S9nURURS5URS5XS5nURSSSSS5RURS5U-URS5U-URS5U-U-5nURS5$) a Utility function to extract patches from a given image tensor. Returns a tensor of shape (1, `rows`, `columns`, `num_channels`x `patch_height` x `patch_width`). Args: image_tensor (torch.Tensor): The image tensor to extract patches from. patch_height (int): The height of the patches to extract. patch_width (int): The width of the patches to extract. torchr)strider) rtorch_extract_patches unsqueezernn functionalunfoldreshapesizepermute) image_tensor patch_height patch_widthpatchess t/home/james-whalen/.local/lib/python3.13/site-packages/transformers/models/pix2struct/image_processing_pix2struct.pyr#r#4s+gY7))!,Lhh!!((k7R\h[v(wGool//2L4E4Ea4H,eghGooaAq!,44! ,! +!|+k9G   Q text text_size text_colorbackground_color left_padding right_padding top_paddingbottom_padding font_bytes font_pathreturnc J[[S5 [R"SS9n U R US9n SR U 5n UbU c[ R"U5n OU bU n O[[S5n [R"U SUS9n [R"[R"S S U55nUR!S X5u nnnUU-U-nUU-U-n[R"S UU4U5n[R"U5nUR#XF4XU S 9 U$) aC Render text. This script is entirely adapted from the original script that can be found here: https://github.com/google-research/pix2struct/blob/main/pix2struct/preprocessing/preprocessing_utils.py Args: text (`str`, *optional*, defaults to ): Text to render. text_size (`int`, *optional*, defaults to 36): Size of the text. text_color (`str`, *optional*, defaults to `"black"`): Color of the text. background_color (`str`, *optional*, defaults to `"white"`): Color of the background. left_padding (`int`, *optional*, defaults to 5): Padding on the left. right_padding (`int`, *optional*, defaults to 5): Padding on the right. top_padding (`int`, *optional*, defaults to 5): Padding on the top. bottom_padding (`int`, *optional*, defaults to 5): Padding on the bottom. font_bytes (`bytes`, *optional*): Bytes of the font to use. If `None`, the default font will be used. font_path (`str`, *optional*): Path to the font to use. If `None`, the default font will be used. visionP)width)r1 z Arial.TTFzUTF-8)encodingr)RGB)rrrr)xyr1fillfont)r render_texttextwrap TextWrapperwrapjoinioBytesIOrDEFAULT_FONT_PATHrtruetyperDrawrnewtextbboxr1)r1r2r3r4r5r6r7r8r9r:wrapperlines wrapped_textrF temp_draw_ text_width text_height image_width image_heightimagedraws r/rGrGOsLk8,"",G LLdL #E99U#L)"3zz*%  0+>   dW9 EDuyy8HIJI$-$6$6v|$R!Aq*k|+m;K,~=L IIek<8:J KE >>% DII,,JL//9?Q?Q3QRS %):K-K!LgVI OOL''(FGP OOELL)!89A?P;QRy)I%i04D4I4II/ ;K;P;PQ r0c^\rSrSrSrSS/rSS\S\S\\\ \ 4S \ S \S S4 U4S jjjr SS \ RS \ S\S\\\ \4S \ R4 SjjrSS \ RS\\\ \4S\\\ \4S \ R4SjjrSSSSSS\R$S4S\S\\ S\\S\\S \\ S\\\ \ 4S\\\ \4S\S\\\ \4S \4SjjrSrU=r$)Pix2StructImageProcessora& Constructs a Pix2Struct image processor. Args: do_convert_rgb (`bool`, *optional*, defaults to `True`): Whether to convert the image to RGB. do_normalize (`bool`, *optional*, defaults to `True`): Whether to normalize the image. Can be overridden by the `do_normalize` parameter in the `preprocess` method. According to Pix2Struct paper and code, the image is normalized with its own mean and standard deviation. patch_size (`dict[str, int]`, *optional*, defaults to `{"height": 16, "width": 16}`): The patch size to use for the image. According to Pix2Struct paper and code, the patch size is 16x16. max_patches (`int`, *optional*, defaults to 2048): The maximum number of patches to extract from the image as per the [Pix2Struct paper](https://huggingface.co/papers/2210.03347). is_vqa (`bool`, *optional*, defaults to `False`): Whether or not the image processor is for the VQA task. If `True` and `header_text` is passed in, text is rendered onto the input images. flattened_patchesattention_maskNdo_convert_rgb do_normalize patch_size max_patchesis_vqar;c t>[TU]"S0UD6 UbUOSSS.UlX lXlX@lXPlg)N)rer?)super__init__rvrurtrwrx)selfrtrurvrwrxri __class__s r/r}!Pix2StructImageProcessor.__init__s@ "6"(2(>*r\^D_(,& r0r\r_c [URS5 [U[RU5n[ R "U5nUSUSpv[U[R5up[R"X&U- -Xy- -5n [[[R"X-U- 5U5S5n [[[R"X-U- 5U5S5n [X-S5n [X-S5n[ RRRUR!S5X4SSSS 9R#S5n[%XU5nUR&nUSnUS nUS nUR)UU-U/5n[ R*"U5R)US/5R-SU5R)UU-S/5n[ R*"U5R)SU/5R-US5R)UU-S/5nUS- nUS- nUR/[ R05nUR/[ R05n[ R2"UUU/S 5n[ RRR5USSSUUU-- /5R75n[9U5nU$) ar Extract flattened patches from an image. Args: image (`np.ndarray`): Image to extract flattened patches from. max_patches (`int`): Maximum number of patches to extract. patch_size (`dict`): Dictionary containing the patch height and width. Returns: result (`np.ndarray`): A sequence of `max_patches` flattened patches. rrer?rrbilinearFT)r)mode align_corners antialiasr"rr )rextract_flattened_patchesr r FIRSTr from_numpyrmathsqrtrcminfloorr%r& interpolater$squeezer#shaper(arangerepeattofloat32catpadfloatr)r~r\rwrvr_rir,r-r[rZscalenum_feasible_rowsnum_feasible_colsresized_height resized_widthr. patches_shaperowscolumnsdepthrow_idscol_idsresults r/r2Pix2StructImageProcessor.extract_flattened_patchess. $88'B,E3C3I3IK\]  '$.x$8*W:Mk$25:J:P:P$Q!  + )DEIbcdDJJu/Cl/R$SU` acdeDJJu/B[/P$QS^ _abc.=qA-;Q? ##// OOA  0 0 '!* ([I Q"a //4'>5"9:,,t$,,dAY7>>q'JRRTX[bTbdeSfg,,w'//G =DDT1MUUW[^eWeghVij 1 1 **U]]+**U]]+GWg6;$$((!Q;$QX.;Y1Z[aac' r0 data_formatc xUR[R:XaUR[R5n[R "U5n[R "U5n[US[R"[R"UR55- 5n[U4UUUUS.UD6$)a Normalize an image. image = (image - image_mean) / image_std. The image std is to mimic the tensorflow implementation of the `per_image_standardization`: https://www.tensorflow.org/api_docs/python/tf/image/per_image_standardization Args: image (`np.ndarray`): Image to normalize. data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format for the output image. If unset, the channel dimension format of the input image is used. input_data_format (`str` or `ChannelDimension`, *optional*): The channel dimension format of the input image. If not provided, it will be inferred. g?)meanstdrr_) dtypenpuint8astyperrrrcrrprodrr )r~r\rr_rirradjusted_stddevs r/r "Pix2StructImageProcessor.normalize5s, ;;"(( "LL,Ewwu~ffUmc32775;;3G)H#HI  #/     r0images header_textreturn_tensorsc 4UbUO URnUbUO URnUbUO URnUbUO URnURn U R S5b [ S5e[U5n[U5(d [ S5eU(aUV s/sHn [U 5PM nn UV s/sHn [U 5PM nn U c[US5n U (aUc [ S5eU RSS5n U RSS5n[U[5(aU/[U5-n[!U5VV s/sHup[#XUXS 9PM nnn U(aUV s/sHoR%XS 9PM nn UV s/sHn UR'XXiS 9PM nn UV s/sH1oR)S S 9S:gR+[,R.5PM3 nn [1UUS.US9nU$s sn fs sn fs sn nfs sn fs sn fs sn f)as Preprocess an image or batch of images. The processor first computes the maximum possible number of aspect-ratio preserving patches of size `patch_size` that can be extracted from the image. It then pads the image with zeros to make the image respect the constraint of `max_patches`. Before extracting the patches the images are standardized following the tensorflow implementation of `per_image_standardization` (https://www.tensorflow.org/api_docs/python/tf/image/per_image_standardization). Args: images (`ImageInput`): Image to preprocess. Expects a single or batch of images. header_text (`Union[list[str], str]`, *optional*): Text to render as a header. Only has an effect if `image_processor.is_vqa` is `True`. do_convert_rgb (`bool`, *optional*, defaults to `self.do_convert_rgb`): Whether to convert the image to RGB. do_normalize (`bool`, *optional*, defaults to `self.do_normalize`): Whether to normalize the image. max_patches (`int`, *optional*, defaults to `self.max_patches`): Maximum number of patches to extract. patch_size (`dict`, *optional*, defaults to `self.patch_size`): Dictionary containing the patch height and width. return_tensors (`str` or `TensorType`, *optional*): The type of tensors to return. Can be one of: - Unset: Return a list of `np.ndarray`. - `TensorType.TENSORFLOW` or `'tf'`: Return a batch of type `tf.Tensor`. - `TensorType.PYTORCH` or `'pt'`: Return a batch of type `torch.Tensor`. - `TensorType.NUMPY` or `'np'`: Return a batch of type `np.ndarray`. - `TensorType.JAX` or `'jax'`: Return a batch of type `jax.numpy.ndarray`. data_format (`ChannelDimension` or `str`, *optional*, defaults to `ChannelDimension.FIRST`): The channel dimension format for the output image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - Unset: Use the channel dimension format of the input image. input_data_format (`ChannelDimension` or `str`, *optional*): The channel dimension format for the input image. If unset, the channel dimension format is inferred from the input image. Can be one of: - `"channels_first"` or `ChannelDimension.FIRST`: image in (num_channels, height, width) format. - `"channels_last"` or `ChannelDimension.LAST`: image in (height, width, num_channels) format. - `"none"` or `ChannelDimension.NONE`: image in (height, width) format. Nrz8data_format is not an accepted input as the outputs are zkInvalid image type. Must be of type PIL.Image.Image, numpy.ndarray, torch.Tensor, tf.Tensor or jax.ndarray.rz.A header text must be provided for VQA models.r9r:)r9r:)r\r_)r\rwrvr_r )axis)rrrs)data tensor_type)rurtrvrwrxget ValueErrorrrr rrpop isinstancestrlen enumeraterbr rsumrrrr)r~rrrtrurwrvrrr_rirxr\r9r:iattention_masksencoded_outputss r/ preprocess#Pix2StructImageProcessor.preprocess\s@j(4'?|TEVEV +9+E4K^K^#-#9Zt %0%vay I  " !QRRL$7J ;5I+s++*mc&k9 !*& 1 1HAe^ ` 1  djkdj[`nn5nVdjFk     * * +   V\\U[EII2I.!3;;BJJGU[\&'-Q_m SA= l ]s$G;:H H4H H28H)rtrurxrwrv)TTNiFN)NN)__name__ __module__ __qualname____firstlineno____doc__model_input_namesboolrdictrrdr}rndarrayrr rr rrrr__static_attributes__ __classcell__)rs@r/rprps(-.>? $!/3 T#s(^,      *EI OzzOO O $E#/?*?$@A O Oh?CDH % zz% eC)9$9:;% $E#/?*?$@A %  % T&*)-'+%)/3;?(8(>(>DHqqc]q! q tn q c] qT#s(^,q!sJ!78q&q$E#/?*?$@Aq qqr0rp) $blackrarrrNNr)5rrLrtypingrrnumpyrhuggingface_hubrimage_processing_utilsrrimage_transformsr r r r image_utilsr rrrrrrutilsrrrrutils.import_utilsrrHPILrrrr get_loggerrloggerrNr#rrdbytesrGrChildProcessErrorrbrp__all__r{r0r/rsp, "+FddRQ3//   H %$ :#"&#@ @@@ @  @  @@@@}@ [[@Jbf' ::'"'7?cK\F\@]7^'TP1Pf & &r0