h,SSKrSSKJrJr SSKJr SSKJr SSKJ r J r J r J r J r Jr SSKrSSKJr SSKJr SS KJr SS KJrJr SS KJrJr \ (a SSKrS S KJr SSS\ 4Sjr!\"SS55r"SSS\#4Sjr$g)N) dataclassfield)BytesIO)Path) TYPE_CHECKINGAnyClassVarDictOptionalUnion)config)DownloadConfig) array_cast) is_local_pathxopen)no_op_if_value_is_nullstring_to_dict) FeatureTypepdfpdfplumber.pdf.PDFreturnc[5nURH(nURURR5 M* UR 5sSSS5 $!,(df  g=f)z-Convert a pdfplumber.pdf.PDF object to bytes.N)rpageswriterstreamgetvalue)rbufferpages O/home/james-whalen/.local/lib/python3.13/site-packages/datasets/features/pdf.py pdf_to_bytesr"s? fIID LL )  s AA A+c\rSrSr%SrSr\\S'\"SSS9r \ \ \S'S r \ \ \S '\R"\R "5\R""5S .5r\ \\S '\"SSSS 9r\ \S'SrS\\ \\\S 4S\4SjrSS\SS 4SjjrS\S\\ S444SjrS\\R<\R>\R@4S\R>4Sjr!SS\R>S\R>4Sjjr"Sr#g)Pdfa **Experimental.** Pdf [`Feature`] to read pdf documents from a pdf file. Input: The Pdf feature accepts as input: - A `str`: Absolute path to the pdf file (i.e. random access is allowed). - A `pathlib.Path`: path to the pdf file (i.e. random access is allowed). - A `dict` with the keys: - `path`: String with relative path of the pdf file in a dataset repository. - `bytes`: Bytes of the pdf file. This is useful for archived files with sequential access. - A `pdfplumber.pdf.PDF`: pdfplumber pdf object. Args: mode (`str`, *optional*): The mode to convert the pdf to. If `None`, the native mode of the pdf is used. decode (`bool`, defaults to `True`): Whether to decode the pdf data. If `False`, returns the underlying dictionary in the format `{"path": pdf_path, "bytes": pdf_bytes}`. Examples: ```py >>> from datasets import Dataset, Pdf >>> ds = Dataset.from_dict({"pdf": ["path/to/pdf/file.pdf"]}).cast_column("pdf", Pdf()) >>> ds.features["pdf"] Pdf(decode=True, id=None) >>> ds[0]["pdf"] >>> ds = ds.cast_column("pdf", Pdf(decode=False)) >>> ds[0]["pdf"] {'bytes': None, 'path': 'path/to/pdf/file.pdf'} ``` TdecodeNF)defaultrepridrdtypebytespathpa_type)r'initr(_typecUR$N)r.)selfs r!__call__ Pdf.__call__Ms ||valuerc[R(aSSKnOSn[U[5(aUSS.$[U[ 5(a[ UR 55SS.$[U[[45(aSUS.$Ub/[XRR5(a [U5$URS5b;[RRUS5(aSURS5S.$URS5cURS5b#URS5URS5S.$[!SUS35e) zEncode example into a format for Arrow. Args: value (`str`, `bytes`, `pdfplumber.pdf.PDF` or `dict`): Data passed as input to Pdf feature. Returns: `dict` with "path" and "bytes" fields rNr-r,r-r+r,zRA pdf sample should have one of 'path' or 'bytes' but they are missing or None in .)rPDFPLUMBER_AVAILABLE pdfplumber isinstancestrrabsoluter, bytearrayrPDFencode_pdfplumber_pdfgetosr-isfile ValueError)r3r7r<s r!encode_examplePdf.encode_examplePs"  & & J eS ! !!D1 1 t $ $ 01DA A y1 2 2 51 1  # 5..:L:L(M(M(/ / YYv  *rww~~eFm/L/L!599V+<= = YYw  +uyy/@/L"YYw/69JK Kdejdkklm r6cUR(d [S5e[R(aSSKnO [ S5eUc0nUSUSpTUcUc[ SUS35e[U5(aUR"U5nU$URS 5S nUR[R5(a[RO[Rn[Xx5S n URU 5n [!U S 9n [#US U S9n UR"U 5$UR"[%U55n U nSSS5 U$![ a Sn N\f=f!,(df  W$=f)aDecode example pdf file into pdf data. Args: value (`str` or `dict`): A string with the absolute pdf file path, a dictionary with keys: - `path`: String with absolute or relative pdf file path. - `bytes`: The bytes of the pdf file. token_per_repo_id (`dict`, *optional*): To access and decode pdf files from private repositories on the Hub, you can pass a dictionary repo_id (`str`) -> token (`bool` or `str`). Returns: `pdfplumber.pdf.PDF` zKDecoding is disabled for this feature. Please use Pdf(decode=True) instead.rNz6To support decoding pdfs, please install 'pdfplumber'.r-r,z@A pdf should have one of 'path' or 'bytes' but both are None in r:::repo_idtokenrbdownload_config)r& RuntimeErrorrr;r< ImportErrorrFropensplit startswith HF_ENDPOINTHUB_DATASETS_URLHUB_DATASETS_HFFS_URLrrCrrr)r3r7token_per_repo_idr<r-bytes_r source_urlpatternrLrNrQfps r!decode_examplePdf.decode_examplessi&{{lm m  & & VW W  $ " V}eGnf >| #cdicjjk!lmm &&$//$/C( %"&D!1"!5J&001C1CDD//#99 %"0"Ei"P 1 5 5g >'55&AOdD/JA%??1--1Q2 &% $% 21 s"EE! EE! E0rcVSSKJn UR(aU$U"S5U"S5S.$)zfIf in the decodable state, return the feature itself, otherwise flatten the feature into a dictionary.r)Valuebinarystringr+)featuresrcr&)r3rcs r!flatten Pdf.flattens2#{{  xh r6storagec[RRUR5(ag[R"S/[ U5-[R "5S9n[RRX!/SS/UR5S9nGO[RRUR5(ag[R"S/[ U5-[R"5S9n[RRX/SS/UR5S9nGO*[RRUR5(aURRS5S:aURS5nO5[R"S/[ U5-[R "5S9nURRS5S:aURS5nO5[R"S/[ U5-[R"5S9n[RRX#/SS/UR5S9n[XR 5$)aCast an Arrow array to the Pdf arrow storage type. The Arrow types that can be converted to the Pdf pyarrow storage type are: - `pa.string()` - it must contain the "path" data - `pa.binary()` - it must contain the image bytes - `pa.struct({"bytes": pa.binary()})` - `pa.struct({"path": pa.string()})` - `pa.struct({"bytes": pa.binary(), "path": pa.string()})` - order doesn't matter - `pa.list(*)` - it must contain the pdf array data Args: storage (`Union[pa.StringArray, pa.StructArray, pa.ListArray]`): PyArrow array to cast. Returns: `pa.StructArray`: Array in the Pdf arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. Ntyper,r-maskr)patypes is_stringrlarraylenrd StructArray from_arraysis_null is_binaryre is_structget_field_indexrrr.)r3ri bytes_array path_arrays r! cast_storagePdf.cast_storages& 88  gll + +((D6CL#8ryy{KKnn00+1G'SYIZahapapar0sG XX   - -4&3w<"7biikJJnn00'1FRXHY`g`o`o`q0rG XX   - -||++G49%mmG4  hhvG '<299;O ||++F3q8$]]62 XXtfs7|&;"))+N nn00+1JWV\L]dkdsdsdu0vG'<<00r6c x^Tc0m[U4Sj5n[R"UR5Vs/sHnUbUSc U"US5OUSOSPM sn[R"5S9n[R"UR S5R5Vs/sH&ofb[ RRU5OSPM( sn[R"5S9n[RRXW/SS/UR5S9n[XR5$s snfs snf)zEmbed PDF files into the Arrow array. Args: storage (`pa.StructArray`): PyArrow array to embed. Returns: `pa.StructArray`: Array in the PDF arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. Ncz>URS5SnUR[R5(a[RO[R n[ X5nUbTRUS5OSn[US9n[USUS9nUR5sSSS5 $!,(df  g=f)NrJrKrLrMrOrP) rUrVrrWrXrYrrCrrread)r-r\r]source_url_fieldsrNrQr^rZs r! path_to_bytes(Pdf.embed_storage..path_to_bytessD)"-J+5+@+@ASAS+T+T''Z`ZvZv !/z C K\Kh%))*;I*FGnrE,59OtT?CqvvxDCCs B,, B:r,r-rkrm)rrorr to_pylistrdrrDr-basenamerertrurvrr.)r3rirZrxrzr-r{s ` r! embed_storagePdf.embed_storages*  $ "     hh!**, ,AUVTaQwZ-?qy)QwZgkk,   XXNUmm\bNcNmNmNo pNod'7RWW  d #T ANo p ..,,k-FRXHY`k`s`s`u,v'<<00  qs %D2'-D7r2)$__name__ __module__ __qualname____firstlineno____doc__r&bool__annotations__rr)r r>r*r rostructrdrer.rr0r4r r,r@dictrGr`r rg StringArrayrt ListArrayr|r__static_attributes__rr6r!r$r$s*#JFDd7B 70E8C=/YYbiik'RSGXc]Su5u=E3=!E#uiG[*[$\!ae!F8D8EY8t  }d3 3E.FFG  #1E".."..",,*V$W#1\^\j\j#1J&1R^^&1PRP^P^&1&1r6r$c[US5(aO[URS5(a4URR(aURRSS.$S[U5S.$)a! Encode a pdfplumber.pdf.PDF object into a dictionary. If the PDF has an associated file path, returns the path. Otherwise, serializes the PDF content into bytes. Args: pdf (pdfplumber.pdf.PDF): A pdfplumber PDF object. Returns: dict: A dictionary with "path" or "bytes" field. rnameNr9)hasattrrrr")rs r!rBrBsPsH'#**f"="=#**// $77|C'899r6)%rD dataclassesrriorpathlibrtypingrrr r r r pyarrowrordownload.download_configrtablerutils.file_utilsrrutils.py_utilsrrr<rfrr,r"r$rrBrr6r!rsy (FF53C%!*!u! f1f1 f1R:3::r6