σ œώφhˆ γσ•SSKJr SSKJr SSKJr SSKJr \(a SSKJ r SSK J r \"5SSS .S S jj5r g) ι)Ϊ annotations)Ϊ TYPE_CHECKING)Ϊunstable©Ϊ_scan_pyarrow_dataset)Ϊ LazyFrame)ΪpyarrowTN©Ϊallow_pyarrow_filterΪ batch_sizecσ•[UUUS9$)uj Scan a pyarrow dataset. .. warning:: This functionality is considered **unstable**. It may be changed at any point without it being considered a breaking change. This can be useful to connect to cloud or partitioned datasets. Parameters ---------- source Pyarrow dataset to scan. allow_pyarrow_filter Allow predicates to be pushed down to pyarrow. This can lead to different results if comparisons are done with null values as pyarrow handles this different than polars does. batch_size The maximum row count for scanned pyarrow record batches. Warnings -------- Don't use this if you accept untrusted user inputs. Predicates will be evaluated with python 'eval'. There is sanitation in place, but it is a possible attack vector. This method can only can push down predicates that are allowed by PyArrow (e.g. not the full Polars API). If :func:`scan_parquet` works for your source, you should use that instead. Notes ----- When using partitioning, the appropriate `partitioning` option must be set on `pyarrow.dataset.dataset` before passing to Polars or the partitioned-on column(s) may not get passed to Polars. Examples -------- >>> import pyarrow.dataset as ds >>> dset = ds.dataset("s3://my-partitioned-folder/", format="ipc") # doctest: +SKIP >>> ( ... pl.scan_pyarrow_dataset(dset) ... .filter("bools") ... .select("bools", "floats", "date") ... .collect() ... ) # doctest: +SKIP shape: (1, 3) β”Œβ”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ bools ┆ floats ┆ date β”‚ β”‚ --- ┆ --- ┆ --- β”‚ β”‚ bool ┆ f64 ┆ date β”‚ β•žβ•β•β•β•β•β•β•β•ͺ════════β•ͺ════════════║ β”‚ true ┆ 2.0 ┆ 1970-05-04 β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ r r)Ϊsourcer r s Ϊ]/home/james-whalen/.local/lib/python3.13/site-packages/polars/io/pyarrow_dataset/functions.pyΪscan_pyarrow_datasetr s€τ| !ΨΨ1Ψρ πσ)rzpa.dataset.Datasetr Ϊboolr z int | NoneΪreturnr)Ϊ __future__rΪtypingrΪpolars._utils.unstablerΪ(polars.io.pyarrow_dataset.anonymous_scanrΪpolarsrΪpolars._dependenciesr Ϊpar©rrΪrsdπέ"ε ε+έJζέ έ2ρ ƒπ"&Ψ!ρ AΨ πAππAππ Aπ τ Aσ ρAr