Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(python): Add make docs command, DataType docs/layout tweak, minor README updates #15386

Merged
merged 6 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 20 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,24 +201,28 @@ pip install 'polars[numpy,pandas,pyarrow]'
| Tag | Description |
| ---------- | ---------------------------------------------------------------------------- |
| **all** | Install all optional dependencies (all of the following) |
| pandas | Install with pandas for converting data to and from pandas DataFrames/Series |
alexander-beedie marked this conversation as resolved.
Show resolved Hide resolved
| adbc | Support for reading Arrow data from SQL databases |
| connectorx | Support for reading Arrow data from SQL databases |
| deltalake | Support for reading and writing Delta Lake Tables |
| fastexcel | Support for reading data from Excel files with the Calamine engine |
| fsspec | Support for reading data from remote file systems |
| numpy | Install with NumPy for converting data to and from NumPy arrays |
| openpyxl | Support for reading data from Excel files |
| pandas | Install with pandas for converting data to and from pandas DataFrames/Series |
| plot | Support for plotting functions on DataFrames |
| pyarrow | Reading data formats using PyArrow |
| fsspec | Support for reading from remote file systems |
| connectorx | Support for reading from SQL databases |
| xlsx2csv | Support for reading from Excel files |
| openpyxl | Support for reading from Excel files with native types |
| deltalake | Support for reading and writing Delta Lake Tables |
| pyiceberg | Support for reading from Apache Iceberg tables |
| plot | Support for plot functions on DataFrames |
| pyiceberg | Support for reading data from Apache Iceberg tables |
| sqlalchemy | Support for reading Python data from SQL databases |
| timezone | Timezone support, only needed if you are on Python<3.9 or Windows |
| xlsx2csv | Support for reading data from Excel files |

Releases happen quite often (weekly / every few days) at the moment, so updating Polars regularly to get the latest bugfixes / features might not be a bad idea.
Releases happen quite often (weekly / every few days) at the moment, so updating Polars regularly
to get the latest bugfixes / features might not be a bad idea.
alexander-beedie marked this conversation as resolved.
Show resolved Hide resolved

### Rust

You can take latest release from `crates.io`, or if you want to use the latest features / performance improvements
point to the `main` branch of this repo.
You can take latest release from `crates.io`, or if you want to use the latest features / performance
improvements point to the `main` branch of this repo.

```toml
polars = { git = "https://github.com/pola-rs/polars", rev = "<optional git tag>" }
Expand Down Expand Up @@ -251,18 +255,18 @@ Note that the Rust crate implementing the Python bindings is called `py-polars`
Rust crate `polars` itself. However, both the Python package and the Python module are named `polars`, so you
can `pip install polars` and `import polars`.

## Use custom Rust function in Python?
## Using custom Rust functions in Python
alexander-beedie marked this conversation as resolved.
Show resolved Hide resolved

Extending Polars with UDFs compiled in Rust is easy. We expose PyO3 extensions for `DataFrame` and `Series`
data structures. See more in https://github.com/pola-rs/pyo3-polars.

## Going big...

Do you expect more than 2^32 (~4.2 billion) rows? Compile Polars with the `bigidx` feature flag.

Or for Python users, install `pip install polars-u64-idx`.
Do you expect more than 2^32 (~4.2 billion) rows? Compile Polars with the `bigidx` feature
flag or, for Python users, install `pip install polars-u64-idx`.

Don't use this unless you hit the row boundary, as the default build of Polars is faster and consumes less memory.
(Don't use this unless you hit the row boundary; the default build of Polars is faster
stinodego marked this conversation as resolved.
Show resolved Hide resolved
and consumes less memory).

## Legacy

Expand Down
13 changes: 11 additions & 2 deletions py-polars/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ build-release-native: .venv ## Same as build-release, except with native CPU op
@$(MAKE) -s -C .. $@

.PHONY: lint
lint: .venv ## Run lint checks
lint: .venv ## Run lint checks (only)
$(VENV_BIN)/ruff check
-$(VENV_BIN)/mypy

Expand Down Expand Up @@ -84,8 +84,17 @@ doctest: .venv build ## Run doctests
$(VENV_BIN)/python tests/docs/run_doctest.py
$(VENV_BIN)/pytest tests/docs/test_user_guide.py -m docs

.PHONY: docs
docs: .venv build ## Build Python docs (incremental)
@cd docs && $(MAKE) html
alexander-beedie marked this conversation as resolved.
Show resolved Hide resolved

.PHONY: docs-clean
docs-clean: .venv ## Build Python docs (full rebuild)
@cd docs && $(MAKE) clean
@cd docs && $(MAKE) html

.PHONY: test-all
test-all: .venv build ## Run all tests
test-all: .venv ## Run all tests
alexander-beedie marked this conversation as resolved.
Show resolved Hide resolved
$(VENV_BIN)/pytest -n auto --dist loadgroup -m "slow or not slow"
$(VENV_BIN)/python tests/docs/run_doctest.py

Expand Down
19 changes: 14 additions & 5 deletions py-polars/docs/source/reference/datatypes.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
==========
Data types
==========
.. currentmodule:: polars
.. currentmodule:: polars.datatypes

DataType
~~~~~~~~
.. autosummary::
:toctree: api/
:nosignatures:
Expand Down Expand Up @@ -47,6 +49,17 @@ Nested
List
Struct

String
~~~~~~
.. autosummary::
:toctree: api/
:nosignatures:

String
Categorical
Enum
stinodego marked this conversation as resolved.
Show resolved Hide resolved
Utf8

Other
~~~~~
.. autosummary::
Expand All @@ -55,10 +68,6 @@ Other

Binary
Boolean
Categorical
Enum
Null
Object
String
Utf8
Unknown
4 changes: 4 additions & 0 deletions py-polars/docs/source/reference/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,10 @@ methods. All classes and functions exposed in the ``polars.*`` namespace are pub
:maxdepth: 2

api

.. toctree::
:maxdepth: 1

plugins


Expand Down