Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(python): Add make docs command, DataType docs/layout tweak, minor README updates #15386

Merged
merged 6 commits into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 13 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -198,27 +198,20 @@ You can also install a subset of all optional dependencies.
pip install 'polars[numpy,pandas,pyarrow]'
```

| Tag | Description |
| ---------- | ---------------------------------------------------------------------------- |
| **all** | Install all optional dependencies (all of the following) |
| pandas | Install with pandas for converting data to and from pandas DataFrames/Series |
alexander-beedie marked this conversation as resolved.
Show resolved Hide resolved
| numpy | Install with NumPy for converting data to and from NumPy arrays |
| pyarrow | Reading data formats using PyArrow |
| fsspec | Support for reading from remote file systems |
| connectorx | Support for reading from SQL databases |
| xlsx2csv | Support for reading from Excel files |
| openpyxl | Support for reading from Excel files with native types |
| deltalake | Support for reading and writing Delta Lake Tables |
| pyiceberg | Support for reading from Apache Iceberg tables |
| plot | Support for plot functions on DataFrames |
| timezone | Timezone support, only needed if you are on Python<3.9 or Windows |
See the [User Guide](https://docs.pola.rs/user-guide/installation/#feature-flags) for more details on optional dependencies

To see the current Polars version and a full list of its optional dependencies, run:

```python
pl.show_versions()
```

Releases happen quite often (weekly / every few days) at the moment, so updating Polars regularly to get the latest bugfixes / features might not be a bad idea.

### Rust

You can take latest release from `crates.io`, or if you want to use the latest features / performance improvements
point to the `main` branch of this repo.
You can take latest release from `crates.io`, or if you want to use the latest features / performance
improvements point to the `main` branch of this repo.

```toml
polars = { git = "https://github.com/pola-rs/polars", rev = "<optional git tag>" }
Expand Down Expand Up @@ -251,18 +244,17 @@ Note that the Rust crate implementing the Python bindings is called `py-polars`
Rust crate `polars` itself. However, both the Python package and the Python module are named `polars`, so you
can `pip install polars` and `import polars`.

## Use custom Rust function in Python?
## Using custom Rust functions in Python
alexander-beedie marked this conversation as resolved.
Show resolved Hide resolved

Extending Polars with UDFs compiled in Rust is easy. We expose PyO3 extensions for `DataFrame` and `Series`
data structures. See more in https://github.com/pola-rs/pyo3-polars.

## Going big...

Do you expect more than 2^32 (~4.2 billion) rows? Compile Polars with the `bigidx` feature flag.

Or for Python users, install `pip install polars-u64-idx`.
Do you expect more than 2^32 (~4.2 billion) rows? Compile Polars with the `bigidx` feature
flag or, for Python users, install `pip install polars-u64-idx`.

Don't use this unless you hit the row boundary, as the default build of Polars is faster and consumes less memory.
Don't use this unless you hit the row boundary as the default build of Polars is faster and consumes less memory.

## Legacy

Expand Down
11 changes: 10 additions & 1 deletion py-polars/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ build-release-native: .venv ## Same as build-release, except with native CPU op
@$(MAKE) -s -C .. $@

.PHONY: lint
lint: .venv ## Run lint checks
lint: .venv ## Run lint checks (only)
$(VENV_BIN)/ruff check
-$(VENV_BIN)/mypy

Expand Down Expand Up @@ -84,6 +84,15 @@ doctest: .venv build ## Run doctests
$(VENV_BIN)/python tests/docs/run_doctest.py
$(VENV_BIN)/pytest tests/docs/test_user_guide.py -m docs

.PHONY: docs
docs: .venv build ## Build Python docs (incremental)
@$(MAKE) -s -C docs html

.PHONY: docs-clean
docs-clean: .venv ## Build Python docs (full rebuild)
@$(MAKE) -s -C docs clean
@$(MAKE) docs

.PHONY: test-all
test-all: .venv build ## Run all tests
$(VENV_BIN)/pytest -n auto --dist loadgroup -m "slow or not slow"
Expand Down
19 changes: 14 additions & 5 deletions py-polars/docs/source/reference/datatypes.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
==========
Data types
==========
.. currentmodule:: polars
.. currentmodule:: polars.datatypes

DataType
~~~~~~~~
.. autosummary::
:toctree: api/
:nosignatures:
Expand Down Expand Up @@ -47,6 +49,17 @@ Nested
List
Struct

String
~~~~~~
.. autosummary::
:toctree: api/
:nosignatures:

String
Categorical
Enum
stinodego marked this conversation as resolved.
Show resolved Hide resolved
Utf8

Other
~~~~~
.. autosummary::
Expand All @@ -55,10 +68,6 @@ Other

Binary
Boolean
Categorical
Enum
Null
Object
String
Utf8
Unknown
4 changes: 4 additions & 0 deletions py-polars/docs/source/reference/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,10 @@ methods. All classes and functions exposed in the ``polars.*`` namespace are pub
:maxdepth: 2

api

.. toctree::
:maxdepth: 1

plugins


Expand Down