Skip to content

Commit

Permalink
[SPARK-47851][CONNECT][DOCS] Document pyspark-connect package
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR proposes to document the installation for `pyspark-connect` package

### Why are the changes needed?

So the end users can install pure Python library.

### Does this PR introduce _any_ user-facing change?

Yes, it documents `pyspark-connect` package, and links the quickstart as well.

### How was this patch tested?

Manually built the docs and checked:

```bash
cd python/docs
make clean html
open build/html/index.html
```

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#46054 from HyukjinKwon/SPARK-47851.

Authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
  • Loading branch information
HyukjinKwon committed Apr 15, 2024
1 parent e6b7950 commit b8354bb
Showing 1 changed file with 14 additions and 1 deletion.
15 changes: 14 additions & 1 deletion python/docs/source/getting_started/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Python 3.8 and above.
Using PyPI
----------

PySpark installation using `PyPI <https://pypi.org/project/pyspark/>`_ is as follows:
PySpark installation using `PyPI (pyspark) <https://pypi.org/project/pyspark/>`_ is as follows:

.. code-block:: bash
Expand Down Expand Up @@ -81,6 +81,19 @@ Supported values in ``PYSPARK_HADOOP_VERSION`` are:
Note that this installation of PySpark with/without a specific Hadoop version is experimental. It can change or be removed between minor releases.


Python Spark Connect Client
~~~~~~~~~~~~~~~~~~~~~~~~~~~

The Python Spark Connect client is a pure Python library that does not rely on any non-Python dependencies such as jars and JRE in your environment.
To install the Python Spark Connect client via `PyPI (pyspark-connect) <https://pypi.org/project/pyspark-connect/>`_, execute the following command:

.. code-block:: bash
pip install pyspark-connect
See also `Quickstart: Spark Connect <quickstart_connect.html>`_ for how to use it.


Using Conda
-----------

Expand Down

0 comments on commit b8354bb

Please sign in to comment.