Lighter is an opensource application for interacting with Apache Spark on Kubernetes or Apache Hadoop YARN. It is heavily inspired by Apache Livy and has some overlapping features.
Lighter supports:
- Interactive Python Sessions through Sparkmagic kernel
- Batch application submissions through the REST API
You can read a breaf description on how Lighter works here.
- Quickstart with Minikube
- Installation on Kubernetes
- Installation on Docker
- Configuration Properties
- Configuring Sparkmagic
- Using REST API
To build Docker image, containing application, you need to run
docker build -t lighter .
Lighter will always try to support the latest patch version for all officially recommended Apache Spark releases i.e.: 3.4.2 and 3.5.1.
Lighter is MIT licensed.