Skip to content

Commit ab4cf70

Browse files
razvansbernauer
andauthored
doc: comment on spark history integration (#559)
* doc: comment on spark history integration * Update docs/modules/spark-k8s/pages/usage-guide/spark-connect.adoc Co-authored-by: Sebastian Bernauer <[email protected]> --------- Co-authored-by: Sebastian Bernauer <[email protected]>
1 parent f8e7951 commit ab4cf70

File tree

2 files changed

+7
-1
lines changed

2 files changed

+7
-1
lines changed

CHANGELOG.md

+2
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ All notable changes to this project will be documented in this file.
1818
- Replace stackable-operator `print_startup_string` with `tracing::info!` with fields.
1919
- BREAKING: Inject the vector aggregator address into the vector config using the env var `VECTOR_AGGREGATOR_ADDRESS` instead
2020
of having the operator write it to the vector config ([#551]).
21+
- Document that Spark Connect doesn't integrate with the history server ([#559])
2122

2223
### Fixed
2324

@@ -28,6 +29,7 @@ All notable changes to this project will be documented in this file.
2829
[#551]: https://github.com/stackabletech/spark-k8s-operator/pull/551
2930
[#553]: https://github.com/stackabletech/spark-k8s-operator/pull/553
3031
[#554]: https://github.com/stackabletech/spark-k8s-operator/pull/554
32+
[#559]: https://github.com/stackabletech/spark-k8s-operator/pull/559
3133

3234
## [25.3.0] - 2025-03-21
3335

docs/modules/spark-k8s/pages/usage-guide/spark-connect.adoc

+5-1
Original file line numberDiff line numberDiff line change
@@ -45,11 +45,15 @@ spec:
4545

4646
The example above adds a new endpoint for application metrics.
4747

48+
== Spark History Server
49+
50+
Unforunately integration with the Spark History Server is not supported yet.
51+
The connect server seems to ignore the `spark.eventLog` properties while also prohibiting clients to set them programatically.
52+
4853
== Notable Omissions
4954

5055
The following features are not supported by the Stackable Spark operator yet
5156

52-
* Integration with the Spark History Server.
5357
* Authorization and authentication. Currently, anyone with access to the Spark Connect service can run jobs.
5458
* Volumes and volume mounts can be added only with pod overrides.
5559
* Job dependencies must be provisioned as custom images or via `--packages` or `--jars` arguments.

0 commit comments

Comments
 (0)