Fix version dropdown issue in version 3.5.1 of pyspark document

[panbingkun]

The pr aims to fix version dropdown issue in version 3.5.1 of pyspark document, includes:

1. master -> 3.5.1
2. _static/versions.json -> https://spark.apache.org/static/versions.json
3. move versions.json from docs/3.5.1/api/python/_static/versions.json to static/versions.json.
4. add a new folder site/static to the entry keep_files in the configuration file _config.yml to prevent build failures.

[panbingkun]

cc @HyukjinKwon @HeartSaVioR @dongjoon-hyun

[panbingkun]

If need to remove this feature from the already released version 3.5.1, please let me know and I will do it.

[panbingkun]

The file versions.json has been moved from the sub folder (eg: docs/3.5.1/api/python/_static/versions.json) of each version to static/versions.json, and the URL(eg: _static/versions.json) in all HTML published in 3.5.1 has been updated.

[panbingkun]

[panbingkun]

Let me check the reason for the build failure and wait a moment.

[panbingkun]

Double-check as follows:

[panbingkun]

Let me check the reason for the build failure and wait a moment.

Done. The issue of build failure has been resolved.

[HyukjinKwon]

Merged to asf-site.

[panbingkun]

At present, it has been restored to its correct state.
https://spark.apache.org/docs/3.5.1/api/python/index.html

[HeartSaVioR]

@panbingkun
Nice, thanks for the work! Followup question.

The version showing as master is confusing. when the version has constructed and which information it will refer to? I built the doc with v3.5.1 tag and the version is shown as master.

Could you please give a try with tagging 3.5.2 from branch-3.5 in your fork repository and build the doc to confirm it won't show up as master?

[panbingkun]

@panbingkun Nice, thanks for the work! Followup question.

The version showing as master is confusing. when the version has constructed and which information it will refer to? I built the doc with v3.5.1 tag and the version is shown as master.

Could you please give a try with tagging 3.5.2 from branch-3.5 in your fork repository and build the doc to confirm it won't show up as master?

Sure, let me check it!

Additional remarks:
In the configuration file python/docs/source/conf.py used to build the pyspark document, there is a local environment variable RELEASE_VERSION`, as follows:
https://github.com/apache/spark/blob/9cac2bb608c65f7fec19927711aff6dd1c54005a/python/docs/source/conf.py#L136-L138

When build the pyspark document, it needs to be set first, because the logic of the drop-down box is as follows:
https://github.com/apache/spark/blob/9cac2bb608c65f7fec19927711aff6dd1c54005a/python/docs/source/_templates/version-switcher.html#L20

[panbingkun]

The verification steps are as follows:

• export RELEASE_VERSION=3.5.2
• git checkout branch-3.5
• git tag v3.5.2
• cd python/docs
• make clean html
• Check the content of the generated html file, such as:
  

[panbingkun]

@HeartSaVioR
In addition, I have submitted a PR regarding Use the same versions.json in the dropdown of different versions of PySpark documents: apache/spark#45400
In order to ensure the correctness of the dropdown data, we should merge this PR before the next version release.

[panbingkun]

view-source:https://spark.apache.org/docs/3.5.1/api/python/index.html

Not related to this PR, in addition, I found a flaw in our online 3.5.1 pyspark documentation, which is also likely caused by the unset the local environment variable RELEASE_VERSION . If it needs to be fixed, I am also very happy to use tools to fix it.

[HeartSaVioR]

@panbingkun
Sorry to visit this too lately. Definitely we need to fix the version in 3.5.1. It'd be awesome if you could submit a PR for the fix. Thanks!

[HeartSaVioR]

Also, will the json file be updated automatically, or is release manager expected to manually add the entry? If it's latter, we'll need to update the release process page in Spark website.

[panbingkun]

Sorry to visit this too lately. Definitely we need to fix the version in 3.5.1. It'd be awesome if you could submit a PR for the fix. Thanks!

Sure, let me do it.

[panbingkun]

Also, will the json file be updated automatically, or is release manager expected to manually add the entry? If it's latter, we'll need to update the release process page in Spark website.

Currently, it needs to be manually updated. Let me update the release process page in Spark website, and then I will think of a way to automate it. 😄