Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update build doc for new spec-parser options #1186

Open
wants to merge 10 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 40 additions & 40 deletions build.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
---
SPDX-FileType: DOCUMENTATION
SPDX-License-Identifier: Community-Spec-1.0
SPDX-FileCopyrightText: Copyright 2024 The SPDX Contributors
---

# Building the specification website

The specification website building flow looks like this:
You may want to build the specification website locally to test your additions
and edits and review if they render as intended.

```text
+-------------------+
Expand All @@ -28,7 +29,7 @@ The specification website building flow looks like this:
| +- mkdocs.yml |
+-------------------+
|
mkdocs
MkDocs
|
v
+-------------------+
Expand All @@ -40,6 +41,19 @@ The specification website building flow looks like this:
+-------------------+
```

## Table of contents

1. [Prerequisites](#1-prerequisites)
1. [Getting input files](#2-getting-input-files)
1. [Processing model files (Markdown and RDF)](#3-processing-model-files-markdown-and-rdf)

- [Generating model files with spec-parser](#31-generating-model-files-with-spec-parser)
- [Generating a complete MkDocs configuration file](#32-generating-a-complete-mkdocs-configuration-file)

1. [Building HTML](#4-building-html)
1. [Configuring the website](#5-configuring-the-website)
1. [Specification versions on spdx.github.io](#6-specification-versions-on-spdxgithubio)

## 1. Prerequisites

Apart from Git and Python, you have to have [MkDocs](http://mkdocs.org)
Expand Down Expand Up @@ -83,67 +97,53 @@ pip3 install -r spec-parser/requirements.txt
*(e.g., chapters and annexes), you can skip to [step 4](#4-building-html).*

Model files in `spdx/spdx-3-model` repository are written in a constrained
Markdown format, with a limited set of allowed headings.
Markdown format, with [a predefined set of section headings][format].
The `spec-parser` processes these model files to generate both ontology files
and final Markdown files suitable for MkDocs.

The `spec-parser` also performs automatic formatting on the resulting Markdown
files. For instance, it converts a list under the "Properties" heading into a
table.

[format]: https://github.com/spdx/spdx-3-model/blob/develop/docs/format.md

### 3.1 Generating model files with spec-parser

To verify the formatting of pre-processed model files and
prepare them for MkDocs, run the following command:

```shell
python3 spec-parser/main.py spdx-3-model/model parser_output
python3 spec-parser/main.py \
--generate-mkdocs --output-mkdocs spdx-spec/docs/model/ \
spdx-3-model/model/
```

*(If `parser_output` already exists, the `spec-parser` will not overwrite it.)*

The command will create files in the `parser_output` directory.
Among its subdirectories, we're particularly interested in:

- `parser_output/mkdocs` - Processed Markdown files for MkDocs:
These files (`.md` extension) are located in various
subdirectories and are intended for processing by MkDocs in the next step.
- `parser_output/rdf` - Ontology (RDF) files:
These files (`spdx-context.jsonld`, `spdx-model.json-ld`, `spdx-model.n3`,
`spdx-model.pretty-xml`,`spdx-model.ttl`, `spdx-model.xml`, etc.)
are ready for immediate use.

- `parser_output/mkdocs`: Processed Markdown files (`.md`) for MkDocs.
These files will be used by MkDocs in the next step.
- `parser_output/rdf`: Ontology (RDF) files, including
`spdx-context.jsonld`, `spdx-model.json-ld`, `spdx-model.ttl`, etc.
These files are ready for immediate use.

Additionally, a `parser_output/model-files.yml` file will be generated.
It contains a list of the files within `parser_output/mkdocs`
and will be used for MkDocs configuration later.

### 3.2 Copying the generated files
The command will instruct the spec-parser to read the input from
`spdx-3-model/model/` and generate processed Markdown files (`.md`),
placing them in the `spdx-spec/docs/model/` directory.
These files will then be used by MkDocs.

Copy the processed Markdown files and ontology files to the `docs/` directory:
An `spdx-spec/docs/model/model-files.yml` file will also be generated.
This file contains a list of the files within `spdx-spec/docs/model/`
and will be used later for MkDocs configuration.
We will move this `model-files.yml` file to the `spdx-spec/` directory
for subsequent use:

```shell
mkdir -p spdx-spec/docs/model
mkdir -p spdx-spec/docs/rdf
cp -R parser_output/mkdocs/* spdx-spec/docs/model
cp -R parser_output/rdf/* spdx-spec/docs/rdf
mv spdx-spec/docs/model/model-files.yml spdx-spec/
```

### 3.3 Generate a complete MkDocs configuration file
### 3.2 Generating a complete MkDocs configuration file

To ensure MkDocs recognizes the new Markdown files,
insert the model file list from `parser_output/model-files.yml`
into the MkDocs configuration file in `spdx-spec/mkdocs.yml`.
insert the model file list from `model-files.yml`
into the MkDocs configuration file in `spdx-spec/mkdocs.yml`,
by using this command:

```shell
spdx-spec/bin/make-mkdocs-config.sh \
-b spdx-spec/mkdocs.yml \
-m parser_output/model-files.yml \
-m spdx-spec/model-files.yml \
-f spdx-spec/mkdocs-full.yml
```

Expand All @@ -156,7 +156,7 @@ we will use MkDocs to assemble them into a website.

*Note: all the commands below use the configuration file*
*with the model file list, `mkdocs-full.yml`,*
*generated in [step 3.3](#33-generate-a-complete-mkdocs-configuration-file).*
*generated in [step 3.2](#32-generating-a-complete-mkdocs-configuration-file).*
*If you only want to review the non-model part of the specification*
*(have skipped step 3), please use `mkdocs.yml` instead.*

Expand Down Expand Up @@ -193,7 +193,7 @@ section. The order of filenames in this section determines the order of the
page in the navigation bar.

After you have modified the configuration file, you may need to rerun
[step 3.3](#33-generate-a-complete-mkdocs-configuration-file)
[step 3.2](#32-generating-a-complete-mkdocs-configuration-file)
to incorporate the changes into the complete configuration file.

## 6. Specification versions on spdx.github.io
Expand Down
2 changes: 1 addition & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ nav:
- '4. Terms and definitions': terms-and-definitions.md
- '5. Conformance': conformance.md
- '6. Model and serializations': serializations.md
- model: # __MODEL_PLACEHOLDER__, to be replaced by content from mkdocs-files.yml. Please do not edit this line.
- model: # __MODEL_PLACEHOLDER__, to be replaced by content from model-files.yml. Please do not edit this line.
- "__MODEL_PLACEHOLDER__": "https://example.com" # allows mkdocs.yml to be build/tested without full model list. Please do not edit this line.
- annexes:
- 'A. RDF model definition and diagrams': annexes/rdf-model.md
Expand Down