Merge remote-tracking branch 'origin/main' into feat/protect-renderers

.github/workflows/main.yml

@@ -216,11 +216,10 @@ jobs:
           - { is_pr: true }
         include:
           - { texlive: latest }
-    runs-on: self-hosted
+    runs-on: ${{ github.event_name == 'pull_request_target' && 'self-hosted' || 'ubuntu-latest' }}
     container:
       image: ghcr.io/witiko/markdown:${{ needs.build-docker-image.outputs[matrix.texlive] }}
     if: ${{ !contains(inputs.runTests, 'false') }}
-    timeout-minutes: 90
     steps:
       - name: Checkout repository
         uses: actions/checkout@v4

CHANGES.md

@@ -1,6 +1,91 @@
 # Changes
 
-## 3.8.0
+## 3.8.1 (2024-11-03)
+
+Fixes:
+
+- Fix backslashes at the ends of hybrid documents.
+  (#502, #503, contributed by @lostenderman)
+
+- Fix hard line breaks in fancy list items.
+  (#508, #509, contributed by @lostenderman)
+
+## 3.8.0 (2024-10-31)
+
+Development:
+
+- Add support for versioned themes and add new Lua option `experimental`.
+  (#466, #512, #514, [matrix.org][matrix-514] reviewed by @TeXhackse, #521)
+
+  The option `experimental` enables experimental features that are planned to
+  be the new default in the next major release of the Markdown package.
+
+  At the moment, this just means that the version `experimental` of the theme
+  `witiko/markdown/defaults` will be loaded and warnings for hard-deprecated
+  features will become errors. However, the effects may extend to other areas
+  in the future as well.
+
+ [matrix-514]: https://matrix.to/#/!UeAwznpYwwsinVTetR:matrix.org/$TTc-m7B5NSdsLBNNyIuFWQ-u2nOZ03lJ5js88hnyFiU?via=matrix.org&via=im.f3l.de
+
+- Add first-class support for YAML documents. (#452, #473, #524)
+  - Add plain TeX macros `\yamlSetup`, `\yamlInput`, `\yamlBegin`, and `\yamlEnd`.
+  - Add LaTeX environment `yaml` and redefine command `\yamlInput` for LaTeX.
+  - Add ConTeXt commands `\setupyaml`, `\inputyaml`, `\startyaml`, and `\stopyaml`.
+
+Documentation:
+
+- Document LaTeX hooks. (#464, #507)
+
+Defaults:
+
+- Improve the compatibility of the default LaTeX packages with PDF tagging:
+  (#466, #512, #514, #521, reported and consulted by @u-fischer)
+
+  - In TeX engines other than LuaTeX, use the package soul instead of the
+    package soulutf8 in TeX Live ≥ 2023.
+
+  - In LuaLaTeX, use the package lua-ul for strike-through/mark renderer
+    prototypes instead of the package soul.
+
+  - Use the package enumitem for tight and fancy lists instead of the package
+    paralist.
+
+    This is a breaking change that is marked as experimental. To enable it,
+    either use the package option `experimental` or specify any test phase in
+    the document metadata:
+
+    1. `\usepackage[experimental]{markdown}`
+    2. `\DocumentMetadata{testphase=phase-III}`
+
+- Define LaTeX renderers for image identifiers.
+  (#520, suggested by @jurf, 6f3dcd0c, 478530f, [matrix.org][matrix-520])
+
+  This establishes a reliable method for authors to reference figures within Markdown:
+
+  ``` tex
+  \documentclass{article}
+  \usepackage[link_attributes, relative_references]{markdown}
+  \begin{document}
+  \begin{markdown}
+
+  ![example image](example-image "An example image"){#example-image-id}
+
+  See Figure <#example-image-id>.
+
+  \end{markdown}
+  \end{document}
+  ```
+
+- Set correct category codes for version 3 of the package minted. (39980934)
+
+Continuous Integration:
+
+- Only use self-hosted runners for the quick CI in pull requests.
+  For full CI outside pull requests, use GitHub-hosted runners.
+  (c2697428, [matrix.org][matrix-950e8269])
+
+ [matrix-950e8269]: https://matrix.to/#/!pznomuvubVyxElflTe:matrix.org/$op_onsRdhtXJsZB5d5LZrOFF75T02XvKehmikdoucx4?via=matrix.org&via=im.f3l.de
+ [matrix-520]: https://matrix.to/#/!lWGKeMcpgwGHpfLYkf:matrix.org/$mZB-_YfreLFSKU5eDzp0AIvMnsV_Bn-rpUoj9HtUVwk?via=matrix.org&via=im.f3l.de
 
 Fixes:
 
@@ -131,7 +216,7 @@ Continuous Integration:
 
 - Only test latest TeX Live in pull requests. (81927ca1)
 
-Unit Tests:
+Tests:
 
 - Test the `outputDir` plain TeX option. (#457, #459)
 
@@ -309,7 +394,7 @@ Default Renderer Prototypes:
 - Do not unnecessarily override math renderer prototypes in LaTeX/ConTeXt.
   (#387, #396, contributed by @zousiyu1995)
 
-Unit Tests:
+Tests:
 
 - In pull requests, process added and modified testfiles first.
   (feafe9b9, 9ff530da, 18deae73)
@@ -413,7 +498,7 @@ Documentation:
 
  [matrix-357]: https://matrix.to/#/!efVbynJpCMjlOTfose:matrix.org/$WR3quuO1fyOl6w_KX6YFA4BACBcL-cWUWBpVfpPWDkI?via=matrix.org&via=im.f3l.de
 
-Unit Tests:
+Tests:
 
 - Restore CommonMark testfiles with trailing tabs and spaces.
   (#348, #353, lostenderman/markdown#2, contributed by @lostenderman)
@@ -456,7 +541,7 @@ Deprecation:
 
  [matrix-351]: https://matrix.to/#/!lWGKeMcpgwGHpfLYkf:matrix.org/$yVeWB0Yzfz6mgORv5VeG9IEl7a4xLogkAUmdNMSCxEg?via=matrix.org&via=im.f3l.de
 
-Unit Tests:
+Tests:
 
 - Add support for YAML metadata in testfiles.
   (#345, #347, [matrix.org][matrix-347])
@@ -503,7 +588,7 @@ Documentation:
   (f3efcb9)
 - Document setting up catcodes in Lua. (#329, #342)
 
-Unit Tests:
+Tests:
 
 - Batch unit tests to improve speed.
   (#245, #316, 8bfd0b3, #317, #319..#325, #327, #328, e3b31696,
@@ -537,7 +622,7 @@ Documentation:
 - Add a link to TUG 2023 slides and video to the README.
   (ce6047b8, adb69e2f)
 
-Unit Tests:
+Tests:
 
 - Rewrite the unit testing framework from Bash to Python
   and implement summarization. (#245, #314)
@@ -715,7 +800,7 @@ Default Renderer Prototypes:
 - In LaTeX and ConTeXt, use just first word of infostring to determine fence
   code block language. (#244)
 
-Unit Tests:
+Tests:
 
 - Do not fold tabs and spaces into a single space token.
   (lostenderman#107, #242)
@@ -1244,7 +1329,7 @@ Licensing:
 - Bump the copyright year. (9462f17)
 - Sublicense the code as LPPL v1.3c. (609aeee)
 
-Unit tests:
+Tests:
 
 - Remove xtrace from `test.sh` for less verbose output. (b307ee6)
 - Treat all files as text with GNU diff in `test.sh`. (ec89e6d)

Makefile

@@ -20,6 +20,7 @@ EXAMPLES_SOURCES=examples/context-mkiv.tex \
   examples/optex.tex
 EXAMPLES=examples/context-mkiv.pdf \
   examples/latex-pdftex.pdf examples/latex-xetex.pdf examples/latex-luatex.pdf \
+  examples/latex-tex4ht.html examples/latex-tex4ht.css \
   examples/optex.pdf
 TESTS=tests/test.sh tests/test.py tests/requirements.txt tests/support/*.tex \
   tests/templates/*/*/head.tex.m4 tests/templates/*/*/body.tex.m4 \
@@ -156,7 +157,7 @@ $(LIBRARIES): force
 # This target typesets the manual.
 $(TECHNICAL_DOCUMENTATION): $(DTXARCHIVE) $(TECHNICAL_DOCUMENTATION_RESOURCES)
 	latexmk -silent $< || (cat $(basename $@).log 1>&2; exit 1)
-	test `tail $(basename $<).log | sed -rn 's/.*\(([0-9]*) pages.*/\1/p'` -ge 380
+	test `tail $(basename $<).log | sed -rn 's/.*\(([0-9]*) pages.*/\1/p'` -ge 450
 
 # This pseudotarget continuously typesets the manual.
 preview: $(DTXARCHIVE) $(TECHNICAL_DOCUMENTATION_RESOURCES)

README.md

@@ -147,8 +147,8 @@ For further information, consult one of the following:
     - [Markdown 2.17.1: What's New, What's Next?][tb135],
     - [Attributes in Markdown][tb136],
     - [Markdown 3 at TUG 2023: Reflections from the Q&A session][tb138],
-    - Fast Regression Testing of TeX Packages: The Unreasonable Effectiveness of Batching ([preprint][tb139]), and
-    - Markdown Themes in Practice ([preprint][tb140-preprint], [work in progress][tb140]).
+    - [Fast Regression Testing of TeX Packages: The Unreasonable Effectiveness of Batching][tb139], and
+    - [Markdown Themes in Practice][tb140].
 5. Journal articles published by [CSTUG Bulletin][csbul] (in Czech and Slovak):
     - [Rendering Markdown inside TeX Documents][10.5300/2016-1-4/78],
     - [Markdown 2.8.1: Boldly Unto the Throne of Lightweight Markup in TeX][10.5300/2020-1-2/48],
@@ -183,7 +183,7 @@ For further information, consult one of the following:
  [tb135]: https://www.tug.org/TUGboat/tb43-3/tb135novotny-markdown.pdf                                                         "Markdown 2.17.1: What's New, What's Next?"
  [tb136]: https://www.tug.org/TUGboat/tb44-1/tb136novotny-markdown-attr.pdf                                                    "Attributes in Markdown"
  [tb138]: https://www.tug.org/TUGboat/tb44-3/tb138starynovotny-markdown-qa.pdf                                                 "Markdown 3 at TUG 2023: Reflections from the Q&A session"
- [tb139]: https://github.com/Witiko/fast-regression-testing/releases/download/latest/tb139starynovotny-testing.pdf             "Fast Regression Testing of TeX Packages: The Unreasonable Effectiveness of Batching"
+ [tb139]: https://www.tug.org/TUGboat/tb45-1/tb139starynovotny-testing.pdf                                                     "Fast Regression Testing of TeX Packages: The Unreasonable Effectiveness of Batching"
  [tb140]: https://github.com/Witiko/markdown-themes-in-practice/releases/download/latest/tb140starynovotny-markdown-themes.pdf "Markdown Themes in Practice"
 
  [tb131-slides]:       https://tug.org/tug2021/assets/pdf/tug2021-novotny-slides.pdf                            "Markdown 2.10.0: LaTeX Themes & Snippets, Two Flavors of Comments, and LuaMetaTeX"

examples/Makefile

@@ -30,6 +30,7 @@ LUACLI_OPTIONS=\
 OUTPUT=\
   context-mkiv.pdf \
   latex-pdftex.pdf latex-xetex.pdf latex-luatex.pdf \
+  latex-tex4ht.html latex-tex4ht.css \
   optex.pdf
 
 # This is the default pseudo-target.

examples/context-mkiv.tex

@@ -57,6 +57,37 @@
 
 \starttext
 
+% Set and typeset the document metadata using a YAML metadata block.
+\setupyaml
+  [
+    jekyllDataRenderers = {
+      title = {\gdef\doctitle{#1}},
+      author = {\gdef\author{#1}},
+      date = {\gdef\date{#1}},
+    },
+    renderers = {
+      jekyllDataEnd = {%
+        \startalignment[center]
+          \blank[force,2*big]
+            {\tfd \doctitle}
+          \blank[3*medium]
+            {\tfa \author}
+          \blank[2*medium]
+            {\tfa \date}
+          \blank[3*medium]
+        \stopalignment
+      },
+    },
+  ]
+
+\startyaml
+
+title:  An Example *Markdown* Document
+author: Vít Starý Novotný
+date:   `\currentdate`{=tex}
+
+\stopyaml
+
 % Typeset the document `example.md` by letting the Markdown package handle
 % the conversion internally. Optionally, we can specify additional options
 % between the square brackets similarly to the command `\setupmarkdown`.

examples/latex-luatex.tex

@@ -12,7 +12,6 @@
   fencedCode,
   hashEnumerators,
   inlineNotes,
-  jekyllData,
   lineBlocks,
   notes,
   pipeTables,
@@ -28,13 +27,11 @@
   texMathSingleBackslash,
 ]{markdown}
 % Set the document metadata using a YAML metadata block.
-\begin{markdown}[hybrid]
----
+\begin{yaml}
 title:  An Example *Markdown* Document
 author: Vít Starý Novotný
-date:   \today
----
-\end{markdown}
+date:   `\today`{=tex}
+\end{yaml}
 \begin{document}
 % Typeset the document `example.md` by letting the Markdown package handle
 % the conversion internally.

examples/latex-pdftex.tex

@@ -13,7 +13,6 @@
   fencedCode,
   hashEnumerators,
   inlineNotes,
-  jekyllData,
   lineBlocks,
   notes,
   pipeTables,
@@ -29,13 +28,11 @@
   texMathSingleBackslash,
 ]{markdown}
 % Set the document metadata using a YAML metadata block.
-\begin{markdown}[hybrid]
----
+\begin{yaml}
 title:  An Example *Markdown* Document
 author: Vít Starý Novotný
-date:   \today
----
-\end{markdown}
+date:   `\today`{=tex}
+\end{yaml}
 \begin{document}
 % Typeset the document `example.md` by letting the Markdown package handle
 % the conversion internally.