Skip to content

Commit 34ef3ca

Browse files
waylanwaylan
authored
Internal Refactor of Pages, Files, and Navigation (mkdocs#1504)
Internal handling of pages, files and navigation has been completely refactored. The changes included in the refactor are summarized below. * Support for hidden pages. All Markdown pages are now included in the build regardless of whether they are included in the navigation configuration (fixes mkdocs#699). * The navigation can now include links to external sites (fixes mkdocs#989, fixes mkdocs#1373, & fixes mkdocs#1406). * Page data (including titles) is properly determined for all pages before any page is rendered (fixes mkdocs#1347). * Automatically populated navigation now sorts index pages to the top. In other words, The index page will be listed as the first child of a directory, while all other documents are sorted alphanumerically by file name after the index page (fixes mkdocs#73 & fixes mkdocs#1042). * A `README.md` file is now treated as an index file within a directory and will be rendered to `index.html` (fixes mkdocs#608). * The URLs for all files are computed once and stored in a files collection. This ensures all internal links are always computed correctly regardless of the configuration. This also allows all internal links to be validated, not just links to other Markdown pages. (fixes mkdocs#842 & fixes mkdocs#872). * An `on_files` plugin event has been added, which could be used to include files not in the `docs_dir`, exclude files, redefine page URLs (i.e. implement extensionless URLs), or to manipulate files in various other ways. Backward incompatible changes are highlighted in the release notes included with this commit. Some notes regarding various decisions follow in no particular order: This started out as the contents of the 'structure' dir from @tomchristie's work in mkdocs#689. All paths must be all Unicode all the time. When a byte string and a Unicode string are both passed to os.path (join ect) then returned value is always a byte string. Therefore, we need every path string to be Unicode. This ensures validation checks that and if the byte string uses the file system encoding, decodes it. For any other encoding, a validation error is raised. Paths which start with a slash are assumed to be relative to the docs_dir root. This behavior fixes mkdocs#192. However, the slash not being present in the output may surprise some users who are trying to create a link relative to the server root when the mkdocs root is not at the server root. The URLs available on a page are: * Page.url is the url relative to the site_dir * Page.canonical_url is the relative url joined with site_url or None if site_url is not defined (the default). * Page.abs_url is the path component of the canonical url or None if canonical_url is None. Note that new behavior is slightly different than before. Previously abs_url ignored site_url and was always url with '' prepended. With the new behavior, if site_url includes a subdir, that subdir will be included in the abs_url. When not on a server, there is no sensable "absolute_url" for a page. Therefore, we shouldn't try to define one. The thinking is that users generating docs to be browsed in the local file system (`file://`) should leave the site_url setting unset, while users who will be serving their docs from a server should be setting the site_url. And if the site_url point sot a subdir of the server, the abs_url will stil be absolute from the server root as it uses the "path" of the canonical_url of the page. Note that without the magical url context all URLs must be prepended by `{{ base_url }}/` in the templates. While this requires a change in third party themes, it is more consistent. Links being ignored in raw HTML is now documented. Fixes mkdocs#991. All related tests that require temp dirs use the `mkdocs.tests.base.tempdir` decorator. Note that any unrelated tests have not yet been updated. That can happen separately from this. The one test in `mkdocs.tests.structure.page_tests` (test_BOM) is unique enough to not use the decorator.
1 parent 3f0e446 commit 34ef3ca

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+4180
-2248
lines changed

‎.coveragerc

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
[report]
2+
show_missing = True

‎docs/about/release-notes.md

+160-35
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,131 @@ The current and past members of the MkDocs team.
2525

2626
### Major Additions to Development Version
2727

28+
#### Internal Refactor of Pages, Files, and Navigation
29+
30+
Internal handling of pages, files and navigation has been completely refactored.
31+
The changes included in the refactor are summarized below.
32+
33+
* Support for hidden pages. All Markdown pages are now included in the build
34+
regardless of whether they are included in the navigation configuration
35+
(#699).
36+
* The navigation can now include links to external sites (#989 #1373 & #1406).
37+
* Page data (including titles) is properly determined for all pages before any
38+
page is rendered (#1347).
39+
* Automatically populated navigation now sorts index pages to the top. In other
40+
words, The index page will be listed as the first child of a directory, while
41+
all other documents are sorted alphanumerically by file name after the index
42+
page (#73 & #1042).
43+
* A `README.md` file is now treated as an index file within a directory and
44+
will be rendered to `index.html` (#608).
45+
* The URLs for all files are computed once and stored in a files collection.
46+
This ensures all internal links are always computed correctly regardless of
47+
the configuration. This also allows all internal links to be validated, not
48+
just links to other Markdown pages. (#842 & #872).
49+
* An [on_files] plugin event has been added, which could be used to include
50+
files not in the `docs_dir`, exclude files, redefine page URLs (i.e.
51+
implement extensionless URLs), or to manipulate files in various other ways.
52+
53+
[on_files]: ../user-guide/plugins.md#on_files
54+
55+
##### Backward Incompatible Changes
56+
57+
As part of the internal refactor, a number of backward incompatible changes have
58+
been introduced, which are summarized below.
59+
60+
###### URLS have changed when `use_directory_urls` is `False`
61+
62+
Previously, all Markdown pages would be have their filenames altered to be index
63+
pages regardless of how the [use_directory_urls] setting was configured.
64+
However, the path munging is only needed when `use_directory_urls` is set to
65+
`True` (the default). The path mungling no longer happens when
66+
`use_directory_urls` is set to `False`, which will result in different URLs for
67+
all pages that were not already index files. As this behavior only effects a
68+
non-default configuration, and the most common user-case for setting the option
69+
to `False` is for local file system (`file://`) browsing, its not likely to
70+
effect most users. However, if you have `use_directory_urls` set to `False`
71+
for a MkDocs site hosted on a web server, most of your URLs will now be broken.
72+
As you can see below, the new URLs are much more sensible.
73+
74+
| Markdown file | Old URL | New URL |
75+
| --------------- | -------------------- | -------------- |
76+
| `index.md` | `index.html` | `index.html` |
77+
| `foo.md` | `foo/index.html` | `foo.html` |
78+
| `foo/bar.md` | `foo/bar/index.html` | `foo/bar.html` |
79+
80+
Note that there has been no change to URLs or file paths when
81+
`use_directory_urls` is set to `True` (the default), except that MkDocs more
82+
consistently includes an ending slash on all internally generated URLs.
83+
84+
[use_directory_urls]: ../user-guide/configuration.md#use_directory_urls
85+
86+
###### The `pages` configuration setting has been renamed to `nav`
87+
88+
The `pages` configuration setting is deprecated and will issue a warning if set
89+
in the configuration file. The setting has been renamed `nav`. To update your
90+
configuration, simply rename the setting to `nav`. In other words, if your
91+
configuration looked like this:
92+
93+
```yaml
94+
pages:
95+
- Home: index.md
96+
- User Guide: user-guide.md
97+
```
98+
99+
Simply edit the configuration as follows:
100+
101+
```yaml
102+
nav:
103+
- Home: index.md
104+
- User Guide: user-guide.md
105+
```
106+
107+
In the current release, any configuration which includes a `pages` setting, but
108+
no `nav` setting, the `pages` configuration will be copied to `nav` and a
109+
warning will be issued. However, in a future release, that may no longer happen.
110+
If both `pages` and `nav` are defined, the `pages` setting will be ignored.
111+
112+
###### Template variables and `base_url`
113+
114+
In previous versions of MkDocs some URLs expected the [base_url] template
115+
variable to be prepended to the URL and others did not. That inconsistency has
116+
been removed. All URLs must now be joined with the `base_url`. As previously, a
117+
slash must be included between the `base_url` and the URL variable. For example,
118+
a theme template might have previously included a link to the `site_name` as:
119+
120+
```django
121+
<a href="{{ nav.homepage.url }}">{{ config.site_name }}</a>
122+
```
123+
124+
And MkDocs would magically return a URL for the homepage which was relative to
125+
the current page. That "magic" has been removed and the `base_url` must now be
126+
explicitly included:
127+
128+
```django
129+
<a href="{{ base_url }}/{{ nav.homepage.url }}">{{ config.site_name }}</a>
130+
```
131+
132+
This change applies to any navigation items and pages, as well as the
133+
`page.next_page` and `page.previous_page` attributes. For the time being, the
134+
`extra_javascript` and `extra_css` variables continue to work as previously
135+
(without `base_url`), but they have been deprecated and the corresponding
136+
configuration values (`config.extra_javascript` and `config.extra_css`
137+
respectively) should be used with `base_url` instead.
138+
139+
Note that navigation can now include links to external sites. Obviously, the
140+
`base_url` should not be prepended to these items. Therefore, all navigation
141+
items contain a `is_link` attribute which can be used to alter the behavior for
142+
external links.
143+
144+
```django
145+
<a href="{% if not nav_item.is_link %}{{ base_url }}/{% endif %}{{ nav_item.url }}">{{ nav_item.title }}</a>
146+
```
147+
148+
Any other URL variables which should not be used with `base_url` are explicitly
149+
documented as such.
150+
151+
[base_url]: ../user-guide/custom-themes.md#base_url
152+
28153
#### Path Based Settings are Relative to Configuration File (#543)
29154

30155
Previously any relative paths in the various configuration options were
@@ -181,7 +306,7 @@ template exists.
181306
##### Context Variables
182307

183308
Page specific variable names in the template context have been refactored as
184-
defined in [Custom Themes](../user-guide/custom-themes/#page). The
309+
defined in [Custom Themes](../user-guide/custom-themes.md#page). The
185310
old variable names issued a warning in version 0.16, but have been removed in
186311
version 1.0.
187312

@@ -199,14 +324,14 @@ user created and third-party templates:
199324
| previous_page | [page.previous_page]|
200325
| next_page | [page.next_page] |
201326

202-
[page]: ../user-guide/custom-themes/#page
203-
[page.title]: ../user-guide/custom-themes/#pagetitle
204-
[page.content]: ../user-guide/custom-themes/#pagecontent
205-
[page.toc]: ../user-guide/custom-themes/#pagetoc
206-
[page.meta]: ../user-guide/custom-themes/#pagemeta
207-
[page.canonical_url]: ../user-guide/custom-themes/#pagecanonical_url
208-
[page.previous_page]: ../user-guide/custom-themes/#pageprevious_page
209-
[page.next_page]: ../user-guide/custom-themes/#pagenext_page
327+
[page]: ../user-guide/custom-themes.md#page
328+
[page.title]: ../user-guide/custom-themes.md#pagetitle
329+
[page.content]: ../user-guide/custom-themes.md#pagecontent
330+
[page.toc]: ../user-guide/custom-themes.md#pagetoc
331+
[page.meta]: ../user-guide/custom-themes.md#pagemeta
332+
[page.canonical_url]: ../user-guide/custom-themes.md#pagecanonical_url
333+
[page.previous_page]: ../user-guide/custom-themes.md#pageprevious_page
334+
[page.next_page]: ../user-guide/custom-themes.md#pagenext_page
210335

211336
Additionally, a number of global variables have been altered and/or removed
212337
and user created and third-party templates should be updated as outlined below:
@@ -286,7 +411,7 @@ the `extra_css` or `extra_javascript` config settings going forward.
286411
##### Page Context
287412

288413
Page specific variable names in the template context have been refactored as
289-
defined in [Custom Themes](../user-guide/custom-themes/#page). The
414+
defined in [Custom Themes](../user-guide/custom-themes.md#page). The
290415
old variable names will issue a warning but continue to work for version 0.16,
291416
but may be removed in a future version.
292417

@@ -304,14 +429,14 @@ user created and third-party templates:
304429
| previous_page | [page.previous_page]|
305430
| next_page | [page.next_page] |
306431

307-
[page]: ../user-guide/custom-themes/#page
308-
[page.title]: ../user-guide/custom-themes/#pagetitle
309-
[page.content]: ../user-guide/custom-themes/#pagecontent
310-
[page.toc]: ../user-guide/custom-themes/#pagetoc
311-
[page.meta]: ../user-guide/custom-themes/#pagemeta
312-
[page.canonical_url]: ../user-guide/custom-themes/#pagecanonical_url
313-
[page.previous_page]: ../user-guide/custom-themes/#pageprevious_page
314-
[page.next_page]: ../user-guide/custom-themes/#pagenext_page
432+
[page]: ../user-guide/custom-themes.md#page
433+
[page.title]: ../user-guide/custom-themes.md#pagetitle
434+
[page.content]: ../user-guide/custom-themes.md#pagecontent
435+
[page.toc]: ../user-guide/custom-themes.md#pagetoc
436+
[page.meta]: ../user-guide/custom-themes.md#pagemeta
437+
[page.canonical_url]: ../user-guide/custom-themes.md#pagecanonical_url
438+
[page.previous_page]: ../user-guide/custom-themes.md#pageprevious_page
439+
[page.next_page]: ../user-guide/custom-themes.md#pagenext_page
315440

316441
##### Global Context
317442

@@ -400,7 +525,7 @@ overriding blocks in the same manner as the built-in themes. Third party themes
400525
are encouraged to wrap the various pieces of their templates in blocks in order
401526
to support such customization.
402527

403-
[blocks]: ../user-guide/styling-your-docs/#overriding-template-blocks
528+
[blocks]: ../user-guide/styling-your-docs.md#overriding-template-blocks
404529

405530
#### Auto-Populated `extra_css` and `extra_javascript` Deprecated. (#986)
406531

@@ -444,7 +569,7 @@ the `docs_dir` is set to the directory which contains your config file rather
444569
than a child directory. You will need to rearrange you directory structure to
445570
better conform with the documented [layout].
446571

447-
[layout]: ../user-guide/writing-your-docs/#file-layout
572+
[layout]: ../user-guide/writing-your-docs.md#file-layout
448573

449574
### Other Changes and Additions to Version 0.16.0
450575

@@ -522,8 +647,8 @@ See the documentation for [Styling your docs] for more information about using
522647
and customizing themes and [Custom themes] for creating and distributing new
523648
themes
524649

525-
[Styling your docs]: /user-guide/styling-your-docs.md
526-
[Custom themes]: /user-guide/custom-themes.md
650+
[Styling your docs]: ../user-guide/styling-your-docs.md
651+
[Custom themes]: ../user-guide/custom-themes.md
527652

528653
### Other Changes and Additions to Version 0.15.0
529654

@@ -544,9 +669,9 @@ themes
544669
* Bugfix: Provide filename to Read the Docs. (#721 and RTD#1480)
545670
* Bugfix: Silence Click's unicode_literals warning. (#708)
546671

547-
[site_description]: /user-guide/configuration.md#site_description
548-
[site_author]: /user-guide/configuration.md#site_author
549-
[ReadTheDocs]: /user-guide/styling-your-docs.md#readthedocs
672+
[site_description]: ../user-guide/configuration.md#site_description
673+
[site_author]: ../user-guide/configuration.md#site_author
674+
[ReadTheDocs]: ../user-guide/styling-your-docs.md#readthedocs
550675

551676
## Version 0.14.0 (2015-06-09)
552677

@@ -604,16 +729,16 @@ This new file is created on every MkDocs build (with `mkdocs build`) and
604729
no configuration is needed to enable it.
605730

606731
[future release]: https://github.com/mkdocs/mkdocs/pull/481
607-
[site_dir]: /user-guide/configuration.md#site_dir
732+
[site_dir]: ../user-guide/configuration.md#site_dir
608733

609734
#### Change the pages configuration
610735

611736
Provide a [new way] to define pages, and specifically [nested pages], in the
612737
mkdocs.yml file and deprecate the existing approach, support will be removed
613738
with MkDocs 1.0.
614739

615-
[new way]: /user-guide/writing-your-docs.md#configure-pages-and-navigation
616-
[nested pages]: /user-guide/writing-your-docs.md#multilevel-documentation
740+
[new way]: ../user-guide/writing-your-docs.md#configure-pages-and-navigation
741+
[nested pages]: ../user-guide/writing-your-docs.md#multilevel-documentation
617742

618743
#### Warn users about the removal of builtin themes
619744

@@ -631,7 +756,7 @@ JavaScript library [lunr.js]. It has been added to both the `mkdocs` and
631756
for adding it to your own themes.
632757

633758
[lunr.js]: http://lunrjs.com/
634-
[supporting search]: /user-guide/styling-your-docs.md#search-and-themes
759+
[supporting search]: ../user-guide/styling-your-docs.md#search-and-themes
635760

636761
#### New Command Line Interface
637762

@@ -659,10 +784,10 @@ can also use Jinja2 syntax and take advantage of the [global variables].
659784
By default MkDocs will use this approach to create a sitemap for the
660785
documentation.
661786

662-
[extra_javascript]: /user-guide/configuration.md#extra_javascript
663-
[extra_css]: /user-guide/configuration.md#extra_css
664-
[extra_templates]: /user-guide/configuration.md#extra_templates
665-
[global variables]: /user-guide/styling-your-docs.md#global-context
787+
[extra_javascript]: ../user-guide/configuration.md#extra_javascript
788+
[extra_css]: ../user-guide/configuration.md#extra_css
789+
[extra_templates]: ../user-guide/configuration.md#extra_templates
790+
[global variables]: ../user-guide/styling-your-docs.md#global-context
666791

667792
### Other Changes and Additions to Version 0.13.0
668793

@@ -679,8 +804,8 @@ documentation.
679804
called index.md (#535)
680805
* Bugfix: Fix errors with Unicode filenames (#542).
681806

682-
[extra config]: /user-guide/configuration.md#extra
683-
[Markdown extension configuration options]: /user-guide/configuration.md#markdown_extensions
807+
[extra config]: ../user-guide/configuration.md#extra
808+
[Markdown extension configuration options]: ../user-guide/configuration.md#markdown_extensions
684809
[wheels]: http://pythonwheels.com/
685810

686811
## Version 0.12.2 (2015-04-22)

0 commit comments

Comments
 (0)