Bug: Pushing change to extant pages with build folder present causes deletion of all but affected pages

[SamPlatek]

Setup:

• Have multi-page confluence deployed
• Run make confluence locally
• Make change to any file
• Run make confluence again
• All pages except the edited one will be deleted

Changing setting s does not appear to affect the outcome. I have changed all of them aside from the Auth ones. Only deleting the previous build artifacts works

[jdknight]

Edit: This comment is outdated; please see the next comment.

---

I wonder if this is a combination of having purging enabled with the new "only update new pages" introduced in v2.1. Haven't had time to test that is the case, but a possible workaround is to set the following option for the project's configuration:

confluence_publish_force = True

This will force publication of all pages relevant to the project, but should not purge any documents in the configured root.

[jdknight]

To start off, a workaround for this use case would be to specify an ignore-environment argument when building Confluence documentation. This does trigger local rebuilds of sources, but will ensure all documents are published/updates on the target Confluence instance:

make confluence -E

---

This issue is a combination of little/incorrect environment processing performed by this extension.

1. First off, this extension make an adjustment (8c23c3d) to configuration triggering with an attempt to have Confluence-related changed options only trigger document updates when targeting the Confluence builder. While these options were flagged as confluence only, Sphinx engine itself does not do anything with that specific value. When this extension's get_outdated_docs is processing, it needs to internally manage a build configuration cache/hash based on its own custom tags (i.e. confluence) and if a change is detected, indicate all documents are outdated. This needs to be introduced.
2. This extension never really relied that much on the outdated document support. In general, the -E argument was used/promoted and documents were always processed (since the idea of the time was we never knew what Confluence's state was, so we always wanted to force process/update pages). If the environment from previous builds are used and documents are detected as "no change", they are skipped (since this extension does change file timestamps for changes). The problem with this is that it breaks cleanup-related options. For example, when purging is enabled, we compiled a list of page identifiers that could be outdated. As we publish updates, we remove any detected identifier as we cycle through the documents. However, this only applied to outdated documents, which in these use cases, are then undesirably purged afterwards.

What this extension should do is most likely rewrite its get_outdated_docs implementation to completely re-handle how it managed outdated documents. This includes:

• Create a configuration hash of Confluence-related options when performing a build, and if a rebuild detects it's changed, flag all documents as outdated.
• Re-examine the configuration options to ensure the use of confluence and env are properly utilized.
• When publishing pages, keep track of a hash of the original page's content and the Confluence ID it was last published to. When we detect that a page is not considered outdated from a matching hash, if a Confluence ID is associated with it, do not consider that Confluence page when tracking cleanup targets.

And in theory, for any stale/undesired out-of-date publishing scenarios that could occur for corner cases when using the above, users could always apply the -E argument to workaround any cache issues.

[jdknight]

Changes have been introduced (#843) to prevent the removal of up-to-date pages on rebuilds. This extension should now have better support when dealing with cached environments of Sphinx (both managing re-publish events and triggering rebuilds of outdated documents from configuration changes).

These changes will be made available next release (v2.3), and can be tested now by using the development version. Feel free to try it out ahead of time any report any issues you may have.

[jdknight]

v2.3 is now available on PyPI -- marking as closed.