Combining metadata from multiple sources in a folders setup

[dominique-pfister]

This is for sites using the template folders feature. With that feature, paths are mapped virtually, that is the target documents do not exist, but only a template document with placeholders.

Currently, such sites use a metadata workbook in the root folder, where the metadata for every virtual path is represented as a row in that workbook, indexed by its path URL.

It would be nice to have a second metadata source for such virtual paths, particularly for situations where the metadata types are fed by different teams.

[rofe]

An example where this could be very useful: Authors want to create a potentially large amount of pages, but instead of creating single documents for each, they would like to use a spreadsheet to manage the content for this set of pages, and have a content life cycle that's independent from the global metadata sheet where access control is often very strict. This can be done using folder mapping today, but the JSON with the metadata needs to be loaded on the client side and adds a performance tax. Loading it as part of the rendering process will eliminate the additional request.

In order to achieve this, a root path for the folder mapping is defined in fstab.yaml, and each mapping specifies the path of a default resource to be rendered (must exist as a document), as well as the path to a JSON file for metadata (or content). By default, this latter file would be generated from a spreadsheet in the same system, but could theoretically come from anywhere (commerce system etc).

Feedback on suggested naming above:

This is for sites using the template feature

It's referred to as folders in its current form. The sheet support would simply be an extension of that schema: a folder could be a string (like today) or an object which defines the default resource as well as an optional sheet to pre-fill content/metadata.

indexed by its path

For consistency with the current metadata sheet, I would reuse URL here instead of path.

[dominique-pfister]

OK, so we'll support both. This will require a fstab schema update in helix-shared-config, it currently expects entries underneath folders to be strings.

[dominique-pfister]

see: adobe/helix-shared#786

[davidnuescheler]

let's try the implied metadata location first (${resource-path}/metadata.json), i don't think we need to initially support both... it seems to be simple, intuitive and consistent with the metadata.json in the root folder.

i take it we would apply the metadata in the root folder as well... meaning that if there is a /** pattern it would also be applied.

[rofe]

I'm fine with starting with an implied metadata location based on the resource path, I just wanted to point out that this would prevent having multiple (smaller) sheets in combo with a single resource, e.g. 1 sheet per language.

[tripodsan]

I just wanted to point out that this would prevent having multiple (smaller) sheets in combo with a single resource, e.g. 1 sheet per language.

I think the folder metadata should be on the mapping root, eg: /products/de/metadata.json

[rofe]

Cache invalidation

Changes (preview/publish) to these folder based metadata sheets should invalidate all pages matching the folder prefix. Can we create a surrogate key to link these "synthetic pages" to their folder prefix?

[tripodsan]

yes. we need to add an additional key, probably something like${hash(mappingRoot)}_metadata and then also check during purging if the metadata.json is not the root one. this also is easier, if we just use a default location instead of a configurable one

[dominique-pfister]

Work to do:

• Load mapped metadata in helix-html-pipeline
• Purge mapped metadata in helix-admin
• Release new helix-pipeline-service (7.3.38)

[dominique-pfister]

All work done, feature is available.