updated publisher spec #147

0x73746F66 · 2025-05-10T11:58:38Z

TLDR

The TEA Publisher OpenAPI specification has inconsistencies with the consumer API
and documentation, causing potential integration issues and confusion for implementers in the upcoming Beta 1 hackathon on May 28th.

Extended parameter documentation to be more complete
We now have schema examples
Version number updated to 0.0.3 to match consumer API

This PR is focussed on the Publisher API spec, that was mostly unchanged since my last contrinution. I noticed it needed significant updates to align with the latest architecture documentation and all the updated that had been given to the consumer API spec (how would consumers consume things that weren't able to be published? This had to be fixed!)

Incompatibilities with consumer spec

Field Naming Inconsistencies:

Consumer: Uses url in artifact format
Publisher: Uses artifact_url in artifact format

Parameter Structure:

Consumer: Uses id-type/id-value for product queries
Publisher: Uses direct filter parameters

Missing Consumer Endpoints:

/release/{uuid}/collection for latest collection
/release/{uuid}/collections for all collections
/artifact/{uuid} for artifact metadata

Schema Differences:

Consumer requires versions field in component schema
Publisher's component doesn't match this requirement

Changes

Terminology Standardization

Before:

Used leaf endpoints and schemas in publisher API
Inconsistent with consumer API's component/release model

After:

Replaced with proper /component and /release endpoints
Aligned object model with documentation

Why: Consistent terminology across consumer/publisher APIs prevents integration errors.

TEI URN Integration

Before:

Limited TEI support in product endpoints
No validation for TEI URN format

After:

Added tei_urns array to product schema
Implemented URI pattern validation: ^urn:tei:[a-zA-Z0-9]+:[a-zA-Z0-9\\.-]+:.+$

Why: TEI URNs are the primary discovery mechanism according to documentation.

Collection Update Mechanism

Before:

Collection update reasons used undocumented enum values
No clear versioning mechanism

After:

Added proper collection_update_reason schema with values from docs:
- INITIAL_RELEASE
- VEX_UPDATED
- ARTIFACT_UPDATED
- ARTIFACT_ADDED
- ARTIFACT_REMOVED
Added /collection/{uuid}/{version} endpoint

Why: Matches documented collection versioning requirements.

Artifact Structure

Before:

Used inconsistent artifact structure with objects
Single checksum value per artifact

After:

Aligned with consumer API using formats array
Multiple checksums with algorithm type support
Consistent properties with consumer API

Why: Allows full interoperability between consumer/publisher systems.

Component/Release Relationship

Before:

Flat hierarchy with limited relationships between objects

After:

Clear product → component → release → collection hierarchy
Proper reference arrays between object types

Why: Matches documented model and enables proper artifact organization.

Signed-off-by: Chris Langton <[email protected]>

- Add authentication schemes - Create discovery endpoints (/.well-known/tea/{id}) - Standardise property naming between consumer/publisher APIs - Implement advanced filtering for components and artifacts - Add lifecycle status endpoints Signed-off-by: Chris Langton <[email protected]>

spec/publisher/openapi.yaml

ppkarwasz · 2025-05-12T10:25:52Z

@0x73746F66,

Thank you for updating the Publisher spec! 💯 It was certainly way of out of date!

However, I notice that the proposed spec has also a lot of subtle differences (schema names, parameter names, schema contents) from the current Consume API. Should we mark this PR as draft and work on it after the hackaton?

Signed-off-by: Chris Langton <[email protected]>

taleodor

Thank you for the Pull Request! Did first read through for the document, there may be more things to discuss later, but raised issues in the comments that require further work and / or discussion.

spec/publisher/openapi.yaml

taleodor · 2025-05-21T15:19:01Z

spec/publisher/openapi.yaml

+    post:
+      description: Create TEA Product entry for the supplied product identifier
+      operationId: createTeaProduct
+      requestBody:


Similar to above, most of those properties are not currently part of the Consumer API spec. Additionally, it would be helpful to create a shared Schema object as we did throughout the Consumer API.

As above, refer to PURL in the consumer API

Again, don't see how this is relevant to PURL at all.

You're going to have to be clear then, because of you're referring to the markdown spec and find these then perhaps the consumer spec should be updated after beta1 to align like this has

See my previous comment, those are listed in markdown as possible identifiers, we shouldn't have a separate field for each type and instead expand TeaIdentifier. We may also want to be a little stricter about identifier types - it's possible that markdown needs to be updated here as well.

The consumer API returns the constructed PURL as a dynamic response property

To be able to do that, the publisher needs to gather data elements and store them to construct the PURL

A string that is a full PURL can be described by th publisher as a loaded optional string, where some values are duplicated every API call

Or an API spec that implements the intent of returning a partial PURL could do so wth partial PURL support by not including some of the additional elements I added

I believe reduction of duplicates, and addition of fields available in a PURL, is the best way to implement the spec.
If we do anything lese it is either adding duplication, and it is putting the responcibility onto publishers to know how to construct a PURL, and being optional this will probabl be skipped...

If we want to do ourselves a favour later when we implement the PURL support for searching, we should be forward thinking now

The consumer API returns the constructed PURL as a dynamic response property

A single entity may have several PURLs - that is the state of the PURL spec at the moment, I specifically brought this question on the last PURL community meeting. Having consistent and unique PURLs is a noble goal, but I don't think that is achievable.

As a basic example, imagine a Docker image that also has code hosted on GitHub. You may have PURL pkg:github/... and pkg:docker/... for the same thing where both are valid. That is also part of the reason why separate TEI is needed.

Therefore, I believe we should actually leave it to publishers to define their identifiers.

Finally, PURL is very domain (type) specific and things like sku and barcode are not even basic PURL elements, so not sure why they are here in this context.

taleodor · 2025-05-21T15:20:39Z

spec/publisher/openapi.yaml

+          $ref: '#/components/responses/404-object-by-id-not-found'
+      tags:
+        - TEA Component
+  /component/{componentIdentifier}:


Suggest using same name for {uuid} across the spec and refer to the Schema uuid definition as in the Consumer API.

that makes sense is we merge them and the meaning has no distinctions from publisher and consumer. i.e. a publisher does not have the primary key yet (until the database record is inserted) so it cannot be the same entity as the resultant response entity in the consumer API spec.

It is common for request models to be distinct from response models in all API designs - this is why I deveated deliberately

spec/publisher/openapi.yaml

taleodor · 2025-05-21T15:29:59Z

spec/publisher/openapi.yaml

+      $ref: '#/components/operations/standardDelete'
+      tags:
+        - TEA Release
+  /collection:


We should discuss whether explicit CRUD on Collections should be allowed, see #152

A lot of the referenced issue has a mixture of obviously good ideas but I believe that dogma might be winning over critical thinking a little here. Without question why these views are better than allowing publishers the flexibilities is problematic, we automatically impose an arbitrary constraint. Perhaps constraints should be sparingly applied to a stnadard like TEA

For me this is actually purely implementation prospective. I think we should add calls related to adding artifacts, then it becomes more clear that having both CRUD on collections and on artifacts conflicts with each other logically. For now, since artifact CRUD is not there it may be less visible.

CRUD is a por description for this spec, given R for read is not apparent and therefore chaining requests that rely on prior call results with new unknown UUIDs can be a logical constraint which only a read can resolve..

Better to reduce chaining requests for creation, and offer incremental limited updater methods expected to be called at much later dates, dependant on having a timely GET request for freshness

Would you agree a single POST with everything is both concise, organised, and performant? With consideration any complications that can be addressed with a GET + PATCH chain should be back ported to the POST to maintain these desirable characteristics and avoid dependency on complicated chaining logic?

This makes sense to me, but right now we have collection type update enum, which has things like Vex Updated, Artifact Updated, Artifact Removed, Artifact Added. If you do batch requests where you both add and update artifacts, this concept becomes murky.

Another issue with batch requests is that it eventually would lead to conversation about collection own lifecycle - which to me is more bureaucracy without much gain. I prefer a system where I upload an artifact and it becomes visible right away, instead of a system where I need to first upload an artifact (or a batch) and then approve a new collection on top of that. This is a point to discuss though.

It's not only "approve" - it's in many cases "sing the collection" too. It's like a commit.

We should have a conversation on this.

My current implementation idea is that signing should happen automatically on the backend - we can add constraints that artifacts should be signed, etc.

In SaaS scenarios, we can easily have dozens of releases per day for a single component and there can be many components. Each of those releases can have multiple collection versions. I cannot imagine a human signing all of them.

Keyless signing (Cosign signature) is definitely a superior choice, however we'll need to support signing with Tuff, CMK, and PGP

Cosign as a service is not a superior choice for a lot of reasons. The software is cool though. Signing is discussed elsewhere and we need to open that discussion.

spec/publisher/openapi.yaml

Signed-off-by: Chris Langton <[email protected]>

0x73746F66 · 2025-05-22T01:40:23Z

Awesome feedback @taleodor ty. I have some cnversations resolved and others with clarifications t odiscuss

Signed-off-by: Chris Langton <[email protected]>

…release paths Signed-off-by: Chris Langton <[email protected]>

updated publisher spec

d6d01f6

Signed-off-by: Chris Langton <[email protected]>

0x73746F66 force-pushed the main branch from fee32ac to d6d01f6 Compare May 10, 2025 12:08

0x73746F66 added 2 commits May 10, 2025 22:31

more inconsistencies from consumer that needed updates

84781b1

Signed-off-by: Chris Langton <[email protected]>

0x73746F66 marked this pull request as ready for review May 10, 2025 13:16

0x73746F66 requested review from oej and madpah as code owners May 10, 2025 13:16

oej reviewed May 12, 2025

View reviewed changes

spec/publisher/openapi.yaml Outdated Show resolved Hide resolved

oej reviewed May 12, 2025

View reviewed changes

spec/publisher/openapi.yaml Outdated Show resolved Hide resolved

Merge branch 'CycloneDX:main' into main

0665151

0x73746F66 marked this pull request as draft May 21, 2025 10:43

0x73746F66 added 4 commits May 21, 2025 20:49

chore(lint): address long lines

637e29f

Signed-off-by: Chris Langton <[email protected]>

chore: use camelCase instea of snake_case

059fd75

Signed-off-by: Chris Langton <[email protected]>

chore: pointless purl for publisher removed

72746cb

Signed-off-by: Chris Langton <[email protected]>

chore: sync version and resolve name inconsistencies

eac17b6

Signed-off-by: Chris Langton <[email protected]>

taleodor reviewed May 21, 2025

View reviewed changes

0x73746F66 mentioned this pull request May 22, 2025

Component UUID in the release (Consumer) #158

Open

chore: align to Consumer udpate

fafe58a

Signed-off-by: Chris Langton <[email protected]>

0x73746F66 added 2 commits May 22, 2025 23:05

chore: move requestBody into ref components

926b98f

Signed-off-by: Chris Langton <[email protected]>

feat: Refactor to use $ref for operations on product, component, and …

59d2ba5

…release paths Signed-off-by: Chris Langton <[email protected]>

Uh oh!

updated publisher spec #147

Are you sure you want to change the base?

updated publisher spec #147

Uh oh!

Conversation

0x73746F66 commented May 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ppkarwasz commented May 12, 2025

Uh oh!

taleodor left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

0x73746F66 commented May 22, 2025

Uh oh!

Uh oh!

0x73746F66 commented May 10, 2025 •

edited

Loading