Proposal: lightweight complement to direct submission to SmartAPI

[cmungall]

I propose a system that leverages SmartAPI but overcomes some of the difficulties some have had with registering (difficulty aligning allowed fields, smartAPI is very particular about what fields are allowed in the swagger.json see SmartAPI/smartapi_registry#1, difficulties with swagger.json generated directly from programmatic annotations e.g. flask, difficulties with non-RPC style APIs).

In this repo, in a directory called registry, we have a yaml file per API. The spec for this is TBD, but it would minimally include

• a short unique name
• a URL for the endpoint
• a URL for endpoint metadata (i.e. swagger.json for OpenAPI v2/3; graphql schema)
• the typo of API / framework (e.g OpenAPI, GraphQL, SPARQL endpoint, even open SQL port)
• a brief textual description
• github name of contact person
• github repo for source of API

The goal is to encourage metadata directly within the API endpoint in a standard way, but to have some minimal subset useful for basic indexing.

It would then be fairly simple to have a CI job that sweeps this repo and attempts to register the API with SmartAPI (unless there is a tag in the yaml that says not to do this, e.g. if manual registration is preferred). It would produce collected error reports for any that could not be registered. This would be useful to SmartAPI developers to pinpoint pain points. But crucially, we would still have the API registered and visible to all project members even if the checks fail.

This pull mechanism is very similar to what we developed for registering ontologies on http://obofoundry.org, it's lightweight and has the advantage of tracking everything in github. Anyone is free to make PRs on any of the registry entries, but only members of this repo with permissions would be able to merge these.

[selewis]

+1

[briandoconnor]

+1

Does this mean my OpenAPI definition can be stock with the extended info needed for the smartAPI stored in the registry file? Or are these extra fields still embedded in the OpenAPI yaml doc? Has anyone run into issues with OpenAPI-based tooling not working with the extended fields present? Or is this not a concern?

[david4096]

For reference as a similar approach:

https://github.com/APIs-guru/graphql-apis

https://github.com/APIs-guru/openapi-directory

[micheldumontier]

This is definitely not the answer. Given that we have made previous investments in building an API repository (http://smart-api.info) that are built on existing standards (OpenAPI v3/smartAPI), I strongly recommend that we use this to our advantage rather than inventing yet another metadata standard that currently has no infrastructure or indexing around it. The only difference between graphql and openAPI is that openAPI includes the connection info. In fact, given their similarity, you can express a graphQL API as an openAPI, and vice versa.
for instance:
https://github.com/yarax/swagger-to-graphql

if there are additional metadata needs, then we can accomodate them by expanding the smartAPI specification. feel free to start tickets on the smartAPI issue tracker (https://github.com/SmartAPI/smartapi_registry/issues) where we can discuss the merits of new metadata fields.

[ctb]

Hi @micheldumontier thanks for your comment! For a counterpoint: I see value in other aspects of this proposal (the pull request mechanism, CI, bulk registration attempts with SmartAPI) and also note that we have a large developer community that needs to move fast. I suspect we all agree that SmartAPI is the right end, but it may be useful for us to innovate rapidly and converge with SmartAPI over time. This will probably reduce frustration on both sides during that period of convergence. But that's just my 2c and perhaps I'm misunderstanding!

[micheldumontier]

ok, so here's the minimum you need for the openAPI which covers the elements listed above:
https://gist.github.com/micheldumontier/785cee1ef746dcf46a57ea93e6d25396

to be a minimal smartAPI you need a few more elements, primarily terms of service, and a summary on the operation

https://gist.github.com/micheldumontier/ba7c1791bcfceb6e39504e92c229b8af

of course, there are many more desirable elements beyond the minimal
https://github.com/SmartAPI/smartAPI-Specification/blob/OpenAPI.next/versions/smartapi-list.md

[cmungall]

Thanks for your comments @micheldumontier

Just some brief comments now. The intent is not to set up a different standard - think of this as an alternate submission system or staging system for submission to SmartAPI that allows us to rapidly collect what we need and work out the issues later.

Regarding graphql - I'm suspicious of automatic conversion as there are many fundamental differences. However, this may be a red herring. It may be the case that 0% of DCPPC participants are producing anything other than OpenAPI APIs, we don't know yet. I diverted this back into the main SmartAPI tracker since it seems of broad interest to the SmartAPI community: SmartAPI/smartapi_registry#3

[micheldumontier]

@cmungall - If we start on the smartAPI path, it will force at least a minimal amount of metadata through validation (failing to be in the registry until it is done correct is a good thing^tm). we might also think of a "DCPPC" profile, for which we develop a json schema to do validation with that covers the elements that the community feels are important. I'd much rather go down this path, and am very open to extending the spec accordingly.
thanks for starting the issue - i'm eager to graphql, sparql, other models in the context of the smartAPI!

[cmungall]

@briandoconnor, you said

Has anyone run into issues with OpenAPI-based tooling not working with the extended fields present

Can you post a new ticket in this repo describing the issues you had when registering? This would be really helpful to the smartAPI team