STAC Sprint Spec Ideas

[cholmes]

A few of us have been discussing the idea of a 'spec' / recommendations for stac & geoparquet.

Some thoughts from the first hour:

• People agreed that we should not stick JSON into Parquet, but should embrace the Parquet structures to represent things, as we'll get better tooling support in the parquet ecosystem, and likely better compression, etc. The use case of being able to easily get JSON out of a stac-geoparquet though is a valid one - we shouldn't have to force everyone to understand the deep structures if they're comfortable in JSON. So the library should support easy export in the ways that people want it. stac-geoparquet does support export to item-collections today, but we could explore other export options that people want.
• Discussion of being able to round-trip with arrow, which is likely just sprinkling a bit of arrow metadata. This would not be required, but would be nice to have mention of. Ideally we'd stick to pyarrow for the stac-geoparquet tool and skip pandas entirely for this tool, as it'll make things go much faster.
• Discussion of how if you feed a dataset with lots of collections into it then stac-geoparquet gets lot of empty assets. Agreed that the recommendation should be that you make one geoparquet file per collection, so you have a tighter 'schema' definition for your geoparquet. For many open data sets this will be a bit weird, as the number of rows per collection is small, like 5 or 50. But it's likely still useful to treat them all as one file. And users could choose to combine those geoparquets and do analysis, but it'd be better to keep the schemas tight and mapped to a single collection. This is what @m-mohr did with Planet data.
• Discussion of what is top level vs nested structures. General agreement that upgrading all properties to the first level makes sense, as @TomAugspurger did in stac-geoparquet. For Planet I also pushed on putting the 'self' link and a stac-browser link at the top level. The use case is for someone to load it in QGIS, browse around, and then be able to view the properties, and click on one that takes them to STAC Browser where they can get a COG or an XYZ link so they can get the actual data into QGIS. Ideally there is a public stac browser to count on, and then clients could just create the browser link on their own (but still could be useful to include it so even naive clients get a link). Kyle pointed out that this would create a lot of extra data that won't compress well as each is unique. I think it'd just be an optional field, if the provider wanted to include it. The other potential field is the 'source' link, so you could construct the browser. It'd be the 'self' url that's nested, but put at the root level for ease of access.

[TomAugspurger]

All this sounds great! I'll be available to review PRs if any come in :)

[m-mohr]

The source link makes sense and if you omit the links (or just the self link) and instead just add a source column, the data size actually decreases. I'm not so sure whether the browser link is required as you can usually easily derive a browser link from the source column or just copy/paste it into a multi-catalog STAC Browser deployment such as the STAC Browser demo.