Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pgstacrs #523

Open
gadomski opened this issue Nov 12, 2024 · 2 comments · May be fixed by #524
Open

pgstacrs #523

gadomski opened this issue Nov 12, 2024 · 2 comments · May be fixed by #524
Labels
[crate] pgstac pgstac enhancement New feature or request

Comments

@gadomski
Copy link
Member

gadomski commented Nov 12, 2024

Overview

pgstacrs will be a "full access" Python API for pgstac backed by the pgstac crate.

Prior art

  • pypgstac is a Python package that interacts with a pgstac database. It's mostly used via it's command-line interface (CLI). It's primary issue is that it is currently tightly-coupled to the pgstac version, meaning that its "impossible" to release new Python package features without releasing a new database schema version 😢. There is (in my opinion) a good argument for keeping Rust code out of the pgstac repo, because that repo does enough stuff already.
  • stac-fastapi-pgstac does a lot of custom SQL queries to pgstac PostgreSQL functions that could be refactored out to their own API. The goal here would be for stac-fastapi-pgstac to contain zero SQL 🤯

Integration and goals

The plan is that pgstacrs will live "above" pypgstac and stac-fastapi-pgstac in the dependency chain:

graph LR
    pgstacrs --> pypgstac
    pgstacrs --> stac-fastapi-pgstac
Loading

To do this, pgstacrs will:

  • Expose every public pgstac PostgreSQL function via a Python function/method
  • Provide a pooled client that can be used directly from stac-fastapi-pgstac for efficient server-side queries
  • Provide "smart loading" that can be used via the pypgstac CLI or via a Python API to load collections and items into the database The Right Way™
  • Provide backward compatibility, meaning that (from some point) it will continue to support older pgstac versions
  • Integrate with pystac objects

Examples

Here's some rough sketches of what pgstacrs might look like:

from pgstacrs import Client
from pystac import Collection

client = Client.open("postgresql://username:password@localhost:5432/pgstac")
print("pgstac version:", client.version())

collection = Collection.from_file(
    "https://raw.githubusercontent.com/radiantearth/stac-spec/refs/heads/master/examples/collection.json"
)
items = list(collection.get_items())
client.add_collection(collection)
client.add_items(items)

client = Client.open("postgresql://username:password@localhost:5432/pgstac", pool=True)
items = client.search(collections=["collection-id"], filter={}, intersects={}) # etc
@gadomski gadomski added enhancement New feature or request [crate] pgstac pgstac labels Nov 12, 2024
@bitner
Copy link

bitner commented Nov 12, 2024

Only thing that I'd modify there is that pgstacrs will expose every public pgstac function. There are a number of functions in pgstac that are purely internal.

@gadomski

This comment was marked as outdated.

@gadomski gadomski linked a pull request Nov 13, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[crate] pgstac pgstac enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants