API doesn't allow adding an Item to a Collection if geometry field is missing or null

[mcucchi9]

Description

When trying to submit a POST /collections/{collection_id}/items request the following behaviors occur:

• if the geometry field is missing, I get a 400 response associated to a RequestValidationError stating that the geometry field is required;
• if the geometry field is null, I get a 424 response associated to a DatabaseError, which appears to be caused by the fact that the geometry column in the postgres datbase is not nullable;

Expected behavior

The STAC Item specification for the geometry field states that

If a geometry is provided, the value must be a Geometry Object according to RFC 7946, section 3.1 with the exception that the type GeometryCollection is not allowed in STAC. If no geometry is provided, the value must be null according to RFC 7946, section 3.2.

I would thus expect it to be possible to at least create an Item with the geometry property set to null.

How to replicate

After cloning the repository, in its root directory launch

docker compose -f docker-compose.yml up

Then

curl -X 'POST' \
  'http://0.0.0.0:8082/collections/joplin/items' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "id": "f2cca2a3-288b-4518-8a3e-a4492bb60b08",
    "type": "Feature",
    "collection": "joplin",
    "links": [],
    "properties": {
        "datetime": "2000-02-02T00:00:00Z"
    },
    "assets": {},
    "stac_extensions": [],
    "stac_version": "1.0.0"
}'

which results in

{"code":"RequestValidationError","description":"[{'type': 'missing', 'loc': ('body', 'Item', 'geometry'), 'msg': 'Field required', 'input': {'id': 'f2cca2a3-288b-4518-8a3e-a4492bb60b08', 'type': 'Feature', 'collection': 'joplin', 'links': [], 'properties': {'datetime': '2000-02-02T00:00:00Z'}, 'assets': {}, 'stac_extensions': [], 'stac_version': '1.0.0'}}, {'type': 'literal_error', 'loc': ('body', 'ItemCollection', 'type'), 'msg': \"Input should be 'FeatureCollection'\", 'input': 'Feature', 'ctx': {'expected': \"'FeatureCollection'\"}}, {'type': 'missing', 'loc': ('body', 'ItemCollection', 'features'), 'msg': 'Field required', 'input': {'id': 'f2cca2a3-288b-4518-8a3e-a4492bb60b08', 'type': 'Feature', 'collection': 'joplin', 'links': [], 'properties': {'datetime': '2000-02-02T00:00:00Z'}, 'assets': {}, 'stac_extensions': [], 'stac_version': '1.0.0'}}]"}

Also, try

curl -X 'POST' \
  'http://0.0.0.0:8082/collections/joplin/items' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "id": "f2cca2a3-288b-4518-8a3e-a4492bb60b08",
    "type": "Feature",
    "geometry": null,
    "collection": "joplin",
    "links": [],
    "properties": {
        "datetime": "2000-02-02T00:00:00Z"
    },
    "assets": {},
    "stac_extensions": [],
    "stac_version": "1.0.0"
}'

which results in

{
  "code": "DatabaseError",
  "description": ""
}

[vincentsarago]

In term for Item specification I think you need a bbox if the geometry is null https://github.com/stac-utils/stac-pydantic/blob/main/stac_pydantic/item.py#L32-L38 but your error message doesn't seems to reflect this 🤷

☝️ edited, in fact you need a BBOX only when the geometry is not Null

For the second error, this would need to be checked with pgstac directly but yeah it might be possible that it doesn't support empty geometries

[mcucchi9]

Hi @vincentsarago, thanks for your fast reply.

but your error message doesn't seems to reflect this

In fact I obtain the same errors if the bbox property is present, in both cases.

[mcucchi9]

Anyway, from a FastAPI perspective I can see that the issue is due to the fact that the Item response model is imported from the stac_fastapi.types.stac module, where in fact the geometry property is not marked as optional:

class Item(TypedDict, total=False):
    """STAC Item."""

    type: Literal["Feature"]
    stac_version: str
    stac_extensions: Optional[List[str]]
    id: str
    geometry: Dict[str, Any]
    bbox: BBox
    properties: Dict[str, Any]
    links: List[Dict[str, Any]]
    assets: Dict[str, Any]
    collection: str

Also, on the postgres side I can see that pgstac defines the geometry column as NOT NULL, here.

So it does look like a deliberate choice, but in this case I would like to know why it looks (at least to my knowledge) in contrast with the STAC specification.

[vincentsarago]

Anyway, from a FastAPI perspective I can see that the issue is due to the fact that the Item response model is imported from the stac_fastapi.types.stac module, where in fact the geometry property is not marked as optional:

The body request is defined in https://github.com/stac-utils/stac-fastapi/blob/4491fa1ee384cd3e789e30aaa1252cdbb2ea3246/stac_fastapi/extensions/stac_fastapi/extensions/core/transaction.py#L19-L23 so it uses the stac-pydantic definition and is then transformed to a dict (but that would not throw any error), I'm pretty sure the geometry can be null, so I'm not sure where your error comes from.

from stac_pydantic import Item
from stac_fastapi.types import stac as stac_types

item_dict = {
    "id": "f2cca2a3-288b-4518-8a3e-a4492bb60b08",
    "type": "Feature",
    "geometry": None,
    "collection": "joplin",
    "links": [],
    "properties": {
        "datetime": "2000-02-02T00:00:00Z"
    },
    "assets": {},
    "stac_extensions": [],
    "stac_version": "1.0.0"
}

item = Item.model_validate(item_dict)
>> Item(bbox=None, type='Feature', geometry=None, properties=ItemProperties(title=None, description=None, datetime=datetime.datetime(2000, 2, 2, 0, 0, tzinfo=datetime.timezone.utc), created=None, updated=None, start_datetime=None, end_datetime=None, license=None, providers=None, platform=None, instruments=None, constellation=None, mission=None, gsd=None), id='f2cca2a3-288b-4518-8a3e-a4492bb60b08', stac_version='1.0.0', assets={}, links=Links(root=[]), stac_extensions=[], collection='joplin')

item = item.model_dump(mode="json")
stac_types.Item(**item)
>> {'type': 'Feature', 'geometry': None, 'properties': {'datetime': '2000-02-02T00:00:00Z'}, 'id': 'f2cca2a3-288b-4518-8a3e-a4492bb60b08', 'stac_version': '1.0.0', 'assets': {}, 'links': [], 'stac_extensions': [], 'collection': 'joplin'}

Which version of stac-fastapi are you using?

Also, on the postgres side I can see that pgstac defines the geometry column as NOT NULL, here.

So it does look like a deliberate choice, but in this case I would like to know why it looks (at least to my knowledge) in contrast with the STAC specification.

Yeah, let's consider this another issue at the moment

[bitner]

Geometry is required to be not null in pgstac.

[mcucchi9]

These is the output of pip list

Package                 Version     Editable project location
----------------------- ----------- -------------------------
annotated-types         0.7.0
anyio                   4.6.2.post1
asyncpg                 0.30.0
attrs                   24.2.0
Brotli                  1.1.0
brotli-asgi             1.4.0
buildpg                 0.4
cachetools              5.3.3
click                   8.1.7
dateparser              1.2.0
fastapi                 0.115.4
fire                    0.4.0
geojson-pydantic        1.1.2
h11                     0.14.0
httptools               0.6.4
idna                    3.10
iso8601                 2.1.0
lark                    0.12.0
orjson                  3.10.11
pip                     24.0
plpygis                 0.2.2
pydantic                2.9.2
pydantic_core           2.23.4
pydantic-settings       2.6.1
pygeofilter             0.2.4
pygeoif                 1.5.0
pypgstac                0.9.1
python-dateutil         2.8.2
python-dotenv           1.0.1
pytz                    2024.2
PyYAML                  6.0.2
regex                   2024.11.6
setuptools              65.5.1
six                     1.16.0
smart-open              7.0.5
sniffio                 1.3.1
stac-fastapi.api        3.0.3
stac-fastapi.extensions 3.0.3
stac-fastapi.pgstac     3.0.0       /app
stac-fastapi.types      3.0.3
stac-pydantic           3.1.3
starlette               0.41.2
tenacity                8.1.0
termcolor               2.5.0
typing_extensions       4.12.2
tzlocal                 5.2
uvicorn                 0.19.0
uvloop                  0.21.0
version-parser          1.0.1
watchfiles              0.24.0
websockets              14.0
wheel                   0.44.0
wrapt                   1.16.0

[mcucchi9]

Geometry is required to be not null in pgstac.

Hi @bitner, thanks. So it is a choice which doesn't comply with the standard, correct?
I'm just wondering if I'm getting it right.