Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

filter extension: temporal intersection not supported #183

Open
StijnCaerts opened this issue Jan 18, 2024 · 4 comments
Open

filter extension: temporal intersection not supported #183

StijnCaerts opened this issue Jan 18, 2024 · 4 comments

Comments

@StijnCaerts
Copy link
Collaborator

Describe the bug
The temporal intersection queries listed in the documentation of the filter API extension do not work.
For the cql2-text query, you get a parsing error as pygeofilter seems to expect a different syntax:

  • filter extension documentation: T_INTERSECTS(datetime, INTERVAL('2020-11-11T00:00:00Z', '2020-11-12T00:00:00Z'))
  • compatible with pygeofilter: datetime T_INTERSECTS INTERVAL('2020-11-11T00:00:00Z', '2020-11-12T00:00:00Z')

This results in the following error:

[18/Jan/2024:08:12:12 +0000] 172.17.0.1:57838 "GET /search?filter=T_INTERSECTS%28datetime%2C+INTERVAL%28%272020-11-11T00%3A00%3A00Z%27%2C+%272020-11-12T00%3A00%3A00Z%27%29%29&filter-lang=cql2-text HTTP/1.0" 500 - 14
Unexpected token Token('$END', '') at line 1, column 80.
Expected one of: 
	* T_BEFORE
	* LIKE
	* T_ENDEDBY
	* GTE
	* T_DURING
	* __ANON_0
	* T_INTERSECTS
	* __ANON_2
	* T_BEGUNBY
	* IS
	* T_OVERLAPPEDBY
	* __ANON_1
	* IN
	* LTE
	* NE
	* EQ
	* GT
	* T_CONTAINS
	* T_EQUALS
	* T_METBY
	* BETWEEN
	* T_OVERLAPS
	* MORETHAN
	* T_BEGINS
	* T_AFTER
	* EQUAL
	* __ANON_3
	* T_ENDS
	* LESSTHAN
	* T_MEETS
	* LT
NoneType: None
2024-01-18 08:12:12,868 - uvicorn.error - ERROR - Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 126, in feed_token
    action, arg = states[state][token.type]
KeyError: '$END'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/fastapi/applications.py", line 1106, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/site-packages/asgi_logger/middleware.py", line 61, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/asgi_logger/middleware.py", line 58, in __call__
    await self.app(scope, receive, inner_send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/authentication.py", line 48, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/stac_fastapi/api/middleware.py", line 75, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/brotli_asgi/__init__.py", line 87, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/usr/local/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 274, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)
  File "/usr/local/lib/python3.10/site-packages/stac_fastapi/api/routes.py", line 57, in _endpoint
    await func(request=request, **request_data.kwargs()), response_class
  File "/usr/local/lib/python3.10/site-packages/stac_fastapi/elasticsearch/core.py", line 384, in get_search
    base_args["filter"] = orjson.loads(to_cql2(parse_cql2_text(filter)))
  File "/usr/local/lib/python3.10/site-packages/pygeofilter/parsers/cql2_text/parser.py", line 211, in parse
    return parser.parse(cql_text)
  File "/usr/local/lib/python3.10/site-packages/lark/lark.py", line 581, in parse
    return self.parser.parse(text, start=start, on_error=on_error)
  File "/usr/local/lib/python3.10/site-packages/lark/parser_frontends.py", line 106, in parse
    return self.parser.parse(stream, chosen_start, **kw)
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 41, in parse
    return self.parser.parse(lexer, start)
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 171, in parse
    return self.parse_from_state(parser_state)
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 188, in parse_from_state
    raise e
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 182, in parse_from_state
    return state.feed_token(end_token, True)
  File "/usr/local/lib/python3.10/site-packages/lark/parsers/lalr_parser.py", line 129, in feed_token
    raise UnexpectedToken(token, expected, state=self, interactive_parser=None)
lark.exceptions.UnexpectedToken: Unexpected token Token('$END', '') at line 1, column 80.
Expected one of: 
	* T_BEFORE
	* LIKE
	* T_ENDEDBY
	* GTE
	* T_DURING
	* __ANON_0
	* T_INTERSECTS
	* __ANON_2
	* T_BEGUNBY
	* IS
	* T_OVERLAPPEDBY
	* __ANON_1
	* IN
	* LTE
	* NE
	* EQ
	* GT
	* T_CONTAINS
	* T_EQUALS
	* T_METBY
	* BETWEEN
	* T_OVERLAPS
	* MORETHAN
	* T_BEGINS
	* T_AFTER
	* EQUAL
	* __ANON_3
	* T_ENDS
	* LESSTHAN
	* T_MEETS
	* LT

But also the cql2-json query does not work:

{
  "detail": "Error with cql2_json filter: 25 validation errors for Clause\nop\n  value is not a valid enumeration member; permitted: 'and', 'or', 'not' (type=type_error.enum; enum_values=[<LogicalOp._and: 'and'>, <LogicalOp._or: 'or'>, <LogicalOp._not: 'not'>])\nop\n  value is not a valid enumeration member; permitted: '=', '<>', '<', '<=', '>', '>=', 'isNull' (type=type_error.enum; enum_values=[<ComparisonOp.eq: '='>, <ComparisonOp.neq: '<>'>, <ComparisonOp.lt: '<'>, <ComparisonOp.lte: '<='>, <ComparisonOp.gt: '>'>, <ComparisonOp.gte: '>='>, <ComparisonOp.is_null: 'isNull'>])\nop\n  value is not a valid enumeration member; permitted: 's_intersects' (type=type_error.enum; enum_values=[<SpatialIntersectsOp.s_intersects: 's_intersects'>])\nargs -> 1 -> op\n  field required (type=value_error.missing)\nargs -> 1 -> args\n  field required (type=value_error.missing)\nargs -> 1 -> property\n  field required (type=value_error.missing)\nargs -> 1 -> timestamp\n  field required (type=value_error.missing)\nargs -> 1 -> date\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> coordinates\n  field required (type=value_error.missing)\nargs -> 1 -> type\n  field required (type=value_error.missing)\nargs -> 1 -> geometries\n  field required (type=value_error.missing)\nargs -> 1\n  int() argument must be a string, a bytes-like object or a real number, not 'dict' (type=type_error)\nargs -> 1\n  str type expected (type=type_error.str)\nargs -> 1\n  value could not be parsed to a boolean (type=type_error.bool)"
}

To Reproduce
Run the temporal intersection query examples listed in the filter API extension documentation: https://github.com/stac-api-extensions/filter/?tab=readme-ov-file#example-6-temporal-intersection

Expected behavior
These queries should work for full support of the filter extension.
@dipstef78
Copy link

I am facing the exact same problem.
Is anybody trying to find a solution?

Thank you.

@jamesfisher-geo
Copy link
Collaborator

jamesfisher-geo commented Jul 24, 2024 •
edited
Loading

The implementation of the Filter extension here does not yet include temporal filters.

You can use the datetime field alongside filter for temporal search.

{
"datetime": ["2020-11-11T00:00:00Z", "2020-11-12T00:00:00Z"],
"filter": {}
}

Would this work?

@sotosoul
Copy link

I noticed the NoneType: None, which is something I'm also getting. Seems to me that it has to do with the chosen logging level (logging.exception vs .error). I'd like it to go away -- has this been resolved elsewhere?

@jamesfisher-geo
Copy link
Collaborator

@sotosoul Does my solution above work for you?

The issue here is that the T_INTERSECTS search operator is not supported in the implementation here. Happy to review any PRs if you are willing to contribute that feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@StijnCaerts @sotosoul @jamesfisher-geo @dipstef78