[BUG] Invalid Field Names in Metric Aggregation Queries that use star tree returns 500 Internal Server Error #16473

expani · 2024-10-24T15:03:52Z

Describe the bug

When running Metric aggregation queries over indices that have star tree enabled, using invalid field names in the query leads to a 500 Internal Server Error.

Related component

Search

Reproduction Steps

Checkout the main branch of OpenSearch.

We need to enable the star tree feature flag
Add this line in OpenSearchNode#createConfiguration() which is the default config used with gradlew run.

baseConfig.put("opensearch.experimental.feature.composite_index.star_tree.enabled", "true");

Build the checkout and run the server ./gradlew run

Enable the Cluster setting indices.composite_index.star_tree.enabled for indexing

curl -X PUT "localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'{
  "persistent" : {
    "indices.composite_index.star_tree.enabled" : "true"
  }
}'

Create Index

curl -XPUT -H'Content-type: application/json' -kv localhost:9200/logs -d '{
  "settings": {
    "index.number_of_shards": 1,
    "index.number_of_replicas": 0,
    "index.composite_index": true
  },
  "mappings": {
    "composite": {
      "startree1": {
        "type": "star_tree",
        "config": {
          "ordered_dimensions": [
            {
              "name": "status"
            },
			{
              "name": "port"
            }
          ],
          "metrics": [
            {
              "name": "size",
              "stats": [
                "sum"
              ]
             },
             {
              "name": "latency",
              "stats": [
                "avg"
              ]
            }
          ]
        }
      }
    },
    "properties": {
      "status": {
        "type": "integer"
      },
      "port": {
        "type": "integer"
      },
      "size": {
        "type": "integer"
      },
      "latency": {
        "type": "scaled_float",
        "scaling_factor": 10
      }
    }
  }
}'

Run a search query containing an invalid field name. request_size in this case.

curl -XPOST -H'Content-type: application/json' -kv localhost:9200/logs/_search -d '{
  "size": 0,
  "aggs": {
    "sum_request_size": {
      "sum": {
        "field": "request_size"
      }
    }
  }
}'

A 500 Internal Server Error is thrown with the below exception seen in logs.

[2024-10-24T19:56:56,719][WARN ][r.suppressed             ] [runTask-0] path: /logs/_search, params: {index=logs}
org.opensearch.action.search.SearchPhaseExecutionException: all shards failed
        at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:775) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:395) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:815) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:548) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:316) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:104) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:75) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:760) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$9.handleException(TransportService.java:1719) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1505) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1619) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1593) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:81) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportChannel.sendErrorResponse(TransportChannel.java:75) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:70) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.ActionRunnable.onFailure(ActionRunnable.java:104) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:54) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.threadpool.TaskAwareRunnable.doRun(TaskAwareRunnable.java:78) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:59) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:982) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
        at java.base/java.lang.Thread.run(Thread.java:1570) [?:?]
Caused by: org.opensearch.OpenSearchException$3: Cannot invoke "org.opensearch.search.aggregations.support.FieldContext.field()" because the return value of "org.opensearch.search.aggregations.support.ValuesSourceConfig.fieldContext()" is null
        at org.opensearch.OpenSearchException.guessRootCauses(OpenSearchException.java:710) ~[opensearch-core-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:393) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        ... 23 more
Caused by: java.lang.NullPointerException: Cannot invoke "org.opensearch.search.aggregations.support.FieldContext.field()" because the return value of "org.opensearch.search.aggregations.support.ValuesSourceConfig.fieldContext()" is null
        at org.opensearch.search.aggregations.support.ValuesSourceAggregatorFactory.getField(ValuesSourceAggregatorFactory.java:107) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.index.compositeindex.datacube.startree.utils.StarTreeQueryHelper.validateStarTreeMetricSupport(StarTreeQueryHelper.java:153) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.index.compositeindex.datacube.startree.utils.StarTreeQueryHelper.getStarTreeQueryContext(StarTreeQueryHelper.java:77) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.search.SearchService.parseSource(SearchService.java:1550) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.search.SearchService.createContext(SearchService.java:1108) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.search.SearchService.executeQueryPhase(SearchService.java:705) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.search.SearchService$2.lambda$onResponse$0(SearchService.java:678) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:74) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        ... 8 more

Expected behavior

400 Bad Request should be returned with appropriate error message.

The text was updated successfully, but these errors were encountered:

expani · 2024-10-24T15:51:49Z

Field Context is retrieved from the Shard Context mapper here and remains null if it's not found.

If we handle null check here and throw appropriate exception here I think the issue can be solved.

Let me know your thoughts on this. @bharath-techie @sandeshkr419 I

expani · 2024-10-24T20:33:39Z

The above checks fixed the issue. But, there are other issues like validation of field name present in query against metrics and dimensions (CompositeDataCubeFieldType ) to ensure it can make use of star tree index.

Will take a deeper look into it later.

expani · 2024-10-25T09:33:45Z

There is no error in the query when Star Tree feature flag is disabled. It just returns empty hits.

{"took":4,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":0,"relation":"eq"},"max_score":null,"hits":[]},"aggregations":{"average_latency":{"value":null},"sum_request_size":{"value":0.0}}}

So, we were never validating fields used in aggregation against the actual fields of index. Most of the existing aggregation queries without star tree have the same limitation.

I was able to manage the same response for metric aggregation queries using star tree with these changes.

expani added bug Something isn't working untriaged labels Oct 24, 2024

github-actions bot added the Search Search query, autocomplete ...etc label Oct 24, 2024

expani mentioned this issue Oct 24, 2024

Add documentation for star tree index feature opensearch-project/documentation-website#8598

Merged

1 task

expani linked a pull request Oct 25, 2024 that will close this issue

Changes to unmapped fields in metric aggregation factory #16481

Draft

sandeshkr419 added Search:Aggregations and removed untriaged labels Oct 30, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG] Invalid Field Names in Metric Aggregation Queries that use star tree returns 500 Internal Server Error #16473

[BUG] Invalid Field Names in Metric Aggregation Queries that use star tree returns 500 Internal Server Error #16473

expani commented Oct 24, 2024 •

edited

Loading

expani commented Oct 24, 2024

expani commented Oct 24, 2024

expani commented Oct 25, 2024 •

edited

Loading

[BUG] Invalid Field Names in Metric Aggregation Queries that use star tree returns 500 Internal Server Error #16473

[BUG] Invalid Field Names in Metric Aggregation Queries that use star tree returns 500 Internal Server Error #16473

Comments

expani commented Oct 24, 2024 • edited Loading

Describe the bug

Related component

Reproduction Steps

Expected behavior

expani commented Oct 24, 2024

expani commented Oct 24, 2024

expani commented Oct 25, 2024 • edited Loading

expani commented Oct 24, 2024 •

edited

Loading

expani commented Oct 25, 2024 •

edited

Loading