Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move the scroll ID from the DELETE /_search/scroll URL to its body. #158

Merged

Conversation

ewanmellor
Copy link
Contributor

Scroll IDs can be too long for an HTTP request header, and so should only be put in the body. Putting them in the URL was deprecated in ElasticSearch 7.0.0 (April 2019).

The SearchClient.scroll method already does this, it was only SearchClient.deleteScroll that did not.

See https://www.elastic.co/guide/en/elasticsearch/reference/current/clear-scroll-api.html

Scroll IDs can be too long for an HTTP request header, and so should
only be put in the body. Putting them in the URL was deprecated in
ElasticSearch 7.0.0 (April 2019).

The SearchClient.scroll method already does this, it was only
SearchClient.deleteScroll that did not.

See https://www.elastic.co/guide/en/elasticsearch/reference/current/clear-scroll-api.html
@jillesvangurp jillesvangurp merged commit 0c69bef into jillesvangurp:master Dec 24, 2024
1 check passed
@jillesvangurp
Copy link
Owner

Thanks, I've merged this.

I don't think this was that big of a deal. The request url + parameters can be a a few KB. I don't think they deprecated the behavior. Also, Opensearch does not document the request body but the matrix tests seem to pass meaning it does work. The OS documentation isn't great.

@ewanmellor
Copy link
Contributor Author

The HTTP header part can be 10k on AWS, and we have scroll IDs that overspilled that, that's how I noticed.

@jillesvangurp
Copy link
Owner

It's not a header but the query string. But still, 10KB is a lot. Mostly ids are very short hashes; I wonder what they are doing differently in AWS. Are you submitting multiple hashes somehow?

@ewanmellor
Copy link
Contributor Author

Right, I was referring to the non-body section of the HTTP request (preamble, URL plus query params, headers). That part is counted together.

I think it's just one scroll ID, but it's a very complex query hitting a lot of documents. They are packing a lot of info about the query in there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants