Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add esql getting started notebook #201

Merged
merged 9 commits into from
Mar 13, 2024

Conversation

leemthompo
Copy link
Contributor

@leemthompo leemthompo commented Mar 11, 2024

Open In Colab

Ports getting started to Python.

TODO

  • Write introduction

Copy link

gitnotebooks bot commented Mar 11, 2024

Found 1 changed notebook. Review the changes at https://gitnotebooks.com/elastic/elasticsearch-labs/pull/201

@leemthompo
Copy link
Contributor Author

leemthompo commented Mar 11, 2024

@miguelgrinberg CI is failing on the enrich policy in this notebook:

BadRequestError: BadRequestError(400, 'verification_exception', 'Found 1 problem\nline 5:10: 
failed to resolve enrich policy [clientip_policy]; reason [Unknown index [.enrich-clientip_policy]]')

Any idea what's at play here? Didn't hit any issues when running the notebook on colab against an 8.13 cluster.

If testing logic is trying to interact with the enrich index directly using a static index name like .enrich-clientip_policy, it will run into issues because the actual index name includes a random ID at the end.

@miguelgrinberg
Copy link
Collaborator

@leemthompo are there any differences between cloud and docker image with regards to this? The tests run against a self-hosted minimal container, so the behavior might be different than running against cloud.

Another issue you are going to hit is that the notebook is also going to run against 8.11 and 8.12. Sounds like I'll need to figure out a mechanism to specify which versions a notebook needs to run on.

@leemthompo
Copy link
Contributor Author

are there any differences between cloud and docker image with regards to this? The tests run against a self-hosted minimal container, so the behavior might be different than running against cloud.

@miguelgrinberg I've tested the notebook locally against a single-node Docker ES instance, and indeed the enrich process doesn't seem to work in this configuration. This doesn't seem to be esql specific.

Another issue you are going to hit is that the notebook is also going to run against 8.11 and 8.12. Sounds like I'll need to figure out a mechanism to specify which versions a notebook needs to run on.

Yes, we hit similar issues with Aurelien's LTR notebook failing pre-8.12 tests.

@miguelgrinberg
Copy link
Collaborator

The easy option is to add this notebook to the exempt list in this file. The hard option would be to figure out this failure and adapt the notebook to work under Docker as well. And I need to expand the file I linked above to be version aware and provide different lists of notebooks to skip for each given stack release, which I'm planning to anyway.

@leemthompo
Copy link
Contributor Author

The hard option would be to figure out this failure and adapt the notebook to work under Docker as well.

I'm not sure if multi-node Docker clusters solves the issue, but unfortunately I ran out of time to confirm this today.

@miguelgrinberg
Copy link
Collaborator

miguelgrinberg commented Mar 12, 2024

I don't think we can stand up a multi-node Docker instance within the resource constrained GitHub Actions environment. I think the best course of action for now is to skip this notebook when testing.

@leemthompo leemthompo marked this pull request as ready for review March 12, 2024 13:01
Copy link
Contributor

@szabosteve szabosteve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome notebook! It has a great flow and is easy to follow. I tested it end-to-end on a 8.14 cluster (es-py 8.12.0) and it worked flawlessly.

@miguelgrinberg miguelgrinberg merged commit 9166206 into elastic:main Mar 13, 2024
5 checks passed
@leemthompo leemthompo deleted the esql-notebook branch March 18, 2024 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants