Skip to content

Commit 572e022

Browse files
github-actions[bot]miguelgrinberg
github-actions[bot]
and
miguelgrinberg
authored
8.18 client documentation updates (#2925) (#2926)
(cherry picked from commit 6074eca) Co-authored-by: Miguel Grinberg <[email protected]>
1 parent bd8d41e commit 572e022

File tree

3 files changed

+370
-5
lines changed

3 files changed

+370
-5
lines changed

‎docs/guide/async.asciidoc

+141
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
[[async]]
2+
== Using with asyncio
3+
4+
The `elasticsearch` package supports async/await with
5+
https://docs.python.org/3/library/asyncio.html[asyncio] and
6+
https://docs.aiohttp.org[aiohttp]. You can either install `aiohttp`
7+
directly or use the `[async]` extra:
8+
9+
[source,bash]
10+
----
11+
$ python -m pip install elasticsearch aiohttp
12+
13+
# - OR -
14+
15+
$ python -m pip install elasticsearch[async]
16+
----
17+
18+
[discrete]
19+
=== Getting Started with Async
20+
21+
After installation all async API endpoints are available via
22+
`~elasticsearch.AsyncElasticsearch` and are used in the same way as
23+
other APIs, with an extra `await`:
24+
25+
[source,python]
26+
----
27+
import asyncio
28+
from elasticsearch import AsyncElasticsearch
29+
30+
client = AsyncElasticsearch()
31+
32+
async def main():
33+
resp = await client.search(
34+
index="documents",
35+
body={"query": {"match_all": {}}},
36+
size=20,
37+
)
38+
print(resp)
39+
40+
loop = asyncio.get_event_loop()
41+
loop.run_until_complete(main())
42+
----
43+
44+
All APIs that are available under the sync client are also available
45+
under the async client.
46+
47+
https://elasticsearch-py.readthedocs.io/en/latest/async.html#api-reference[Reference documentation]
48+
49+
[discrete]
50+
=== ASGI Applications and Elastic APM
51+
52+
https://asgi.readthedocs.io[ASGI] (Asynchronous Server Gateway
53+
Interface) is a way to serve Python web applications making use of
54+
async I/O to achieve better performance. Some examples of ASGI
55+
frameworks include FastAPI, Django 3.0+, and Starlette. If you're
56+
using one of these frameworks along with Elasticsearch then you should
57+
be using `~elasticsearch.AsyncElasticsearch` to avoid blocking the event
58+
loop with synchronous network calls for optimal performance.
59+
60+
https://www.elastic.co/guide/en/apm/agent/python/current/index.html[Elastic
61+
APM] also supports tracing of async Elasticsearch queries just the same
62+
as synchronous queries. For an example on how to configure
63+
`AsyncElasticsearch` with a popular ASGI framework
64+
https://fastapi.tiangolo.com/[FastAPI] and APM tracing there is a
65+
https://github.com/elastic/elasticsearch-py/tree/master/examples/fastapi-apm[pre-built
66+
example] in the `examples/fastapi-apm` directory.
67+
68+
See also the <<opentelemetry>> page.
69+
70+
[discrete]
71+
=== Frequently Asked Questions
72+
73+
[discrete]
74+
==== ValueError when initializing `AsyncElasticsearch`?
75+
76+
If when trying to use `AsyncElasticsearch` you receive
77+
`ValueError: You must have 'aiohttp' installed to use AiohttpHttpNode`
78+
you should ensure that you have `aiohttp` installed in your environment
79+
(check with `$ python -m pip freeze | grep aiohttp`). Otherwise,
80+
async support won't be available.
81+
82+
[discrete]
83+
==== What about the `elasticsearch-async` package?
84+
85+
Previously asyncio was supported separately via the
86+
https://github.com/elastic/elasticsearch-py-async[elasticsearch-async]
87+
package. The `elasticsearch-async` package has been deprecated in favor
88+
of `AsyncElasticsearch` provided by the `elasticsearch` package in v7.8
89+
and onwards.
90+
91+
[discrete]
92+
==== Receiving 'Unclosed client session / connector' warning?
93+
94+
This warning is created by `aiohttp` when an open HTTP connection is
95+
garbage collected. You'll typically run into this when closing your
96+
application. To resolve the issue ensure that
97+
`~elasticsearch.AsyncElasticsearch.close` is called before the
98+
`~elasticsearch.AsyncElasticsearch` instance is garbage collected.
99+
100+
For example if using FastAPI that might look like this:
101+
102+
[source,python]
103+
----
104+
import os
105+
from contextlib import asynccontextmanager
106+
107+
from fastapi import FastAPI
108+
from elasticsearch import AsyncElasticsearch
109+
110+
ELASTICSEARCH_URL = os.environ["ELASTICSEARCH_URL"]
111+
client = None
112+
113+
@asynccontextmanager
114+
async def lifespan(app: FastAPI):
115+
global client
116+
client = AsyncElasticsearch(ELASTICSEARCH_URL)
117+
yield
118+
await client.close()
119+
120+
app = FastAPI(lifespan=lifespan)
121+
122+
@app.get("/")
123+
async def main():
124+
return await client.info()
125+
----
126+
127+
You can run this example by saving it to `main.py` and executing
128+
`ELASTICSEARCH_URL=http://localhost:9200 uvicorn main:app`.
129+
130+
[discrete]
131+
=== Async Helpers
132+
133+
Async variants of all helpers are available in `elasticsearch.helpers`
134+
and are all prefixed with `async_*`. You'll notice that these APIs
135+
are identical to the ones in the sync <<client-helpers>> documentation.
136+
137+
All async helpers that accept an iterator or generator also accept async
138+
iterators and async generators.
139+
140+
https://elasticsearch-py.readthedocs.io/en/latest/async.html#async-helpers[Reference documentation]
141+

‎docs/guide/examples.asciidoc

+176
Original file line numberDiff line numberDiff line change
@@ -109,3 +109,179 @@ method:
109109
----------------------------
110110
client.delete(index="test-index", id=1)
111111
----------------------------
112+
113+
[discrete]
114+
[[ex-interactive]]
115+
=== Interactive examples
116+
117+
The https://github.com/elastic/elasticsearch-labs[elasticsearch-labs]
118+
repo contains interactive and executable
119+
https://github.com/elastic/elasticsearch-labs/tree/main/notebooks[Python
120+
notebooks], sample apps, and resources for testing out Elasticsearch,
121+
using the Python client. These examples are mainly focused on vector
122+
search, hybrid search and generative AI use cases, but you'll also find
123+
examples of basic operations like creating index mappings and performing
124+
lexical search.
125+
126+
[discrete]
127+
==== Search notebooks
128+
129+
The
130+
https://github.com/elastic/elasticsearch-labs/tree/main/notebooks/search[Search]
131+
folder is a good place to start if you're new to Elasticsearch. This
132+
folder contains a number of notebooks that demonstrate the fundamentals
133+
of Elasticsearch, like indexing vectors, running lexical, semantic and
134+
_hybrid_ searches, and more.
135+
136+
The following notebooks are available:
137+
138+
[arabic, start=0]
139+
* https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/00-quick-start.ipynb[Quick
140+
start]
141+
* https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/01-keyword-querying-filtering.ipynb[Keyword&#44;
142+
querying&#44; filtering]
143+
* https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/02-hybrid-search.ipynb[Hybrid
144+
search]
145+
* https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/03-ELSER.ipynb[Semantic
146+
search with ELSER]
147+
* https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/04-multilingual.ipynb[Multilingual
148+
semantic search]
149+
* https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/05-query-rules.ipynb[Query
150+
rules]
151+
* https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/06-synonyms-api.ipynb[Synonyms
152+
API quick start]
153+
154+
Here's a brief overview of what you'll learn in each notebook.
155+
156+
[discrete]
157+
===== Quick start
158+
159+
In the
160+
https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/00-quick-start.ipynb[00-quick-start.ipynb]
161+
notebook you'll learn how to:
162+
163+
* Use the Elasticsearch Python client for various operations.
164+
* Create and define an index for a sample dataset with
165+
`dense_vector` fields.
166+
* Transform book titles into embeddings using
167+
https://www.sbert.net[Sentence Transformers] and index them into
168+
Elasticsearch.
169+
* Perform k-nearest neighbors (knn) semantic searches.
170+
* Integrate traditional text-based search with semantic search, for a
171+
hybrid search system.
172+
* Use reciprocal rank fusion (RRF) to intelligently combine search
173+
results from different retrieval systems.
174+
175+
[discrete]
176+
===== Keyword, querying, filtering
177+
178+
In the
179+
https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/01-keyword-querying-filtering.ipynb[01-keyword-querying-filtering.ipynb]
180+
notebook, you'll learn how to:
181+
182+
* Use
183+
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-filter-context.html[query
184+
and filter contexts] to search and filter documents in Elasticsearch.
185+
* Execute full-text searches with `match` and `multi-match` queries.
186+
* Query and filter documents based on `text`, `number`, `date`, or
187+
`boolean` values.
188+
* Run multi-field searches using the `multi-match` query.
189+
* Prioritize specific fields in the `multi-match` query for tailored
190+
results.
191+
192+
[discrete]
193+
===== Hybrid search
194+
195+
In the
196+
https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/02-hybrid-search.ipynb[02-hybrid-search.ipynb]
197+
notebook, you'll learn how to:
198+
199+
* Combine results of traditional text-based search with semantic search,
200+
for a hybrid search system.
201+
* Transform fields in the sample dataset into embeddings using the
202+
Sentence Transformer model and index them into Elasticsearch.
203+
* Use the
204+
https://www.elastic.co/guide/en/elasticsearch/reference/current/rrf.html#rrf-api[RRF
205+
API] to combine the results of a `match` query and a `kNN` semantic
206+
search.
207+
* Walk through a super simple toy example that demonstrates, step by
208+
step, how RRF ranking works.
209+
210+
[discrete]
211+
===== Semantic search with ELSER
212+
213+
In the
214+
https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/03-ELSER.ipynb[03-ELSER.ipynb]
215+
notebook, you'll learn how to:
216+
217+
* Use the Elastic Learned Sparse Encoder (ELSER) for text
218+
expansion-powered semantic search, out of the box — without training,
219+
fine-tuning, or embeddings generation.
220+
* Download and deploy the ELSER model in your Elastic environment.
221+
* Create an Elasticsearch index named [.title-ref]#search-movies# with
222+
specific mappings and index a dataset of movie descriptions.
223+
* Create an ingest pipeline containing an inference processor for ELSER
224+
model execution.
225+
* Reindex the data from [.title-ref]#search-movies# into another index,
226+
[.title-ref]#elser-movies#, using the ELSER pipeline for text expansion.
227+
* Observe the results of running the documents through the model by
228+
inspecting the additional terms it adds to documents, which enhance
229+
searchability.
230+
* Perform simple keyword searches on the [.title-ref]#elser-movies#
231+
index to assess the impact of ELSER's text expansion.
232+
* Execute ELSER-powered semantic searches using the `text_expansion`
233+
query.
234+
235+
[discrete]
236+
===== Multilingual semantic search
237+
238+
In the
239+
https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/04-multilingual.ipynb[04-multilingual.ipynb]
240+
notebook, you'll learn how to:
241+
242+
* Use a multilingual embedding model for semantic search across
243+
languages.
244+
* Transform fields in the sample dataset into embeddings using the
245+
Sentence Transformer model and index them into Elasticsearch.
246+
* Use filtering with a `kNN` semantic search.
247+
* Walk through a super simple toy example that demonstrates, step by
248+
step, how multilingual search works across languages, and within
249+
non-English languages.
250+
251+
[discrete]
252+
===== Query rules
253+
254+
In the
255+
https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/05-query-rules.ipynb[05-query-rules.ipynb]
256+
notebook, you'll learn how to:
257+
258+
* Use the query rules management APIs to create and edit promotional
259+
rules based on contextual queries.
260+
* Apply these query rules by using the `rule_query` in Query DSL.
261+
262+
[discrete]
263+
===== Synonyms API quick start
264+
265+
In the
266+
https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/search/06-synonyms-api.ipynb[06-synonyms-api.ipynb]
267+
notebook, you'll learn how to:
268+
269+
* Use the synonyms management API to create a synonyms set to enhance
270+
your search recall.
271+
* Configure an index to use search-time synonyms.
272+
* Update synonyms in real time.
273+
* Run queries that are enhanced by synonyms.
274+
275+
[discrete]
276+
==== Other notebooks
277+
278+
* https://github.com/elastic/elasticsearch-labs/tree/main/notebooks/generative-ai[Generative
279+
AI]. Notebooks that demonstrate various use cases for Elasticsearch as
280+
the retrieval engine and vector store for LLM-powered applications.
281+
* https://github.com/elastic/elasticsearch-labs/blob/main/notebooks/integrations[Integrations].
282+
Notebooks that demonstrate how to integrate popular services and
283+
projects with Elasticsearch, including OpenAI, Hugging Face, and
284+
LlamaIndex
285+
* https://github.com/elastic/elasticsearch-labs/tree/main/notebooks/langchain[Langchain].
286+
Notebooks that demonstrate how to integrate Elastic with LangChain, a
287+
framework for developing applications powered by language models.

0 commit comments

Comments
 (0)