-
Notifications
You must be signed in to change notification settings - Fork 389
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: set logger level to error to reduce noise from Elasticsearch and OpenSearch client libraries #4979
chore: set logger level to error to reduce noise from Elasticsearch and OpenSearch client libraries #4979
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -21,14 +21,16 @@ | |
import re | ||
import warnings | ||
from pathlib import Path | ||
from typing import List, Optional | ||
from typing import List, Literal, Optional, Union | ||
from urllib.parse import urlparse | ||
|
||
from argilla_server.constants import ( | ||
DEFAULT_LABEL_SELECTION_OPTIONS_MAX_ITEMS, | ||
DEFAULT_MAX_KEYWORD_LENGTH, | ||
DEFAULT_SPAN_OPTIONS_MAX_ITEMS, | ||
DEFAULT_TELEMETRY_KEY, | ||
SEARCH_ENGINE_ELASTICSEARCH, | ||
SEARCH_ENGINE_OPENSEARCH, | ||
) | ||
from argilla_server.pydantic_v1 import BaseSettings, Field, root_validator, validator | ||
|
||
|
@@ -97,7 +99,10 @@ | |
|
||
es_mapping_total_fields_limit: int = 2000 | ||
|
||
search_engine: str = "elasticsearch" | ||
search_engine: Union[ | ||
Literal[SEARCH_ENGINE_ELASTICSEARCH], | ||
Literal[SEARCH_ENGINE_OPENSEARCH], | ||
] = SEARCH_ENGINE_ELASTICSEARCH | ||
|
||
vectors_fields_limit: int = Field( | ||
default=5, | ||
|
@@ -217,6 +222,21 @@ | |
return index_name.replace("<NAMESPACE>", "") | ||
return index_name.replace("<NAMESPACE>", f".{ns}") | ||
|
||
@property | ||
def search_engine_is_elasticsearch(self) -> bool: | ||
return self.search_engine == SEARCH_ENGINE_ELASTICSEARCH | ||
|
||
@property | ||
def search_engine_is_opensearch(self) -> bool: | ||
return self.search_engine == SEARCH_ENGINE_OPENSEARCH | ||
|
||
@property | ||
def humanized_search_engine(self) -> str: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The name of this method gives me chills... "I've seen things you people wouldn't believe. Attack ships on fire off the shoulder of Orion. I watched C-beams glitter in the dark near the Tannhäuser Gate. All those moments will be lost in time like tears in rain..." Could we just call There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "Humanize" is a very common concept and naming when formatting strings but I don't have any problem changing it to "human_readable_*". 🙂 Some examples:
|
||
if self.search_engine_is_elasticsearch: | ||
return "Elasticsearch" | ||
elif self.search_engine_is_opensearch: | ||
return "OpenSearch" | ||
|
||
def obfuscated_elasticsearch(self) -> str: | ||
"""Returns configured elasticsearch url obfuscating the provided password, if any""" | ||
parsed = urlparse(self.elasticsearch) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even if we can define constants for official engines, we shouldn't fix the available options.
Creating new custom engine or specific engine connection could be helpful for the community but, by fixing this we’re avoiding this possibility. (See how engine are registered)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But in the case that a PR from the community adds a new engine they should modify this
Union
as part of the PR right? I don't get the problem.I missed that one. I will use the constant there too.