Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug-1945556: omit example data when elasticsearch would error #6890

Merged
merged 2 commits into from
Feb 4, 2025

Conversation

relud
Copy link
Member

@relud relud commented Feb 3, 2025

and enable full version of all text fields so they still have example data under elasticsearch 8

context: the page /documentation/datadictionary/dataset/{dataset}/field/{field} for dataset=processed currently throws a 500 error for text fields with no full version and no fielddata

this only changes the mapping to include multi-fields where there was none before, which is a valid mapping update, so it can theoretically be applied in-place in stage so we don't need a backfill

and enable full version of all text fields so they can have example data
@relud relud requested a review from willkg February 3, 2025 19:17
@relud relud requested a review from a team as a code owner February 3, 2025 19:17
# text fields only get example data if they have a full version or
# include fielddata, like for semicolon_keywords fields.
super_search_field["has_full_version"]
or super_search_field["storage_mapping"]["type"] != "text"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will also work for es 1.4 because there the type is "string" and does support aggregations

"type": "text",
},
},
"minidump_sha256_hash": text_field("minidump_sha256_hash"),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be a keyword field?

Copy link
Contributor

@willkg willkg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I computed FIELDS in the main branch and in this branch and manually compared the two structures and everything looks good.

I had one cosmetic suggestion about Elasticsearch and that's it.

@@ -78,16 +78,15 @@ <h1>{{ field_name }}</h1>
{% for item in example_data %}
<div class="field-example"><code>{{ item|string|truncate(100, killwords=True, end="...") }}</code></div>
{% endfor %}
{% elif "protected" in permissions %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird. I wonder why I didn't do an elif before.

Co-authored-by: Will Kahn-Greene <[email protected]>
@relud relud enabled auto-merge February 4, 2025 22:13
@relud relud added this pull request to the merge queue Feb 4, 2025
Merged via the queue into main with commit 121b783 Feb 4, 2025
1 check passed
@relud relud deleted the relud-bug-1945556-example-data branch February 4, 2025 22:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants