Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query performance observability #7725

Closed
7 tasks done
msullivan opened this issue Sep 6, 2024 · 1 comment
Closed
7 tasks done

Query performance observability #7725

msullivan opened this issue Sep 6, 2024 · 1 comment
Assignees

Comments

@msullivan
Copy link
Member

msullivan commented Sep 6, 2024

Mostly a placeholder issue for now.
We want to provide visibility into query performance

I think the overall plan here is to use pg_stat_statements (https://www.postgresql.org/docs/current/pgstatstatements.html) to collect information.

I think the core questions are:

  1. How do we connect our EdgeQL queries with information from pg_stat_statments?

    We forked pg_stat_statements to record custom information like EdgeQL query string, compilation settings, etc.

  2. How do we expose the information to our users?

    sys::QueryStats is added as a view of the pg_stat_statements backend view.

@msullivan msullivan added performance server under discussion Not ready for implementation labels Sep 6, 2024
fantix added a commit that referenced this issue Nov 21, 2024
Add edb_stat_statements Postgres extension (forked from the master
branch of the upstream pg_stat_statement extension) to handle custom
query performance statistics. `sys::QueryStats` is added as a view of
the statistics.

This is done in a way that, for each stats-significant SQL we send to the
 backend, one or more comment lines of "query stats info" JSONs are
prepended for the Postgres extension to ingest and record in the modified
statistics hash table. Among the stats info fields, `id: uuid` is especially
important to identify different queries and accumulate stats of the same
query onto the same hash table entry, which reflects some settings that
affected the compilation (excluding the user schema version for common
grouping of stats). Particularly, the first-8-bytes of `id` is also used by
the Postgres extension to replace the underlying `queryId` of the SQL
statement, so that the same frontend query can be recognized across all
PARSE/EXECUTE operations in Postgres for stats recording.

System queries, DDLs, and unrecognized queries are not recorded.

Refs #7725
@fantix fantix removed the under discussion Not ready for implementation label Nov 21, 2024
@fantix
Copy link
Member

fantix commented Dec 2, 2024

Closing as fixed, please see each linked PR for details.

@fantix fantix closed this as completed Dec 2, 2024
@beerose beerose mentioned this issue Dec 10, 2024
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants