Skip to content

Commit

Permalink
Storage: Add Indexes for common queries
Browse files Browse the repository at this point in the history
In #306 it's reported that SQLite is slow.  This adds indexes so lookups will be
faster.  The intention is to speed up the bottom 3 queries from
#306 (comment) as
they are "the ones to focus on".

Specifically this should speed up:

> * Load graph data for a file
>
>         SELECT value FROM graphs WHERE file = ?
>
>   Called many times during path stitching
>
> * Load paths for a file node
>
>         SELECT file,value from file_paths WHERE file = ? AND local_id = ?
>
>   Called many times during path stitching.
>
> * Load paths for root node
>
>         SELECT file, value from root_paths WHERE symbol_stack = ?
>
>   Called many times during path stitching.

If the data that you're fetching is in the index then SQLite won't even bother
reading the row proper, just pulling the data streight from the index (good for
data locality).  As such I've included not just the columns we're using in the
`WHERE` clause, but also the ones from `SELECT` too.

I've not actually tested the performance impact as I'm not familiar with this
project (this is more of a drive-by) and don't know what benchmarks to use.
  • Loading branch information
wmanley committed Aug 7, 2023
1 parent ee490e9 commit 1f5de0e
Showing 1 changed file with 15 additions and 0 deletions.
15 changes: 15 additions & 0 deletions stack-graphs/src/storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,12 @@ const SCHEMA: &str = r#"
) STRICT;
"#;

const INDEXES: &str = r#"
CREATE INDEX IF NOT EXISTS graph_file_value ON graph(file, value);
CREATE INDEX IF NOT EXISTS file_paths_file_local_id_value ON file_paths(file, local_id, value);
CREATE INDEX IF NOT EXISTS root_paths_symbol_stack_file_value ON root_paths(symbol_stack, file, value);
"#;

const PRAGMAS: &str = r#"
PRAGMA journal_mode = WAL;
PRAGMA foreign_keys = false;
Expand Down Expand Up @@ -147,6 +153,7 @@ impl SQLiteWriter {
pub fn open_in_memory() -> Result<Self> {
let mut conn = Connection::open_in_memory()?;
Self::init(&mut conn)?;
Self::init_indexes(&mut conn)?;
Ok(Self { conn })
}

Expand All @@ -161,6 +168,7 @@ impl SQLiteWriter {
} else {
check_version(&conn)?;
}
Self::init_indexes(&mut conn)?;
Ok(Self { conn })
}

Expand All @@ -173,6 +181,13 @@ impl SQLiteWriter {
Ok(())
}

fn init_indexes(conn: &mut Connection) -> Result<()> {
let tx = conn.transaction()?;
tx.execute_batch(INDEXES)?;
tx.commit()?;
Ok(())
}

/// Clean all data from the database.
pub fn clean_all(&mut self) -> Result<usize> {
let tx = self.conn.transaction()?;
Expand Down

0 comments on commit 1f5de0e

Please sign in to comment.