Speed up initialization #1977
Speed up initialization #1977
Conversation
core/src/graph.jl
Outdated
There was a problem hiding this comment.
This is responsible for the speedup, going from n database queries to 1.
| ) | ||
| if idx <= 0 | ||
| error("Node ID #$value of type $type is not in the Node table.") | ||
| function NodeID(node_type, value::Integer, node_ids::Vector{NodeID})::NodeID |
There was a problem hiding this comment.
Often you know the node_type, and since node_ids has it as well we can check if it is correct. Especially in test code I find NodeID(:Pump, 5, v) easier to read than NodeID(5, v). Both is possible however, we call the latter for e.g. listen_node_type handling where we don't know the type.
There was a problem hiding this comment.
LGTM, good to bring back the queries from N to 1. I remember having a discussion on this a while (year?) back, and we were opinion that even though it was suboptimal, it wasn't hurting is yet. Nice to see we have reached that stage/growth now.
Also, TIL about counter (I knew it existed, hadn't seen an example yet).
core/src/read.jl
Outdated
| @@ -1395,11 +1397,52 @@ function Parameters(db::DB, config::Config)::Parameters | |||
| return p | |||
| end | |||
|
|
|||
| function get_ids(db::DB, nodetype)::Vector{Int32} | |||
| sql = "SELECT node_id FROM Node WHERE node_type = $(esc_id(nodetype)) ORDER BY node_id" | |||
| function get_node_ints(db::DB, node_type)::Vector{Int32} | |||
There was a problem hiding this comment.
Nit: This signature is very similar to get_node_ids and I almost thought you made a typo. get_node_ids_int32 or similar?
For lhm_2024_12_3 this speeds up initialization (
Ribasim.Model(toml_path)), from 234 seconds to 4 seconds. Large models were especially slow to initialize since increate_graphwe were constructing NodeID structs for each node with a separate call to the database.This changes
get_node_idsto return aVector{NodeID}rather than aVector{Int32}. This means that the type and index of each ID is now directly included, leading to simpler code.