Bundle Analysis: add ability to ingest dynamic imports #445
Conversation
| @@ -0,0 +1,424 @@ | |||
| import json | |||
There was a problem hiding this comment.
This entire file is basically a carbon copy of v2, except what is identified below. Note that each time a new stats file json schema is updated we create a new parser version 95% of the code between each version would be the same.
| "initial": bool, | ||
| "files": [str], | ||
| "names": [str], | ||
| "dynamicImports": [str] |
There was a problem hiding this comment.
This is the only change to the schema, also the "version": "3"
| # reset parser state | ||
| self.module = None | ||
| self.module_chunk_unique_external_ids = [] | ||
|
|
There was a problem hiding this comment.
This function below is a new one, it collects all the pairs of (chunk, asset) that is a dynamic import to be saved into the DB at the end.
| self.db_session.execute(insert_modules) | ||
|
|
||
| self.db_session.flush() | ||
|
|
There was a problem hiding this comment.
Below is where we have the list of items to insert into the DB. Also a small above change
self.db_session.add_all(self.chunk_list)
compared to before of Chunk.__table__.insert().values
| asset_id integer not null, | ||
| primary key (chunk_id, asset_id), | ||
| foreign key (chunk_id) references chunks (id), | ||
| foreign key (asset_id) references assets (id) |
There was a problem hiding this comment.
Will we need any indexes on this?
create index idx_chunk_id on dynamic_imports (chunk_id);
create index idx_asset_id on dynamic_imports (asset_id);
There was a problem hiding this comment.
The good thing about these "databases" are that it is super small (like 100kB), it only matches bundle data for 1 singular commit. This means we don't need to worry about performance on querying things haha.
| self.module_list = [] | ||
|
|
||
| # dynamic imports: mapping between Chunk and each file name of its dynamic imports | ||
| self.dynamic_imports_mapping = defaultdict( |
There was a problem hiding this comment.
Thoughts on naming this dynamic_import_file_names_by_chunk or similar?
| ] | ||
| ) | ||
| if inserts: | ||
| self.db_session.execute(chunks_modules.insert(), inserts) |
There was a problem hiding this comment.
Do we need any transaction safety/rollback on the assets_chunks.insert() in above and this chunks_modules.insert() if one succeeds but the next fails?
There was a problem hiding this comment.
This entire parse should be already in a transaction, we don't actually call session.commit in here which writes everything into the filesystem, that happens on the caller of the parser.
| self.module_chunk_unique_external_ids = [] | ||
|
|
||
| def _parse_dynamic_imports(self) -> List[dict]: | ||
| dynamic_imports_list = [] |
There was a problem hiding this comment.
May be nice to have a more specific type on this dynamic_imports_list and then we can omit the type from the var name. Can see that being a pain though
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #445 +/- ##
==========================================
- Coverage 90.69% 90.02% -0.68%
==========================================
Files 394 324 -70
Lines 12002 9310 -2692
Branches 2018 1663 -355
==========================================
- Hits 10885 8381 -2504
+ Misses 1028 865 -163
+ Partials 89 64 -25
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Legal Boilerplate
Look, I get it. The entity doing business as "Sentry" was incorporated in the State of Delaware in 2015 as Functional Software, Inc. In 2022 this entity acquired Codecov and as result Sentry is going to need some rights from me in order to utilize my contributions in this PR. So here's the deal: I retain all rights, title and interest in and to my contributions, and by keeping this boilerplate intact I confirm that Sentry can use, modify, copy, and redistribute my contributions, under Sentry's choice of terms.