Open
Description
Compiling schemeshard and schemeshard operations takes significant amount of time.
(8394 seconds, which is 2.5 hours on a single core or 8.7 minutes on a 16 cores)
And while schemeshard consists of several parts including the core, schema operations support mechanics, split-merge subsystem, background compaction subsystem, long running schema processes ("meta-operations") like database backup/restore, and so on, the real concern this issue is trying to address (beyond the total schemeshard building time) is the build time of the single schema operation.
Schemeshard includes implementations of dozens of scheme operations (78 to be precise)...
TODO...
- move
TCompactionPolicy
fromflat_scheme_op.proto
to some core .proto - and completely replace
TCompactionPolicy
with something else to use in public api -- current specification is unusable for that - break
ydb/core/base
dependency onflat_scheme_op.proto
-
localdb.h
-- remove header dependency onflat_scheme_op.proto
-
schemeshard_impl.h
-- split operations iface intoschemeshard__operation_iface.h
-
schemeshard_impl.h
-- remove use by operations - separate schemeshard core and operations "api"
- make non intrusive separation
- get rid of static_cast to TSchemeShard* in operations code
- real api, may be with intrusive changes in operations
- make anything coming from
schemeshard__operation_iface.h
forward declared - move static methods out of
TSchemeShard
(into some lib/utils module) -
schemeshard_tx_infly.h
-- remove dependency onflat_scheme_op.proto
andflat_tx_scheme.proto
-
schemeshard_identifiers.h
-- remove dependency onflat_scheme_op.proto
-
schemeshard__operation_memory_changes.h
-- make clean interface without leaking specific types -
schemeshard_info_types.h
-- separate different types into separate files -
schemeshard__operation_side_effects.h
-- make clean interface without leaking specific types -
schemeshard__operation_db_changes.h
-- also clean from specific types -
schemeshard.h
,schemeshard_private.h
-- separate event interface into client, core, operations etc interfaces and include relevant parts only where necessary -
schemeshard__operation_common.h
-- remove dependency onydb/core/base/hive.h
-
schemeshard__operation_common.h
-- remove dependency onschemeshard_private.h
-
schemeshard_utils.h
- do not include from
schemeshard_impl.h
and other heavy headers - move
TShardDeleter
out ofschemeshard_utils.h
- move
TSelfPinger
out ofschemeshard_utils.h
- may be dismantle even
- do not include from
- tests -- reduce influence of a single change on single ut_-module rebuild time
- ...
- build_bloat: add paths sorting to improve visual stability #10604
- base: move TCompactionPolicy out of flat_scheme_op.proto #10495
- schemeshard: split schemeshard__operation_common.cpp, move code from .h #10631
- change some event scopes from
struct
tonamespace
#10685 - rename event scope namespaces from TEv... to NEv... #10835 -- postponed until before stable-25-1
- schemeshard: forward declare events in a core header for operations #11655
- schemeshard: stop using schemeshard_utils.h as public header #11895
- schemeshard: remove schemeshard_utils.h from common dependencies #12012
... - DRAFT: schemeshard build time works #11670
Metadata
Metadata
Assignees
Labels
No labels