Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve schemeshard operations build time #10633

Open
8 of 22 tasks
ijon opened this issue Oct 18, 2024 · 0 comments
Open
8 of 22 tasks

Improve schemeshard operations build time #10633

ijon opened this issue Oct 18, 2024 · 0 comments
Assignees

Comments

@ijon
Copy link
Collaborator

ijon commented Oct 18, 2024

Compiling schemeshard and schemeshard operations takes significant amount of time.
(8394 seconds, which is 2.5 hours on a single core or 8.7 minutes on a 16 cores)

TODO...

  • schemeshard_impl.h -- split operations iface into schemeshard__operation_iface.h
  • schemeshard_impl.h -- remove use by operations
  • get rid of static_cast to TSchemeShard* in operations code
  • move static methods out of TSchemeShard (into some lib/utils module)
  • localdb.h -- remove header dependency on flat_scheme_op.proto
  • schemeshard_tx_infly.h -- remove dependency on flat_scheme_op.proto and flat_tx_scheme.proto
  • schemeshard_identifiers.h -- remove dependency on flat_scheme_op.proto
  • schemeshard__operation_memory_changes.h -- make clean interface without leaking specific types
  • schemeshard_info_types.h -- separate different types into separate files
  • schemeshard__operation_side_effects.h -- make clean interface without leaking specific types
  • schemeshard__operation_db_changes.h -- also clean from specific types
  • schemeshard.h, schemeshard_private.h -- separate event interface into client, core, operations etc interfaces and include relevant parts only where necessary
  • schemeshard__operation_common.h -- remove dependency on ydb/core/base/hive.h
  • schemeshard__operation_common.h -- remove dependency on schemeshard_private.h
  • schemeshard_utils.h -- clean up from schemeshard.h and other hard headers inclusion, may be dismantle even
  • break ydb/core/base dependency on flat_scheme_op.proto
  • move TShardDeleter out of schemeshard_utils.h
  • make anything coming from schemeshard__operation_iface.h forward declared
  • move TCompactionPolicy from flat_scheme_op.proto to some core .proto
  • and completely replace TCompactionPolicy with something else to use in public api -- current specification is unusable for that
  • tests -- reduce influence of a single change on single ut_-module rebuild time
  • etc

@ijon ijon self-assigned this Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant