Add more pebble database metrics collection

[weiihann]

Adds the following metrics:

db_compaction_time - Total time spent in database compaction
db_compaction_read - Total bytes read during compaction
db_compaction_write - Total bytes written during compaction
db_write_delay_count - Total write delay counts due to database compaction
db_write_delay_time - Total time spent in write stalls
db_disk_size - Tracks the size of all of the levels of the database
db_disk_write - Measures the effective amount of data written to disk
db_mem_comp - Tracks the amount of memory allocated for compaction
db_level0_comp - Tracks the number of level-zero compactions
db_nonlevel0_comp - Tracks the number of non level-zero compactions
db_seek_comp - Tracks the number of table compactions caused by read operations
db_manual_mem_alloc - Tracks the amount of non-managed memory currently allocated

[codecov]

Codecov Report

Attention: Patch coverage is 87.43719% with 25 lines in your changes missing coverage. Please review.

Project coverage is 78.79%. Comparing base (73be13e) to head (a915ecf).
Report is 87 commits behind head on main.

Files with missing lines	Patch %	Lines
db/pebble/db.go	87.56%	22 Missing and 1 partial ⚠️
db/remote/db.go	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2187      +/-   ##
==========================================
+ Coverage   78.56%   78.79%   +0.22%     
==========================================
  Files         102      102              
  Lines        9205     9383     +178     
==========================================
+ Hits         7232     7393     +161     
- Misses       1342     1361      +19     
+ Partials      631      629       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[rianhughes]

Do you think it would be worth adding a test for this?

[weiihann]

I don't think we need to. It's not easy to add tests for them as we have to create an actual pebbleDB instead of relying on in-memory. If it goes wrong, it shouldn't affect the main application, and most likely we will detect the weird behaviors when we see it on Grafana.

[rianhughes]

Do we not need other fields like BytesMoved or BytesIngested?

[weiihann]

You raised a good point :)

BytesMoved represents the bytes moved during compactions. Since we're already counting BytesCompacted, we don't need to include BytesMoved to avoid double-counting.

BytesIngested represents bytes added through ingestion (e.g., bulk loading). We should include this in our nWrite calculation as it represents additional data written to the database.

[github-actions]

This pull request is stale because it has been open 35 days with no activity. Remove stale label or comment or this will be closed in 14 days.