Are High Disk Writes Expected?

[JohnTravolski]

I have been running Bitcoin Core 27.0 with txindex=1 and ElectrumX 1.16.0 on a Samsung 860 Evo SSD on Xubuntu. This is a dedicated machine doing nothing but running those two programs all day. I've also been using smartmontools to monitor the total disk writes once a day over time, and I've noticed the numbers are shockingly high. Obviously this is a concern since SSDs can only perform so many writes before they die.

Unfortunately, I only have a few data points right now, but all of these were recorded AFTER both the initial block download and after the transaction index was finished being built:

30 total gigabytes written to the SSD from June 24th to June 25th
108 total gigabytes written to the SSD from June 25th to June 26th
158 total gigabytes written to the SSD from June 26th to June 27th

To be clear, this is the difference in total GiB (512 bytes * Total LBAs Written /1024**3) written to the disk from day to day, not actual capacity being used. The amount of disk space being used is about what I would expect (increasing a few hundred megabytes per day).

I'm getting the LBAs written from smartmontools using:
sudo smartctl -a /dev/sda
and pulling the Total_LBAs_Written field, in case anybody else wants to try to monitor this for themselves and compare.

Obviously there's a lot more testing I need to do before I can determine what's causing this for sure but of course my intuition is that it's either related to Bitcoin Core or ElectrumX since the machine is dedicated to running them.

It is also possible I have the LBA size wrong for this drive and I'm inflating the gigabytes written without realizing it, but 512 bytes is standard from what I can tell.

Do these disk write numbers seem reasonable to you, or are these way too high?

[JohnTravolski]

I started using iotop --accumulated to measure disk writes and after just 15 minutes I'm seeing this:

Disk Writes          Command
3.36 G                python3 /home/user/Desktop/electrumx/electrumx_server
17.97 M               [jdb2/sda2-8]
6.52 M                systemd-journald
3.84 M                bitcoin-qt -min -chain=main [b-scheduler]
3.75 M                bitcoin-qt -min -chain=main [b-msghand]
3.36 M                tor --defaults-torrc /usr/share/tor/tor-service-defaults-torrc -f /etc/tor/torrc --RunAsDaemon 0

So it seems like it is ElectrumX that is doing all the writes. Do I have something configured wrong or is this expected?

My electrumx configuration is the same as in this issue:
#263
It looks like there is a CACHE_MB variable I can set. My system has sufficient RAM (16GB). Should I try increasing this significantly? Might this reduce disk writes? If not, is there something else I should do to limit the total disk writes so I don't chew through the SSD so fast?

[cculianu]

Are you using rocksdb or leveldb? Might be some leveldb and/or rocksdb setting that can get tuned? No idea.

[JohnTravolski]

Are you using rocksdb or leveldb? Might be some leveldb and/or rocksdb setting that can get tuned? No idea.

It is using whatever the default is, so I am pretty sure it is leveldb. I suppose I can try using rocksdb but I'm guessing it will be very similar. I don't even know where to start when it comes trying to tune the settings of each.

I moved the electrumx db folder to a dedicated SSD (not the OS drive, and not the drive bitcoin core is on) just to confirm it is the db folder casuing the problem, and it definitely is. The OS drive gigabytes written flatlined and the new drive jumped. I started monitoring the writes by the hour on this new drive and it seems to be averaging at about 5 GB per hour. It is pretty continuous for the most part.

Is anyone else running electrumx observing this for themselves? I'm wondering if I have something configured wrong and it's only me.