Why does stat(2) .st_blocks reflect ZFS compression?

[dechamps]

(For context: st_blocks is what du shows unless --apparent-size is used.)

In systemd/systemd#18353, @poettering wrote:

ahum, that's quite some weird semantics, if zfs reports disk space usage like that. the general assumption is that .st_blocks and .st_size difference are about sparse files, not compression. If compression is used we cannot do sane calculations about sizes, I am not sure we possibly could. Maybe file a bug against ZFS that they break all forms of assumptions?

I think this is an interesting question.

On the hand, the Linux stat(2) manpage states:

The st_blocks field indicates the number of blocks allocated to the file, 512-byte units. (This may be smaller than st_size/512 when the file has holes.)

Which suggests that a file that isn't sparse should have st_blocks equal to st_size/512. ZFS breaks this assumption.

On the other hand, one could argue that st_blocks representing actual disk usage (including filesystem compression) is more useful, because that makes it easy to answer the question "how much actual disk space are these files using?", a question that would be much more difficult to answer otherwise if filesystem compression is used (I guess one would have to resort to zdb or something).

POSIX seems silent on this topic.

Is there a rationale for the original decision to make st_blocks reflect ZFS compression? Should this be revisited?

[ahrens]

The st_blocks field indicates the number of blocks allocated to the file, 512-byte units.

We interpret this to mean that st_blocks should reflect the amount of space allocated on disk for the file. It can be used to determine how much space a file is using. For ZFS, it's always reflected compression, metadata (indirect blocks), the copies property, and "unexpected" RAIDZ parity/pad space due to small blocks.

For traditional filesystems (verified on ext2 and Solaris UFS), st_blocks also includes metadata (indirect blocks).

I don't think it's reasonable for applications to make assumptions about the relationship between st_blocks and st_size.

[poettering]

Note that the Linux man pages make the relationship between .st_blocks and .st_size explicit, and say it's about sparse files. If you come up with different semantics you contradict the docs. Which means either your code should be adjusted to match expected, documented behaviour, or change the docs to match the code. But it's a bad situation if they contradict, as people rely on that, and ZFS is the only outlier in behaviour here.

[Blub]

The stat manpage literally also states that the semantics can differ even on the same system if for example NFS mounts are involved. To me the manpage really doesn't make any promises there at all.

[behlendorf]

That's a good question. It appears that the definition of st_blocks has been interpreted somewhat differently between platforms. This behavior was originally implemented on Solaris and it was defined in the man page as:

The total number of physical blocks of size 512 bytes actually allocated on disk. This field is not defined for block special or character special files

Unlike Linux's man page it makes no mention of why a file might take fewer physical blocks to store on disk. This is really an internal implementation detail that user space shouldn't need to concern itself with. This philosophy is the same on FreeBSD and it's man page says something very similar.

Actual number of blocks allocated for the file in 512-byte units. As short symbolic links are stored in the inode, this number may be zero.

Only the Linux man pages calls out sparse files as the sole reason why st_blocks might be smaller. I'm not sure why that's the case, but if I were to hazard a guess I'd wager it's because that it has historically been true for the vast majority of Linux filesystems. Even today transparent compression is a pretty rare feature for filesystems to support.

https://en.wikipedia.org/wiki/Comparison_of_file_systems#Allocation_and_layout_policies

Since the POSIX standards never clearly specified how this should be handled it was interpreted to mean the amount of space allocated on disk. Since as you mentioned that's just more useful.

(including filesystem compression) is more useful, because that makes it easy to answer the question "how much actual disk space are these files using?", a question that would be much more difficult to answer otherwise if filesystem compression is used

[ghost]

Sparse files are just a special/simple case of compression.