Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement defaultuserquota/defaultgroupquota #16283

Open
wants to merge 5 commits into
base: master
Choose a base branch
from

Conversation

seidelma
Copy link
Contributor

@seidelma seidelma commented Jun 18, 2024

Motivation and Context

This change implements default user and group quotas on Linux, an oft-requested feature present in Solaris 11.3.

#5431

Description

This change adds the 'defaultuserquota' and 'defaultgroupquota' properties to ZFS datasets, which apply a quota to users or groups that don't have a specific quota assigned:

# zfs create tank/fs
# zfs set compression=off tank/fs
# zfs set defaultuserquota=100M tank/fs
# dd if=/dev/zero of=/tank/fs/data.file
dd: writing to '/tank/fs/data.file': Disk quota exceeded

It includes tests (some of which fail by design since the behavior is slightly different from Oracle's) and manpage changes.

How Has This Been Tested?

The code changes were tested on Rocky Linux 9 (x86_64 and aarch64) and Ubuntu 24.04 (aarch64) using the zfs-test suite, as well as a comparative behavior analysis between OpenZFS and ZFS on Oracle Solaris 11.4 (x86_64).

In addition, upgrading OpenZFS from 2.1.15 as well as current master/HEAD to my branch was also tested on Rocky 9.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

@seidelma
Copy link
Contributor Author

If anyone can point me at some docs on how to correctly incorporate these changes into a FreeBSD source tree, I'll happily work on making this work on that as well.

@seidelma seidelma marked this pull request as ready for review June 24, 2024 17:18
@tonyhutter
Copy link
Contributor

@seidelma sorry no one has had a chance to take a look at this yet. I'll try to take a look at it early next week. In the meantime you may want to rebase it on master since there have been some important ZTS fixes that have gone in recently.

@tonyhutter
Copy link
Contributor

@seidelma would you mind re-basing your commits on top of master? We've had some test suite fixes go in recently that should fix some of the failures on Fedora 39-40.

@seidelma seidelma force-pushed the seidelma/feature/defaultuserquota branch from b4c6c6e to 6f20f8c Compare July 16, 2024 21:43
@tonyhutter
Copy link
Contributor

Hmm... checkstyle isn't happy with something ABI related. Can you try removing your ABI changes, let checkstyle run in the PR runner, and then download the new ABI files, and then include them in your commits?

@tonyhutter
Copy link
Contributor

Can you try removing your ABI changes, let checkstyle run in the PR runner, and then download the new ABI files, and then include them in your commits?

Actually, looks like the newly generated ABI files are here:
https://github.com/openzfs/zfs/actions/runs/9964537036/artifacts/1708333231

Would you mind including them in your commits?

@allanjude
Copy link
Contributor

To apply these sames changes to FreeBSD, from reviewing the diff, it looks like all you'll need to do is replicate the changes you made to module/os/linux/zfs/zfs_vfsops.c in module/os/freebsd/zfs/zfs_vfsops.c and the corresponding .h file change to struct zfsvfs

This change adds the 'defaultuserquota' and 'defaultgroupquota'
properties to ZFS datasets to apply a quota to users and groups that do
not have a specific quota assigned. The default quota checking
mechanism works alongside the existing 'userquota' and 'groupquota'
checks, only taking effect if no quota is assigned for a particular
user/group. This means that it's possible to exceed a default quota
by quite a lot before the user/groupused property is updated and
further writes are denied, which was already the case for
user/groupquota.

Default quotas are implemented similarly to 'normal' user quotas,
but instead of being user properties that are preserved across
snapshots, they default back to none. NB: this is different from the
observed Solaris behavior, which is to preserve default quotas across
snapshot/clone/promote.

For instance, Solaris has:
    # zfs set defaultuserquota=100M tank/fs
    # zfs snap tank/fs@snap
    # zfs clone tank/fs@snap tank/fs-clone
    # zfs get -H defaultuserquota tank/fs-clone
    tank/fs-clone   defaultuserquota        100M    -

Whereas this commit does:
    # zfs set defaultuserquota=100M tank/fs
    # zfs snap tank/fs@snap
    # zfs clone tank/fs@snap tank/fs-clone
    # zfs get -H defaultuserquota tank/fs-clone
    tank/fs-clone	defaultuserquota	none	default

It should also be possible to implement a default project quota using
an analogous process, if doing so makes sense.

Signed-off-by: Todd Seidelmann <[email protected]>
These are based on the existing userquota tests. Note that because the
default{user|group}quota behavior differs from Solaris during
snapshot/clone/promote, some of the tests fail by design.

Signed-off-by: Todd Seidelmann <[email protected]>
Add defaultuserquota and defaultgroupquota dataset properties to the
zfsprops.7 manpage.

Signed-off-by: Todd Seidelmann <[email protected]>
@seidelma seidelma force-pushed the seidelma/feature/defaultuserquota branch from 6f20f8c to 91b5c44 Compare August 2, 2024 21:25
@seidelma
Copy link
Contributor Author

seidelma commented Aug 3, 2024

I've tested the FreeBSD changes against 15-CURRENT (specifically, freebsd/freebsd-src@2b4aa2816). It appears to function with the manual testing I've done, so I'm working on getting the new test cases working with the automated testing framework.

@seidelma seidelma force-pushed the seidelma/feature/defaultuserquota branch from 0426a40 to 8889165 Compare August 9, 2024 16:25
@PHSpline
Copy link
Contributor

PHSpline commented Sep 2, 2024

Would it be useful to implement defaults for userobjquota and groupobjquota as well?

@amotin amotin added the Status: Code Review Needed Ready for review and testing label Oct 29, 2024
Copy link
Member

@amotin amotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need another ZAP to store just two quota values? Can't we just store them as a regular dataset properties? Or as alternative, store it in normal quota ZAPs, just with a special key value, since it seems we use string key there for some reason?

While there, wouldn't it be good for completeness to also add default project quotas?

And could you please rebase it to resolve the conflicts.

@amotin amotin added the Status: Revision Needed Changes are required for the PR to be accepted label Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Code Review Needed Ready for review and testing Status: Revision Needed Changes are required for the PR to be accepted
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants