Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2.2] Various backports for the 2.2 release #149

Merged
merged 25 commits into from
Jul 25, 2023

Commits on Jul 20, 2023

  1. FreeBSD: Fix build on stable/13 after 1302506.

    Starting approximately from version 1302506 vn_lock_pair() grown two
    additional arguments following head.  There is a one week hole, but
    that is closet reference point we have.
    
    Reviewed-by: Mateusz Guzik <[email protected]>
    Signed-off-by:  Alexander Motin <[email protected]>
    Sponsored by:   iXsystems, Inc.
    Closes openzfs#15047
    amotin authored and behlendorf committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    1266ceb View commit details
    Browse the repository at this point in the history
  2. FreeBSD: catch up to __FreeBSD_version 1400093

    Reviewed-by: Alexander Motin <[email protected]>
    Signed-off-by: Mateusz Guzik <[email protected]>
    Closes openzfs#15036
    mjguzik authored and behlendorf committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    bf6cd30 View commit details
    Browse the repository at this point in the history
  3. Add missed DMU_PROJECTUSED_OBJECT prefetch.

    It seems 9c5167d "Project Quota on ZFS" missed to add prefetch
    for DMU_PROJECTUSED_OBJECT during scan (scrub/resilver).  It should
    not cause visible problems, but may affect scub/resilver performance.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by:	Alexander Motin <[email protected]>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#15024
    amotin authored and behlendorf committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    b4e630b View commit details
    Browse the repository at this point in the history
  4. Avoid extra snprintf() in dsl_deadlist_merge().

    Since we are already iterating the ZAP, we have exact string key to
    remove, we do not need to call zap_remove_int() with the int key we
    just converted, we can call zap_remove() for the original string.
    
    This should make no functional change, only a micro-optimization.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by:	Alexander Motin <[email protected]>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#15056
    
    (cherry picked from commit fdba8cb)
    amotin authored and behlendorf committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    e613e4b View commit details
    Browse the repository at this point in the history
  5. Fix raw receive with different indirect block size.

    Unlike regular receive, raw receive require destination to have the
    same block structure as the source.  In case of dnode reclaim this
    triggers two special cases, requiring special handling:
     - If dn_nlevels == 1, we can change the ibs, but dnode_set_blksz()
    should not dirty the data buffer if block size does not change, or
    durign receive dbuf_dirty_lightweight() will trigger assertion.
     - If dn_nlevels > 1, we just can't change the ibs, dnode_set_blksz()
    would fail and receive_object would trigger assertion, so we should
    destroy and recreate the dnode from scratch.
    
    Reviewed-by: Paul Dagnelie <[email protected]>
    Signed-off-by:	Alexander Motin <[email protected]>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#15039
    
    (cherry picked from commit c4e8742)
    amotin authored and behlendorf committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    56ed389 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2023

  1. Fix the ZFS checksum error histograms with larger record sizes

    My analysis in PR openzfs#14716 was incorrect.  Each histogram bucket contains
    the number of incorrect bits, by position in a 64-bit word, over the
    entire record.  8-bit buckets can overflow for record sizes above 2k.
    To forestall that, saturate each bucket at 255.  That should still get
    the point across: either all bits are equally wrong, or just a couple
    are.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by: Alan Somers <[email protected]>
    Sponsored-by: Axcient
    Closes openzfs#15049
    asomers authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    f917cf1 View commit details
    Browse the repository at this point in the history
  2. set autotrim default to 'off' everywhere

    As it turns out having autotrim default to 'on' on FreeBSD never really
    worked due to mess with defines where userland and kernel module were
    getting different default values (userland was defaulting to 'off',
    module was thinking it's 'on').
    
    Reviewed-by: Tino Reichardt <[email protected]>
    Reviewed-by: Brian Behlendorf <[email protected]>
    Reviewed-by: Alexander Motin <[email protected]>
    Signed-off-by: Yuri Pankov <[email protected]>
    Closes openzfs#15079
    Yuri Pankov authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    5299f4f View commit details
    Browse the repository at this point in the history
  3. Linux 6.5 compat: intptr_t definition is canonically signed

    Make the version here match that elsewhere in the kernel and system
    headers.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Reviewed-by: Alexander Motin <[email protected]>
    Signed-off-by: Coleman Kane <[email protected]>
    Closes openzfs#15058
    Signed-off-by: Brian Behlendorf <[email protected]>
    ckane authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    931dc70 View commit details
    Browse the repository at this point in the history
  4. Linux 6.5 compat: BLK_STS_NEXUS renamed to BLK_STS_RESV_CONFLICT

    This change was introduced in Linux commit
    7ba150834b840f6f5cdd07ca69a4ccf39df59a66
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Reviewed-by: Alexander Motin <[email protected]>
    Signed-off-by: Coleman Kane <[email protected]>
    Closes openzfs#15059
    ckane authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    1bc244a View commit details
    Browse the repository at this point in the history
  5. Linux 6.5 compat: disk_check_media_change() was added

    The disk_check_media_change() function was added which replaces
    bdev_check_media_change.  This change was introduced in 6.5rc1
    444aa2c58cb3b6cfe3b7cc7db6c294d73393a894 and the new function takes a
    gendisk* as its argument, no longer a block_device*. Thus, bdev->bd_disk
    is now used to pass the expected data.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by: Coleman Kane <[email protected]>
    Closes openzfs#15060
    ckane authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    73ba5df View commit details
    Browse the repository at this point in the history
  6. Do not request data L1 buffers on scan prefetch.

    Set ARC_FLAG_NO_BUF when prefetching data L1 buffers for scan.  We
    do not prefetch data L0 buffers, so we do not need the L1 buffers,
    only want them to be ready in ARC. This saves some CPU time on the
    buffers decompression.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by:	Alexander Motin <[email protected]>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#15029
    amotin authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    83b0967 View commit details
    Browse the repository at this point in the history
  7. Rollback before zfs root is mounted

    On my machines I observe random failures caused by rollback happening 
    after zfs root is mounted. I've observed two types of failures:
    
    - zfs-rollback-bootfs.service fails saying that rollback must be
      done just before mounting the dataset
    - boot process fails and rescue console is entered.
    
    After making this modification and testing it for couple of days 
    none of those problems have been observed anymore.
    
    I don't know if `dracut-mount.service` is still needed in the 
    `After` directive. Maybe someone else is able to address this?
    
    Reviewed-by: Gregory Bartholomew <[email protected]>
    Signed-off-by: Wojciech Małota-Wójcik <[email protected]>
    Closes openzfs#15025
    outofforest authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    f5f5a2d View commit details
    Browse the repository at this point in the history
  8. Ignore pool ashift property during vdev attachment

    Ashift can be set for a vdev only during its creation, and the
    top-level vdev does not change when a vdev is attached or replaced.
    The ashift property should not be used during attachment, as it
    does not allow attaching/replacing a vdev if the pool's ashift
    property is increased after the existing vdev was created. Instead,
    we should be able to attach the vdev if the attached vdev can
    satisfy the ashift requirement with its parent.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Reviewed-by: Alexander Motin <[email protected]>
    Signed-off-by: Ameer Hamza <[email protected]>
    Closes openzfs#15061
    ixhamza authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    d801170 View commit details
    Browse the repository at this point in the history
  9. Don't panic if setting vdev properties is unsupported for this vdev type

    Check that vdev has valid zap and bail out early.
    
    While here, move objid selection out of the loop, it's not going to
    change.
    
    Reviewed-by: Allan Jude <[email protected]>
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by: Yuri Pankov <[email protected]>
    Closes openzfs#15063
    Yuri Pankov authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    1a2e486 View commit details
    Browse the repository at this point in the history
  10. spa_min_alloc should be GCD, not min

    Since spa_min_alloc may not be a power of 2, unlike ashifts, in the
    case of DRAID, we should not select the minimal value among several
    vdevs. Rounding to a multiple of it is unlikely to work for other
    vdevs. Instead, using the greatest common divisor produces smaller
    yet more reasonable results.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Reviewed-by: Alexander Motin <[email protected]>
    Signed-off-by: Ameer Hamza <[email protected]>
    Closes openzfs#15067
    ixhamza authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    e037327 View commit details
    Browse the repository at this point in the history
  11. Fix zpl_test_super race with zfs_umount

    We cannot call zpl_enter in zpl_test_super, because zpl_test_super is
    under spinlock so we can't sleep, and also because zpl_test_super is
    called without sb->s_umount taken, so it's possible we would race with
    zfs_umount and call zpl_enter on freed zfsvfs.
    
    Here's an stack trace when this happens:
    [ 2379.114837] VERIFY(cvp->cv_magic == CV_MAGIC) failed
    [ 2379.114845] PANIC at spl-condvar.c:497:__cv_broadcast()
    [ 2379.114854] Kernel panic - not syncing: VERIFY(cvp->cv_magic == CV_MAGIC) failed
    [ 2379.115012] Call Trace:
    [ 2379.115019]  dump_stack+0x74/0x96
    [ 2379.115024]  panic+0x114/0x2f6
    [ 2379.115035]  spl_panic+0xcf/0xfc [spl]
    [ 2379.115477]  __cv_broadcast+0x68/0xa0 [spl]
    [ 2379.115585]  rrw_exit+0xb8/0x310 [zfs]
    [ 2379.115696]  rrm_exit+0x4a/0x80 [zfs]
    [ 2379.115808]  zpl_test_super+0xa9/0xd0 [zfs]
    [ 2379.115920]  sget+0xd1/0x230
    [ 2379.116033]  zpl_mount+0xdc/0x230 [zfs]
    [ 2379.116037]  legacy_get_tree+0x28/0x50
    [ 2379.116039]  vfs_get_tree+0x27/0xc0
    [ 2379.116045]  path_mount+0x2fe/0xa70
    [ 2379.116048]  do_mount+0x80/0xa0
    [ 2379.116050]  __x64_sys_mount+0x8b/0xe0
    [ 2379.116052]  do_syscall_64+0x35/0x50
    [ 2379.116054]  entry_SYSCALL_64_after_hwframe+0x61/0xc6
    [ 2379.116057] RIP: 0033:0x7f9912e8b26a
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by: Chunwei Chen <[email protected]>
    Closes openzfs#15077
    tuxoko authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    b221f43 View commit details
    Browse the repository at this point in the history
  12. zed: Fix zed ASSERT on slot power cycle

    We would see zed assert on one of our systems if we powered off a
    slot.  Further examination showed zfs_retire_recv() was reporting
    a GUID of 0, which in turn would return a NULL nvlist.  Add
    in a check for a zero GUID.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by: Tony Hutter <[email protected]>
    Closes openzfs#15084
    tonyhutter authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    8c81c0b View commit details
    Browse the repository at this point in the history
  13. Don't emit checksum histograms in ereport.fs.zfs.checksum events

    The checksum histograms were intended to be used with ATA and parallel
    SCSI, which are obsolete.  With modern storage hardware, they will
    almost always look like white noise; all bits will be wrong.  They only
    serve to bloat the event.  That's a particular problem on FreeBSD, where
    events must fit into a 1016 byte buffer.
    
    This fixes issue openzfs#14717 for RAIDZ pools, but not for mirror pools.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Reviewed-by: Rich Ercolani <[email protected]>
    Signed-off-by: Alan Somers <[email protected]>
    Sponsored-by: Axcient
    Closes openzfs#15052
    asomers authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    51a2b59 View commit details
    Browse the repository at this point in the history
  14. Don't emit cksum_{actual_expected} in ereport.fs.zfs.checksum events

    With anything but fletcher-4, even a tiny change in the input will cause
    the checksum value to change completely.  So knowing the actual and
    expected checksums doesn't provide much more information than "they
    don't match".  The harm in sending them is simply that they bloat the
    event.  In particular, on FreeBSD the event must fit into a 1016 byte
    buffer.
    
    Fixes openzfs#14717 for mirrored pools.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Reviewed-by: Rich Ercolani <[email protected]>
    Signed-off-by: Alan Somers <[email protected]>
    Sponsored-by: Axcient
    Closes openzfs#14717
    Closes openzfs#15052
    asomers authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    b6f618f View commit details
    Browse the repository at this point in the history
  15. Add explicit prefetches to bpobj_iterate().

    To simplify error handling bpobj_iterate_blkptrs() iterates through
    the list of block pointers backwards.  Unfortunately speculative
    prefetcher is currently unable to detect such patterns, that makes
    each block read there synchronous and very slow on HDD pools.
    
    According to my tests, added explicit prefetch reduces time needed
    to asynchronously delete 8 snapshots of 4 million blocks each from
    20 seconds to less than one, that should free sync thread for other
    useful work, such as async writes, scrub, etc.
    
    While there, plug one memory leak in case of bpobj_open() error and
    harmonize some variable names.
    
    Reviewed-by: Allan Jude <[email protected]>
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by:	Alexander Motin <[email protected]>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#15071
    amotin authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    8a6fde8 View commit details
    Browse the repository at this point in the history
  16. Adjust prefetch parameters.

    - Reduce maximum prefetch distance for 32bit platforms to 8MB as it
    was previously.  Those systems didn't grow much probably, so better
    stay conservative there.
     - Retire array_rd_sz tunable, blocking prefetch for large requests.
    We should not penalize applications trying to be more efficient. The
    speculative prefetcher by itself has reasonable distance limits, and
    1MB is not much at all these days.
    
    Reviewed-by: Allan Jude <[email protected]>
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by:	Alexander Motin <[email protected]>
    Sponsored by:	iXsystems, Inc.
    Closes openzfs#15072
    amotin authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    81be809 View commit details
    Browse the repository at this point in the history
  17. metaslab: tuneable to better control force ganging

    metaslab_force_ganging isn't enough to actually force ganging, because
    it still only forces 3% of the time. This adds
    metaslab_force_ganging_pct so we can configure how often to force
    ganging.
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by: Rob Norris <[email protected]>
    Sponsored-by: Klara, Inc.
    Sponsored-by: Wasabi Technology, Inc.
    Closes openzfs#15088
    robn authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    685ae44 View commit details
    Browse the repository at this point in the history
  18. shellcheck: disable "unreachable command" check [SC2317]

    This new check in 0.9.0 appears to have some issues with various forms
    of "early return", like trap, exit and return. This is tripping up (at
    least):
    
      cmd/zed/zed.d/history_event-zfs-list-cacher.sh
      /etc/zfs/zfs-functions
    
    Its not obvious what its complaining about or what the remedy is, so it
    seems sensible to disable this check for now.
    
    See also:
    
      https://www.shellcheck.net/wiki/SC2317
      koalaman/shellcheck#2542
      koalaman/shellcheck#2613
    
    Reviewed-by: Brian Behlendorf <[email protected]>
    Signed-off-by: Rob Norris <[email protected]>
    Closes openzfs#15089
    robn authored and behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    c5273e0 View commit details
    Browse the repository at this point in the history
  19. Tag 2.2.0-rc2

    Signed-off-by: Brian Behlendorf <[email protected]>
    behlendorf committed Jul 21, 2023
    Configuration menu
    Copy the full SHA
    7023248 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2023

  1. Configuration menu
    Copy the full SHA
    53b858b View commit details
    Browse the repository at this point in the history