Replies: 4 comments 4 replies
-
|
unit tests are by definition meant to test the smallest unit of code so you're better off mocking your use of ZFS in your tests. edit: I realise not everyone understands what mocking is; https://trepo.tuni.fi/bitstream/handle/123456789/24806/%C3%85hman.pdf?sequence=4&isAllowed=y a few pages in you'll find the table of contents of how you mock kernel level code. you essentially fake the existence of the kernel and return whatever type of data structure the calling code requires through stub methods. it's a handy way to run unit tests on the actual kernel without having to boot it. and for userland code, |
Beta Was this translation helpful? Give feedback.
-
|
Thank you, bghira! I'm not actually in a position to mock kernel code but I will read that presentation. I'm running a series of unit and functional tests against ZFS, I'm not testing ZFS code or other kernel code at this time. I'd like to be able to move the testing I run for pre-commit into a container-like environment and I think WSL is a great target for that if I can get it to work. So the problem I'm looking for help with isn't the testing side, so much as on how to deploy a Ubuntu system with a statically linked kernel that can run ZFS in WSL. The showstopper problem is that I can't create a zpool. It is probably just an issue with configuration, but perhaps it is just not possible? These things are working:
These things are not working:
What I would see as a good outcome of this discussion is to understand what I'm doing wrong in configuration or deployment, and then if feasible add an error check that catches the problem and throws a more helpful error message. |
Beta Was this translation helpful? Give feedback.
-
I don't understand, this seems directly contradictory, can you elaborate? |
Beta Was this translation helpful? Give feedback.
-
|
as for the rest, relating to WSL, this project doesn't support WSL, so you probably don't have much luck here. I believe it's a problem with WSL2 - have you tried to do a .ko and insmod that? |
Beta Was this translation helpful? Give feedback.
-
I built it as a loadable module and received the same result: # lsmod
Module Size Used by
# modprobe zfs
# lsmod
Module Size Used by
zfs 4300800 0
zunicode 335872 1 zfs
zzstd 569344 1 zfs
zlua 172032 1 zfs
zavl 16384 1 zfs
icp 311296 1 zfs
zcommon 86016 2 zfs,icp
znvpair 69632 2 zfs,zcommon
spl 90112 6 zfs,icp,zzstd,znvpair,zcommon,zavl
# zpool create testpool /root/vdev1.img
cannot create 'testpool': no such pool or dataset |
Beta Was this translation helpful? Give feedback.
-
I'm running functional tests of my code that uses ZFS. I need to be able to create a pool as part of that testing. |
Beta Was this translation helpful? Give feedback.
-
|
The loadable kernel module version is actually making more progress than I realized. Here is some strace output: Open up the target as a block device (it's a file): openat(AT_FDCWD, "/root/vdev1.img", O_RDONLY|O_DIRECT|O_CLOEXEC) = 6
ioctl(6, BLKSSZGET, 0x7fff7b84d80c) = -1 ENOTTY (Inappropriate ioctl for device)
close(6) = 0
stat("/root/vdev1.img", {st_mode=S_IFREG|0644, st_size=73728000, ...}) = 0Now it will attempt to access as a regular file: openat(AT_FDCWD, "/root/vdev1.img", O_RDONLY|O_DIRECT) = 6
ioctl(6, SG_IO, {interface_id='S', dxfer_direction=SG_DXFER_FROM_DEV, cmd_len=6, cmdp="\x12\x00\x00\x00\x60\x00", mx_sb_len=32,
iovec_count=0, dxfer_len=96, timeout=10, flags=0, dxferp="\x1d\x80\x61\x4b\x49\x56\x00\x00\x80\xdf\x5b\x4c\x49\x56\x00\x00\x30\xd9\x84\x7b\xff\x7f\x00\x00\x78\xef\x70\x63\x70\x7f\x00\x00"..., status=0, masked_status=0, msg_status=0, sb_len_wr=0, sbp="", host_status=0, driver_status=0, resid=0, duration=0, info=0}) = -1 ENOTTY (Inappropriate ioctl for device)
close(6) = 0
openat(AT_FDCWD, "/root/vdev1.img", O_RDONLY) = 6
fstat(6, {st_mode=S_IFREG|0644, st_size=73728000, ...}) = 0
mmap(NULL, 466944, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7062aeb000
sched_getparam(137, [0]) = 0
sched_getscheduler(137) = 0 (SCHED_OTHER)
rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
mmap(NULL, 37056, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f7062ae1000
mprotect(0x7f7062ae2000, 32960, PROT_READ|PROT_WRITE) = 0
clone(child_stack=0x7f7062ae6ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tid=[138], tls=0x7f7062ae97c0, child_tidptr=0x7f7062ae9a90) = 138
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
futex(0x7f7063036200, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x7fff7b84c3a8, FUTEX_WAIT_PRIVATE, 4, NULL) = -1 EAGAIN (Resource temporarily unavailable)
munmap(0x7f7062aeb000, 466944) = 0
close(6) = 0
openat(AT_FDCWD, "/testpool", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = -1 ENOENT (No such file or directory)trimmed out several missed loads of locale messages openat(AT_FDCWD, "/usr/share/locale-langpack/C/LC_MESSAGES/zfs-linux-user.mo", O_RDONLY) = -1 ENOENT (No such file or directory)ioctl(3, _IOC(_IOC_NONE, 0x5a, 0, 0), 0x7fff7b84b1d0) = -1 ENOENT (No such file or directory)
write(2, "cannot create 'testpool': no suc"..., 50cannot create 'testpool': no such pool or dataset
) = 50
close(3) = 0
close(4) = 0
close(5) = 0
exit_group(1) = ?
+++ exited with 1 +++ |
Beta Was this translation helpful? Give feedback.
-
|
I used exactly the same build script I used to build on WSL2 Ubuntu 20.04 to build on Hyper-V Ubuntu 20.04 and everything worked fine. |
Beta Was this translation helpful? Give feedback.
-
I'm trying to build a statically linked kernel for Windows 10 WSL, following a pattern I've seen successfully used here. I'm building from linux 5.13.9 and zfs 2.1.0.
The kernel builds fine as a built-in, and I can also build the loadable modules, but I think I am missing some system element to activate the module. For example, on a reference Ubuntu 20.04 with zfs 0.8.3-3 system, I can see that the (dynamically loaded) modules are operating:
# ls /sys/module/zfs coresize features.pool initsize notes properties.dataset refcnt srcversion uevent features.kernel holders initstate parameters properties.pool sections taint version(version info for my reference system):
# zpool --version zfs-0.8.3-3 zfs-kmod-0.8.3-3On my Windows 10 WSL session the same module:
# ls /sys/module/zfs parameters uevent version(version info for the statically linked WSL environment):
# zpool --version zfs-2.1.0-1 zfs-kmod-2.1.0-1I tried various things, the biggest stumbling block of all is that I can't create a zpool:
Is there some guide for configuring the environment for a non-ZFS-root scenario? I can't/won't have ZFS root volumes in this environment, at least not on this build. My sole purpose for even trying this is for a testing environment. I'd like to be able to start up a WSL environment, run several unit tests for a code base that uses ZFS, and then shut down, no data to be retained.
Thank you!
Update:
The zfs module is definitely being loaded at kernel boot because it printk's:
2# dmesg | grep ZFS [ 2.470483] ZFS: Loaded module v2.1.0-1, ZFS pool version 5000, ZFS filesystem version 5Update:
ztest seems to run:
Pass 5, SIGKILL, 0 ENOSPC, 22.9% of 3.00G used, 96% done, 12s to go Workload summary: Calls Time Function ----- ---- -------- 48 0s ztest_dmu_read_write 31 3m03s ztest_dmu_write_parallel 34 22s ztest_dmu_object_alloc_free 7 0s ztest_dmu_object_next_chunk 29 0s ztest_dmu_commit_callbacks 37 1s ztest_zap 38 0s ztest_zap_parallel 40 28s ztest_split_pool 37 2s ztest_zil_commit 7 8s ztest_zil_remount 23 17s ztest_dmu_read_write_zcopy 25 1m35s ztest_dmu_objset_create_destroy 33 1m00s ztest_dsl_prop_get_set 5 1s ztest_spa_prop_get_set 5 0s ztest_fzap 4 2s ztest_dmu_snapshot_create_destroy 6 5s ztest_spa_create_destroy 5 0s ztest_fault_inject 4 15s ztest_dmu_snapshot_hold 9 8s ztest_mmp_enable_disable 0 0s ztest_reguid 0 0s ztest_scrub 0 0s ztest_spa_upgrade 0 0s ztest_dsl_dataset_promote_busy 3 3s ztest_vdev_attach_detach 1 0s ztest_vdev_LUN_growth 0 0s ztest_vdev_add_remove 0 0s ztest_vdev_class_add 0 0s ztest_vdev_aux_add_remove 5 8s ztest_device_removal 1 0s ztest_spa_checkpoint_create_discard 6 7s ztest_initialize 5 6s ztest_trim 1 1s ztest_fletcher 1 1s ztest_fletcher_incr 4 0s ztest_verify_dnode_bt verifying concrete vdev 0, metaslab 11 of 12 ... Verifying deleted livelist entries Verifying metaslab entries Dataset mos [META], ID 0, cr_txg 4, 19.0M, 396 objects Dataset ztest/temp_17 [OTHER], ID 246, cr_txg 1612, 863K, 5 objects Dataset ztest/ds_5@5 [OTHER], ID 541, cr_txg 951, 29.9M, 35 objects Dataset ztest/ds_5@19 [OTHER], ID 140, cr_txg 989, 31.2M, 35 objects Dataset ztest/ds_5 [OTHER], ID 124, cr_txg 107, 63.3M, 36 objects Dataset ztest/ds_2@16 [OTHER], ID 158, cr_txg 825, 22.0M, 36 objects Dataset ztest/ds_2@2 [OTHER], ID 780, cr_txg 392, 15.5M, 34 objects Dataset ztest/ds_2@9 [OTHER], ID 1239, cr_txg 1257, 34.4M, 36 objects Dataset ztest/ds_2 [OTHER], ID 86, cr_txg 101, 41.6M, 36 objects Dataset ztest/temp_22 [OTHER], ID 448, cr_txg 1682, 3.49M, 5 objects Dataset ztest/temp_10@0 [OTHER], ID 401, cr_txg 1661, 1.28M, 5 objects Dataset ztest/temp_10 [OTHER], ID 390, cr_txg 1659, 4.40M, 5 objects Dataset ztest/temp_9 [OTHER], ID 304, cr_txg 1649, 224K, 1 objects Dataset ztest/temp_3@1 [OTHER], ID 557, cr_txg 1723, 1.42M, 5 objects Dataset ztest/temp_3@0 [OTHER], ID 495, cr_txg 1703, 527K, 5 objects Dataset ztest/temp_3 [OTHER], ID 470, cr_txg 1694, 336K, 5 objects Dataset ztest/temp_7 [OTHER], ID 478, cr_txg 1695, 288K, 5 objects Dataset ztest/temp_14 [OTHER], ID 369, cr_txg 1643, 415K, 5 objects Dataset ztest/ds_6 [OTHER], ID 133, cr_txg 109, 54.4M, 36 objects Dataset ztest/temp_21 [OTHER], ID 569, cr_txg 1725, 655K, 5 objects Dataset ztest/ds_1@15 [OTHER], ID 362, cr_txg 1643, 81.6M, 48 objects Dataset ztest/ds_1@22 [OTHER], ID 798, cr_txg 1143, 66.7M, 47 objects Dataset ztest/ds_1@1 [OTHER], ID 164, cr_txg 1535, 79.2M, 48 objects Dataset ztest/ds_1@sh1_8 [OTHER], ID 584, cr_txg 1729, 86.8M, 48 objects Dataset ztest/ds_1 [OTHER], ID 24, cr_txg 99, 86.8M, 48 objects Dataset ztest/temp_19 [OTHER], ID 576, cr_txg 1726, 256K, 1 objects Dataset ztest/temp_0 [OTHER], ID 1437, cr_txg 1306, 256K, 5 objects Dataset ztest/temp_13@0 [OTHER], ID 1412, cr_txg 1298, 911K, 5 objects Dataset ztest/temp_13 [OTHER], ID 1405, cr_txg 1296, 607K, 5 objects Dataset ztest/ds_3@10 [OTHER], ID 767, cr_txg 1131, 35.1M, 34 objects Dataset ztest/ds_3@17 [OTHER], ID 222, cr_txg 704, 26.9M, 34 objects Dataset ztest/ds_3 [OTHER], ID 107, cr_txg 103, 46.1M, 35 objects Dataset ztest/temp_2@0 [OTHER], ID 539, cr_txg 1720, 527K, 5 objects Dataset ztest/temp_2 [OTHER], ID 502, cr_txg 1703, 1.65M, 5 objects Dataset ztest/temp_8@0 [OTHER], ID 1355, cr_txg 1288, 863K, 5 objects Dataset ztest/temp_8 [OTHER], ID 1344, cr_txg 1285, 863K, 5 objects Dataset ztest/temp_11 [OTHER], ID 487, cr_txg 1699, 591K, 5 objects Dataset ztest/temp_5@3 [OTHER], ID 560, cr_txg 1086, 1.14M, 5 objects Dataset ztest/temp_5@0 [OTHER], ID 450, cr_txg 1050, 911K, 5 objects Dataset ztest/temp_5 [OTHER], ID 428, cr_txg 1048, 1.14M, 5 objects Dataset ztest/temp_16@1 [OTHER], ID 421, cr_txg 1671, 1.62M, 5 objects Dataset ztest/temp_16@0 [OTHER], ID 396, cr_txg 1661, 1.23M, 5 objects Dataset ztest/temp_16 [OTHER], ID 298, cr_txg 1649, 1.62M, 5 objects Dataset ztest/ds_4@11 [OTHER], ID 166, cr_txg 1535, 36.6M, 36 objects Dataset ztest/ds_4@18 [OTHER], ID 469, cr_txg 261, 16.4M, 29 objects Dataset ztest/ds_4 [OTHER], ID 115, cr_txg 105, 43.2M, 36 objects Dataset ztest/ds_0@7 [OTHER], ID 321, cr_txg 205, 17.8M, 31 objects Dataset ztest/ds_0@0 [OTHER], ID 435, cr_txg 931, 42.9M, 47 objects Dataset ztest/ds_0 [OTHER], ID 5, cr_txg 6, 80.5M, 47 objects Dataset ztest/temp_20 [OTHER], ID 529, cr_txg 1713, 256K, 1 objects Dataset ztest/temp_12@0 [OTHER], ID 276, cr_txg 1647, 1.36M, 5 objects Dataset ztest/temp_12@2 [OTHER], ID 385, cr_txg 1659, 543K, 5 objects Dataset ztest/temp_12 [OTHER], ID 339, cr_txg 1633, 543K, 5 objects Dataset ztest/temp_1@0 [OTHER], ID 323, cr_txg 1629, 1.33M, 5 objects Dataset ztest/temp_1 [OTHER], ID 235, cr_txg 1612, 1.33M, 5 objects Dataset ztest/temp_18 [OTHER], ID 254, cr_txg 1568, 479K, 5 objects Dataset ztest/temp_15@1 [OTHER], ID 544, cr_txg 1721, 1.12M, 5 objects Dataset ztest/temp_15 [OTHER], ID 519, cr_txg 1711, 479K, 5 objects Dataset ztest/temp_6@1 [OTHER], ID 1530, cr_txg 1356, 1.84M, 5 objects loading concrete vdev 0, metaslab 11 of 12 ... 674M completed ( 74MB/s) estimated time remaining: 0hr 00min 00sec Dataset ztest/temp_6 [OTHER], ID 1489, cr_txg 1327, 1.84M, 5 objects Dataset ztest [ZPL], ID 54, cr_txg 1, 32.0K, 0 objects Verified large_blocks feature refcount of 0 is correct Verified large_dnode feature refcount of 55 is correct Verified sha512 feature refcount of 17 is correct Verified skein feature refcount of 19 is correct Verified edonr feature refcount of 18 is correct Verified userobj_accounting feature refcount of 0 is correct Verified encryption feature refcount of 34 is correct Verified project_quota feature refcount of 0 is correct Verified redaction_bookmarks feature refcount of 0 is correct Verified redacted_datasets feature refcount of 0 is correct Verified bookmark_written feature refcount of 0 is correct Verified livelist feature refcount of 0 is correct Verified zstd_compress feature refcount of 41 is correct Verified device_removal feature refcount of 0 is correct Verified indirect_refcount feature refcount of 0 is correct Traversing all blocks to verify checksums and verify nothing leaked ... No leaks (block sum matches space maps exactly) bp count: 20361 ganged count: 5 bp logical: 616931328 avg: 30299 bp physical: 99420160 avg: 4882 compression: 6.21 bp allocated: 729477120 avg: 35827 compression: 0.85 bp deduped: 3870720 ref>1: 11 deduplication: 1.01 Normal class: 712785920 used: 22.13% Embedded log class 0 used: -nan% additional, non-pointer bps of type 0: 1147 Dittoed blocks on same vdev: 13553 capacity operations bandwidth ---- errors ---- description used avail read write read write read write cksum ztest 680M 2.34G 19.9K 0 96.7M 0 0 0 0 draid1 680M 2.34G 19.9K 0 96.7M 0 0 0 0 /tmp/ztest.0a 1.35K 0 6.62M 0 0 0 0 /tmp/ztest.1a 1.01K 0 4.95M 0 0 0 0 /tmp/ztest.2a 1.07K 0 5.36M 0 0 0 0 /tmp/ztest.3a 1.05K 0 5.15M 0 0 0 0 /tmp/ztest.4a 854 0 4.11M 0 0 0 0 /tmp/ztest.5a 1.02K 0 4.83M 0 0 0 0 draid1-0-0 1.08K 0 4.45M 0 0 0 0 /tmp/ztest.7a 1.19K 0 5.74M 0 0 0 0 /tmp/ztest.8a 1.54K 0 7.30M 0 0 0 0 /tmp/ztest.9a 1.43K 0 6.76M 0 0 0 0 /tmp/ztest.10a 1.76K 0 8.39M 0 0 0 0 /tmp/ztest.11a 1.70K 0 8.29M 0 0 0 0 /tmp/ztest.12a 1.64K 0 8.14M 0 0 0 0 /tmp/ztest.13a 1.38K 0 6.72M 0 0 0 0 /tmp/ztest.14a 1.35K 0 6.56M 0 0 0 0 /tmp/ztest.15a 1.55K 0 7.77M 0 0 0 0 ZFS_DBGMSG(zdb) START: spa.c:6083:spa_import(): spa_import: importing ztest spa_misc.c:418:spa_load_note(): spa_load(ztest, config trusted): LOADING vdev.c:152:vdev_dbgmsg(): file vdev '/tmp/ztest.0a': best uberblock found for spa ztest. txg 1729 spa_misc.c:418:spa_load_note(): spa_load(ztest, config untrusted): using uberblock with txg=1729 spa.c:8336:spa_async_request(): spa=ztest async request task=2048 spa_misc.c:418:spa_load_note(): spa_load(ztest, config trusted): LOADED spa.c:8336:spa_async_request(): spa=ztest async request task=32 ZFS_DBGMSG(zdb) END 3 killed, 2 completed, 60% kill rateBeta Was this translation helpful? Give feedback.
All reactions