src-test/sys/cddl/contrib, branch main

[cddl] Fix lz4 function definitions to not tri pup compile.

2020-11-17T17:11:07Z

This tripped up in llvm compilation on amd64 noting that lz4_init/lz4_fini were lacking in being previously defined. Reviewed by: emaste, freqlabs, brooks Differential Revision: https://reviews.freebsd.org/D27240

Update OpenZFS to 2.0.0-rc3-gfc5966

2020-10-17T01:06:04Z

- fix panic due to tqid overflow - Improve libzfs_error_init messages - Expose zfetch_max_idistance tunable - Make dbufstat work on FreeBSD - Fix EIO after resuming receive of new dataset over an existing one

Add zstd support to the boot loader.

2020-10-12T22:19:07Z

Add support to the _STANDALONE environment enough bits of the kernel that we can compile it. We still have a small zstd_shim.c since there were 3 items that were a bit hard to nail down and may be cleaned up in the future. These go hand in hand with a number of commits to sys/sys in the past weeks, should this need be MFCd. Discussed with: mmacy (in review and on IRC/Slack) Reviewed by: freqlabs (on openzfs repo) Differential Revision: https://reviews.freebsd.org/D26218

ZFS: band-aid for -DNO_CLEAN

2020-08-25T23:35:55Z

Submitted by: Neal Chauhan Approved by: imp@ Differential Revision: https://reviews.freebsd.org/D26183

Merge OpenZFS support in to HEAD.

2020-08-25T02:21:27Z

The primary benefit is maintaining a completely shared code base with the community allowing FreeBSD to receive new features sooner and with less effort. I would advise against doing 'zpool upgrade' or creating indispensable pools using new features until this change has had a month+ to soak. Work on merging FreeBSD support in to what was at the time "ZFS on Linux" began in August 2018. I first publicly proposed transitioning FreeBSD to (new) OpenZFS on December 18th, 2018. FreeBSD support in OpenZFS was finally completed in December 2019. A CFT for downstreaming OpenZFS support in to FreeBSD was first issued on July 8th. All issues that were reported have been addressed or, for a couple of less critical matters there are pull requests in progress with OpenZFS. iXsystems has tested and dogfooded extensively internally. The TrueNAS 12 release is based on OpenZFS with some additional features that have not yet made it upstream. Improvements include: project quotas, encrypted datasets, allocation classes, vectorized raidz, vectorized checksums, various command line improvements, zstd compression. Thanks to those who have helped along the way: Ryan Moeller, Allan Jude, Zack Welch, and many others. Sponsored by: iXsystems, Inc. Differential Revision: https://reviews.freebsd.org/D25872

zfs: add an option to the bootloader to rewind the ZFS checkpoint

2020-08-18T19:48:04Z

The checkpoints are another way of keeping the state of ZFS. During the rewind, the pool has to be exported. This makes checkpoints unusable when using ZFS as root. Add the option to rewind the ZFS checkpoint at the boot time. If checkpoint exists, a new option for rewinding a checkpoint will appear in the bootloader menu. We fully support boot environments. If the rewind option is selected, the boot loader will show a list of boot environments that existed before the checkpoint. Reviewed by: tsoome, allanjude, kevans (ok with high-level overview) Differential Revision: https://reviews.freebsd.org/D24920

Fix linker error in libuutil with recent LLVM

2020-08-07T16:04:21Z

Not marking the function as static can result in a linker error: undefined reference to __assfail [--no-allow-shlib-undefined] I noticed this error after updating our CHERI LLVM to the latest upstream LLVM HEAD revision. This change effectively reverts r329984 and marks dmu_buf_init_user as static (which keeps the GCC build happy). Reviewed By: #zfs, asomers, freqlabs, mav Differential Revision: https://reviews.freebsd.org/D25663

Fix cddl tools bootstrapping on macOS and Linux

2020-08-07T16:03:55Z

Reviewed By: brooks Differential Revision: https://reviews.freebsd.org/D25979

MFOpenZFS: Add support for boot environment data to be stored in the label

2020-08-05T14:32:20Z

We are building new bootonce mechanism (previously zfs bootnext) and it is based on this OpenZFS change. Since this patch is nicely self contained, I am commiting it as is, and we can stack our changes. Original patch description follows: Modern bootloaders leverage data stored in the root filesystem to enable some of their powerful features. GRUB specifically has a grubenv file which can store large amounts of configuration data that can be read and written at boot time and during normal operation. This allows sysadmins to configure useful features like automated failover after failed boot attempts. Unfortunately, due to the Copy-on-Write nature of ZFS, the standard behavior of these tools cannot handle writing to ZFS files safely at boot time. We need an alternative way to store data that allows the bootloader to make changes to the data. This work is very similar to work that was done on Illumos to enable similar functionality in the FreeBSD bootloader. This patch is different in that the data being stored is a raw grubenv file; this file can store arbitrary variables and values, and the scripting provided by grub is powerful enough that special structures are not required to implement advanced behavior. We repurpose the second padding area in each label to store the grubenv file, protected by an embedded checksum. We add two ioctls to get and set this data, and libzfs_core and libzfs functions to access them more easily. There are no direct command line interfaces to these functions; these will be added directly to the bootloader utilities. Reviewed-by: Pavel Zakharov Reviewed-by: Matthew Ahrens Reviewed-by: Brian Behlendorf Signed-off-by: Paul Dagnelie Closes #10009 Obtained from: OpenZFS Sponsored by: Netflix, Klara Inc.

zfs_keys_nextboot array is missing ZPOOL_CONFIG_POOL_GUID and ZPOOL_CONFIG_GUID

2020-08-05T14:08:44Z

As we do check the incomint nvlist, we either need to list all possible keys or use wildcard. PR: 248462 Reported by: larafercue@gmail.com Sponsored by: Netflix, Klara Inc.