diff options
| author | Matt Macy <mmacy@FreeBSD.org> | 2020-10-01 23:09:24 +0000 |
|---|---|---|
| committer | Matt Macy <mmacy@FreeBSD.org> | 2020-10-01 23:09:24 +0000 |
| commit | e2228bd99047bb6a0cef0da931147b1f28f155c2 (patch) | |
| tree | 2859f65ec0723fe7424b97206b84c5d760d12b1c | |
| parent | 01ea34f1d8bc20a76845a15e968388679693ee44 (diff) | |
Notes
53 files changed, 1340 insertions, 495 deletions
diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md new file mode 100644 index 000000000000..9bc2e7ef0684 --- /dev/null +++ b/.github/CONTRIBUTING.md @@ -0,0 +1,345 @@ +# Contributing to OpenZFS +<p align="center"> + <img alt="OpenZFS Logo" + src="https://openzfs.github.io/openzfs-docs/_static/img/logo/480px-Open-ZFS-Secondary-Logo-Colour-halfsize.png"/> +</p> + +*First of all, thank you for taking the time to contribute!* + +By using the following guidelines, you can help us make OpenZFS even better. + +## Table Of Contents +[What should I know before I get +started?](#what-should-i-know-before-i-get-started) + + * [Get ZFS](#get-zfs) + * [Debug ZFS](#debug-zfs) + * [Where can I ask for help?](#where-can-I-ask-for-help) + +[How Can I Contribute?](#how-can-i-contribute) + + * [Reporting Bugs](#reporting-bugs) + * [Suggesting Enhancements](#suggesting-enhancements) + * [Pull Requests](#pull-requests) + * [Testing](#testing) + +[Style Guides](#style-guides) + + * [Coding Conventions](#coding-conventions) + * [Commit Message Formats](#commit-message-formats) + * [New Changes](#new-changes) + * [OpenZFS Patch Ports](#openzfs-patch-ports) + * [Coverity Defect Fixes](#coverity-defect-fixes) + * [Signed Off By](#signed-off-by) + +Helpful resources + + * [OpenZFS Documentation](https://openzfs.github.io/openzfs-docs/) + * [OpenZFS Developer Resources](http://open-zfs.org/wiki/Developer_resources) + * [Git and GitHub for beginners](https://openzfs.github.io/openzfs-docs/Developer%20Resources/Git%20and%20GitHub%20for%20beginners.html) + +## What should I know before I get started? + +### Get ZFS +You can build zfs packages by following [these +instructions](https://openzfs.github.io/openzfs-docs/Developer%20Resources/Building%20ZFS.html), +or install stable packages from [your distribution's +repository](https://openzfs.github.io/openzfs-docs/Getting%20Started/index.html). + +### Debug ZFS +A variety of methods and tools are available to aid ZFS developers. +It's strongly recommended that when developing a patch the `--enable-debug` +configure option should be set. This will enable additional correctness +checks and all the ASSERTs to help quickly catch potential issues. + +In addition, there are numerous utilities and debugging files which +provide visibility into the inner workings of ZFS. The most useful +of these tools are discussed in detail on the [Troubleshooting +page](https://openzfs.github.io/openzfs-docs/Basic%20Concepts/Troubleshooting.html). + +### Where can I ask for help? +The [zfs-discuss mailing +list](https://openzfs.github.io/openzfs-docs/Project%20and%20Community/Mailing%20Lists.html) +or IRC are the best places to ask for help. Please do not file +support requests on the GitHub issue tracker. + +## How Can I Contribute? + +### Reporting Bugs +*Please* contact us via the [zfs-discuss mailing +list](https://openzfs.github.io/openzfs-docs/Project%20and%20Community/Mailing%20Lists.html) +or IRC if you aren't certain that you are experiencing a bug. + +If you run into an issue, please search our [issue +tracker](https://github.com/openzfs/zfs/issues) *first* to ensure the +issue hasn't been reported before. Open a new issue only if you haven't +found anything similar to your issue. + +You can open a new issue and search existing issues using the public [issue +tracker](https://github.com/openzfs/zfs/issues). + +#### When opening a new issue, please include the following information at the top of the issue: +* What distribution (with version) you are using. +* The spl and zfs versions you are using, installation method (repository +or manual compilation). +* Describe the issue you are experiencing. +* Describe how to reproduce the issue. +* Including any warning/errors/backtraces from the system logs. + +When a new issue is opened, it is not uncommon for developers to request +additional information. + +In general, the more detail you share about a problem the quicker a +developer can resolve it. For example, providing a simple test case is always +exceptionally helpful. + +Be prepared to work with the developers investigating your issue. Your +assistance is crucial in providing a quick solution. They may ask for +information like: + +* Your pool configuration as reported by `zdb` or `zpool status`. +* Your hardware configuration, such as + * Number of CPUs. + * Amount of memory. + * Whether your system has ECC memory. + * Whether it is running under a VMM/Hypervisor. + * Kernel version. + * Values of the spl/zfs module parameters. +* Stack traces which may be logged to `dmesg`. + +### Suggesting Enhancements +OpenZFS is a widely deployed production filesystem which is under active +development. The team's primary focus is on fixing known issues, improving +performance, and adding compelling new features. + +You can view the list of proposed features +by filtering the issue tracker by the ["Type: Feature" +label](https://github.com/openzfs/zfs/issues?q=is%3Aopen+is%3Aissue+label%3A%22Type%3A+Feature%22). +If you have an idea for a feature first check this list. If your idea already +appears then add a +1 to the top most comment, this helps us gauge interest +in that feature. + +Otherwise, open a new issue and describe your proposed feature. Why is this +feature needed? What problem does it solve? + +### Pull Requests + +#### General + +* All pull requests must be based on the current master branch and apply +without conflicts. +* Please attempt to limit pull requests to a single commit which resolves +one specific issue. +* Make sure your commit messages are in the correct format. See the +[Commit Message Formats](#commit-message-formats) section for more information. +* When updating a pull request squash multiple commits by performing a +[rebase](https://git-scm.com/docs/git-rebase) (squash). +* For large pull requests consider structuring your changes as a stack of +logically independent patches which build on each other. This makes large +changes easier to review and approve which speeds up the merging process. +* Try to keep pull requests simple. Simple code with comments is much easier +to review and approve. +* All proposed changes must be approved by an OpenZFS organization member. +* If you have an idea you'd like to discuss or which requires additional testing, consider opening it as a draft pull request. +Once everything is in good shape and the details have been worked out you can remove its draft status. +Any required reviews can then be finalized and the pull request merged. + +#### Tests and Benchmarks +* Every pull request will by tested by the buildbot on multiple platforms by running the [zfs-tests.sh and zloop.sh]( +https://openzfs.github.io/openzfs-docs/Developer%20Resources/Building%20ZFS.html#running-zloop-sh-and-zfs-tests-sh) test suites. +* To verify your changes conform to the [style guidelines]( +https://github.com/openzfs/zfs/blob/master/.github/CONTRIBUTING.md#style-guides +), please run `make checkstyle` and resolve any warnings. +* Static code analysis of each pull request is performed by the buildbot; run `make lint` to check your changes. +* Test cases should be provided when appropriate. +This includes making sure new features have adequate code coverage. +* If your pull request improves performance, please include some benchmarks. +* The pull request must pass all required [ZFS +Buildbot](http://build.zfsonlinux.org/) builders before +being accepted. If you are experiencing intermittent TEST +builder failures, you may be experiencing a [test suite +issue](https://github.com/openzfs/zfs/issues?q=is%3Aissue+is%3Aopen+label%3A%22Type%3A+Test+Suite%22). +There are also various [buildbot options](https://openzfs.github.io/openzfs-docs/Developer%20Resources/Buildbot%20Options.html) +to control how changes are tested. + +### Testing +All help is appreciated! If you're in a position to run the latest code +consider helping us by reporting any functional problems, performance +regressions or other suspected issues. By running the latest code to a wide +range of realistic workloads, configurations and architectures we're better +able quickly identify and resolve potential issues. + +Users can also run the [ZFS Test +Suite](https://github.com/openzfs/zfs/tree/master/tests) on their systems +to verify ZFS is behaving as intended. + +## Style Guides + +### Repository Structure + +OpenZFS uses a standardised branching structure. +- The "development and main branch", is the branch all development should be based on. +- "Release branches" contain the latest released code for said version. +- "Staging branches" contain selected commits prior to being released. + +**Branch Names:** +- Development and Main branch: `master` +- Release branches: `zfs-$VERSION-release` +- Staging branches: `zfs-$VERSION-staging` + +`$VERSION` should be replaced with the `major.minor` version number. +_(This is the version number without the `.patch` version at the end)_ + +### Coding Conventions +We currently use [C Style and Coding Standards for +SunOS](http://www.cis.upenn.edu/%7Elee/06cse480/data/cstyle.ms.pdf) as our +coding convention. + +This repository has an `.editorconfig` file. If your editor [supports +editorconfig](https://editorconfig.org/#download), it will +automatically respect most of this project's whitespace preferences. + +Additionally, Git can help warn on whitespace problems as well: + +``` +git config --local core.whitespace trailing-space,space-before-tab,indent-with-non-tab,-tab-in-indent +``` + +### Commit Message Formats +#### New Changes +Commit messages for new changes must meet the following guidelines: +* In 72 characters or less, provide a summary of the change as the +first line in the commit message. +* A body which provides a description of the change. If necessary, +please summarize important information such as why the proposed +approach was chosen or a brief description of the bug you are resolving. +Each line of the body must be 72 characters or less. +* The last line must be a `Signed-off-by:` tag. See the +[Signed Off By](#signed-off-by) section for more information. + +An example commit message for new changes is provided below. + +``` +This line is a brief summary of your change + +Please provide at least a couple sentences describing the +change. If necessary, please summarize decisions such as +why the proposed approach was chosen or what bug you are +attempting to solve. + +Signed-off-by: Contributor <contributor@email.com> +``` + +#### OpenZFS Patch Ports +If you are porting OpenZFS patches, the commit message must meet +the following guidelines: +* The first line must be the summary line from the most important OpenZFS commit being ported. +It must begin with `OpenZFS dddd, dddd - ` where `dddd` are OpenZFS issue numbers. +* Provides a `Authored by:` line to attribute each patch for each original author. +* Provides the `Reviewed by:` and `Approved by:` lines from each original +OpenZFS commit. +* Provides a `Ported-by:` line with the developer's name followed by +their email for each OpenZFS commit. +* Provides a `OpenZFS-issue:` line with link for each original illumos +issue. +* Provides a `OpenZFS-commit:` line with link for each original OpenZFS commit. +* If necessary, provide some porting notes to describe any deviations from +the original OpenZFS commits. + +An example OpenZFS patch port commit message for a single patch is provided +below. +``` +OpenZFS 1234 - Summary from the original OpenZFS commit + +Authored by: Original Author <original@email.com> +Reviewed by: Reviewer One <reviewer1@email.com> +Reviewed by: Reviewer Two <reviewer2@email.com> +Approved by: Approver One <approver1@email.com> +Ported-by: ZFS Contributor <contributor@email.com> + +Provide some porting notes here if necessary. + +OpenZFS-issue: https://www.illumos.org/issues/1234 +OpenZFS-commit: https://github.com/openzfs/openzfs/commit/abcd1234 +``` + +If necessary, multiple OpenZFS patches can be combined in a single port. +This is useful when you are porting a new patch and its subsequent bug +fixes. An example commit message is provided below. +``` +OpenZFS 1234, 5678 - Summary of most important OpenZFS commit + +1234 Summary from original OpenZFS commit for 1234 + +Authored by: Original Author <original@email.com> +Reviewed by: Reviewer Two <reviewer2@email.com> +Approved by: Approver One <approver1@email.com> +Ported-by: ZFS Contributor <contributor@email.com> + +Provide some porting notes here for 1234 if necessary. + +OpenZFS-issue: https://www.illumos.org/issues/1234 +OpenZFS-commit: https://github.com/openzfs/openzfs/commit/abcd1234 + +5678 Summary from original OpenZFS commit for 5678 + +Authored by: Original Author2 <original2@email.com> +Reviewed by: Reviewer One <reviewer1@email.com> +Approved by: Approver Two <approver2@email.com> +Ported-by: ZFS Contributor <contributor@email.com> + +Provide some porting notes here for 5678 if necessary. + +OpenZFS-issue: https://www.illumos.org/issues/5678 +OpenZFS-commit: https://github.com/openzfs/openzfs/commit/efgh5678 +``` + +#### Coverity Defect Fixes +If you are submitting a fix to a +[Coverity defect](https://scan.coverity.com/projects/zfsonlinux-zfs), +the commit message should meet the following guidelines: +* Provides a subject line in the format of +`Fix coverity defects: CID dddd, dddd...` where `dddd` represents +each CID fixed by the commit. +* Provides a body which lists each Coverity defect and how it was corrected. +* The last line must be a `Signed-off-by:` tag. See the +[Signed Off By](#signed-off-by) section for more information. + +An example Coverity defect fix commit message is provided below. +``` +Fix coverity defects: CID 12345, 67890 + +CID 12345: Logically dead code (DEADCODE) + +Removed the if(var != 0) block because the condition could never be +satisfied. + +CID 67890: Resource Leak (RESOURCE_LEAK) + +Ensure free is called after allocating memory in function(). + +Signed-off-by: Contributor <contributor@email.com> +``` + +#### Signed Off By +A line tagged as `Signed-off-by:` must contain the developer's +name followed by their email. This is the developer's certification +that they have the right to submit the patch for inclusion into +the code base and indicates agreement to the [Developer's Certificate +of Origin](https://www.kernel.org/doc/html/latest/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin). +Code without a proper signoff cannot be merged. + +Git can append the `Signed-off-by` line to your commit messages. Simply +provide the `-s` or `--signoff` option when performing a `git commit`. +For more information about writing commit messages, visit [How to Write +a Git Commit Message](https://chris.beams.io/posts/git-commit/). + +#### Co-authored By +If someone else had part in your pull request, please add the following to the commit: +`Co-authored-by: Name <gitregistered@email.address>` +This is useful if their authorship was lost during squashing, rebasing, etc., +but may be used in any situation where there are co-authors. + +The email address used here should be the same as on the GitHub profile of said user. +If said user does not have their email address public, please use the following instead: +`Co-authored-by: Name <[username]@users.noreply.github.com>` diff --git a/cmd/zfs/zfs_main.c b/cmd/zfs/zfs_main.c index 20579157d7d6..42c180890fec 100644 --- a/cmd/zfs/zfs_main.c +++ b/cmd/zfs/zfs_main.c @@ -363,16 +363,16 @@ get_usage(zfs_help_t idx) return (gettext("\tuserspace [-Hinp] [-o field[,...]] " "[-s field] ...\n" "\t [-S field] ... [-t type[,...]] " - "<filesystem|snapshot>\n")); + "<filesystem|snapshot|path>\n")); case HELP_GROUPSPACE: return (gettext("\tgroupspace [-Hinp] [-o field[,...]] " "[-s field] ...\n" "\t [-S field] ... [-t type[,...]] " - "<filesystem|snapshot>\n")); + "<filesystem|snapshot|path>\n")); case HELP_PROJECTSPACE: return (gettext("\tprojectspace [-Hp] [-o field[,...]] " "[-s field] ... \n" - "\t [-S field] ... <filesystem|snapshot>\n")); + "\t [-S field] ... <filesystem|snapshot|path>\n")); case HELP_PROJECT: return (gettext("\tproject [-d|-r] <directory|file ...>\n" "\tproject -c [-0] [-d|-r] [-p id] <directory|file ...>\n" @@ -2481,11 +2481,13 @@ zfs_do_upgrade(int argc, char **argv) /* * zfs userspace [-Hinp] [-o field[,...]] [-s field [-s field]...] - * [-S field [-S field]...] [-t type[,...]] filesystem | snapshot + * [-S field [-S field]...] [-t type[,...]] + * filesystem | snapshot | path * zfs groupspace [-Hinp] [-o field[,...]] [-s field [-s field]...] - * [-S field [-S field]...] [-t type[,...]] filesystem | snapshot + * [-S field [-S field]...] [-t type[,...]] + * filesystem | snapshot | path * zfs projectspace [-Hp] [-o field[,...]] [-s field [-s field]...] - * [-S field [-S field]...] filesystem | snapshot + * [-S field [-S field]...] filesystem | snapshot | path * * -H Scripted mode; elide headers and separate columns by tabs. * -i Translate SID to POSIX ID. @@ -3191,7 +3193,7 @@ zfs_do_userspace(int argc, char **argv) } while (delim != NULL); } - if ((zhp = zfs_open(g_zfs, argv[0], ZFS_TYPE_FILESYSTEM | + if ((zhp = zfs_path_to_zhandle(g_zfs, argv[0], ZFS_TYPE_FILESYSTEM | ZFS_TYPE_SNAPSHOT)) == NULL) return (1); if (zhp->zfs_head_type != ZFS_TYPE_FILESYSTEM) { diff --git a/config/kernel-config-defined.m4 b/config/kernel-config-defined.m4 index 0ee4231cc2db..fe778e649454 100644 --- a/config/kernel-config-defined.m4 +++ b/config/kernel-config-defined.m4 @@ -91,7 +91,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_CONFIG_DEBUG_LOCK_ALLOC], [ AC_DEFUN([ZFS_AC_KERNEL_CONFIG_DEBUG_LOCK_ALLOC], [ AC_MSG_CHECKING([whether mutex_lock() is GPL-only]) - ZFS_LINUX_TEST_RESULT([config_debug_lock_alloc], [ + ZFS_LINUX_TEST_RESULT([config_debug_lock_alloc_license], [ AC_MSG_RESULT(no) ],[ AC_MSG_RESULT(yes) diff --git a/config/kernel-objtool.m4 b/config/kernel-objtool.m4 index bf60e7869213..c560c41954c6 100644 --- a/config/kernel-objtool.m4 +++ b/config/kernel-objtool.m4 @@ -6,10 +6,11 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_OBJTOOL], [ dnl # 4.6 API for compile-time stack validation ZFS_LINUX_TEST_SRC([objtool], [ #undef __ASSEMBLY__ + #include <asm/ptrace.h> #include <asm/frame.h> ],[ #if !defined(FRAME_BEGIN) - CTASSERT(1); + #error "FRAME_BEGIN is not defined" #endif ]) @@ -18,7 +19,7 @@ AC_DEFUN([ZFS_AC_KERNEL_SRC_OBJTOOL], [ #include <linux/frame.h> ],[ #if !defined(STACK_FRAME_NON_STANDARD) - CTASSERT(1); + #error "STACK_FRAME_NON_STANDARD is not defined." #endif ]) ]) diff --git a/configure.ac b/configure.ac index f149ab6d1b83..a1664151bc9a 100644 --- a/configure.ac +++ b/configure.ac @@ -204,6 +204,7 @@ AC_CONFIG_FILES([ tests/zfs-tests/Makefile tests/zfs-tests/callbacks/Makefile tests/zfs-tests/cmd/Makefile + tests/zfs-tests/cmd/badsend/Makefile tests/zfs-tests/cmd/btree_test/Makefile tests/zfs-tests/cmd/chg_usr_exec/Makefile tests/zfs-tests/cmd/devname2devid/Makefile diff --git a/contrib/initramfs/scripts/zfs b/contrib/initramfs/scripts/zfs index a795fd39f605..130aad5debd3 100644 --- a/contrib/initramfs/scripts/zfs +++ b/contrib/initramfs/scripts/zfs @@ -15,8 +15,8 @@ # See "4.5 Disable root prompt on the initramfs" of Securing Debian Manual: # https://www.debian.org/doc/manuals/securing-debian-howto/ch4.en.html shell() { - if type panic > /dev/null 2>&1; then - panic $@ + if command -v panic > /dev/null 2>&1; then + panic else /bin/sh fi @@ -26,22 +26,23 @@ shell() { # pools and mounting any filesystems. pre_mountroot() { - if type run_scripts > /dev/null 2>&1 && \ - [ -f "/scripts/local-top" -o -d "/scripts/local-top" ] + if command -v run_scripts > /dev/null 2>&1 then - [ "$quiet" != "y" ] && \ - zfs_log_begin_msg "Running /scripts/local-top" - run_scripts /scripts/local-top - [ "$quiet" != "y" ] && zfs_log_end_msg - fi + if [ -f "/scripts/local-top" ] || [ -d "/scripts/local-top" ] + then + [ "$quiet" != "y" ] && \ + zfs_log_begin_msg "Running /scripts/local-top" + run_scripts /scripts/local-top + [ "$quiet" != "y" ] && zfs_log_end_msg + fi - if type run_scripts > /dev/null 2>&1 && \ - [ -f "/scripts/local-premount" -o -d "/scripts/local-premount" ] - then - [ "$quiet" != "y" ] && \ - zfs_log_begin_msg "Running /scripts/local-premount" - run_scripts /scripts/local-premount - [ "$quiet" != "y" ] && zfs_log_end_msg + if [ -f "/scripts/local-premount" ] || [ -d "/scripts/local-premount" ] + then + [ "$quiet" != "y" ] && \ + zfs_log_begin_msg "Running /scripts/local-premount" + run_scripts /scripts/local-premount + [ "$quiet" != "y" ] && zfs_log_end_msg + fi fi } @@ -57,10 +58,10 @@ disable_plymouth() # Get a ZFS filesystem property value. get_fs_value() { - local fs="$1" - local value=$2 + fs="$1" + value=$2 - "${ZFS}" get -H -ovalue $value "$fs" 2> /dev/null + "${ZFS}" get -H -ovalue "$value" "$fs" 2> /dev/null } # Find the 'bootfs' property on pool $1. @@ -68,7 +69,7 @@ get_fs_value() # pool by exporting it again. find_rootfs() { - local pool="$1" + pool="$1" # If 'POOL_IMPORTED' isn't set, no pool imported and therefore # we won't be able to find a root fs. @@ -84,7 +85,7 @@ find_rootfs() # Make sure it's not '-' and that it starts with /. if [ "${ZFS_BOOTFS}" != "-" ] && \ - $(get_fs_value "${ZFS_BOOTFS}" mountpoint | grep -q '^/$') + get_fs_value "${ZFS_BOOTFS}" mountpoint | grep -q '^/$' then # Keep it mounted POOL_IMPORTED=1 @@ -101,14 +102,13 @@ find_rootfs() # Support function to get a list of all pools, separated with ';' find_pools() { - local CMD="$*" - local pools pool + CMD="$*" pools=$($CMD 2> /dev/null | \ grep -E "pool:|^[a-zA-Z0-9]" | \ sed 's@.*: @@' | \ - while read pool; do \ - echo -n "$pool;" + while read -r pool; do \ + printf "%s" "$pool;" done) echo "${pools%%;}" # Return without the last ';'. @@ -117,8 +117,6 @@ find_pools() # Get a list of all available pools get_pools() { - local available_pools npools - if [ -n "${ZFS_POOL_IMPORT}" ]; then echo "$ZFS_POOL_IMPORT" return 0 @@ -159,9 +157,8 @@ get_pools() # Filter out any exceptions... if [ -n "$ZFS_POOL_EXCEPTIONS" ] then - local found="" - local apools="" - local pool exception + found="" + apools="" OLD_IFS="$IFS" ; IFS=";" for pool in $available_pools @@ -194,8 +191,7 @@ get_pools() # Import given pool $1 import_pool() { - local pool="$1" - local dirs dir + pool="$1" # Verify that the pool isn't already imported # Make as sure as we can to not require '-f' to import. @@ -205,7 +201,7 @@ import_pool() # to something we can use later with the real import(s). We want to # make sure we find all by* dirs, BUT by-vdev should be first (if it # exists). - if [ -n "$USE_DISK_BY_ID" -a -z "$ZPOOL_IMPORT_PATH" ] + if [ -n "$USE_DISK_BY_ID" ] && [ -z "$ZPOOL_IMPORT_PATH" ] then dirs="$(for dir in $(echo /dev/disk/by-*) do @@ -213,7 +209,7 @@ import_pool() echo "$dir" | grep -q /by-vdev && continue [ ! -d "$dir" ] && continue - echo -n "$dir:" + printf "%s" "$dir:" done | sed 's,:$,,g')" if [ -d "/dev/disk/by-vdev" ] @@ -277,7 +273,7 @@ import_pool() # with more logging etc. load_module_initrd() { - if [ "$ZFS_INITRD_PRE_MOUNTROOT_SLEEP" > 0 ] + if [ "$ZFS_INITRD_PRE_MOUNTROOT_SLEEP" -gt 0 ] 2>/dev/null then if [ "$quiet" != "y" ]; then zfs_log_begin_msg "Sleeping for" \ @@ -288,9 +284,9 @@ load_module_initrd() fi # Wait for all of the /dev/{hd,sd}[a-z] device nodes to appear. - if type wait_for_udev > /dev/null 2>&1 ; then + if command -v wait_for_udev > /dev/null 2>&1 ; then wait_for_udev 10 - elif type wait_for_dev > /dev/null 2>&1 ; then + elif command -v wait_for_dev > /dev/null 2>&1 ; then wait_for_dev fi @@ -300,7 +296,7 @@ load_module_initrd() # Load the module load_module "zfs" || return 1 - if [ "$ZFS_INITRD_POST_MODPROBE_SLEEP" > 0 ] + if [ "$ZFS_INITRD_POST_MODPROBE_SLEEP" -gt 0 ] 2>/dev/null then if [ "$quiet" != "y" ]; then zfs_log_begin_msg "Sleeping for" \ @@ -316,12 +312,10 @@ load_module_initrd() # Mount a given filesystem mount_fs() { - local fs="$1" - local mountpoint + fs="$1" # Check that the filesystem exists - "${ZFS}" list -oname -tfilesystem -H "${fs}" > /dev/null 2>&1 - [ "$?" -ne 0 ] && return 1 + "${ZFS}" list -oname -tfilesystem -H "${fs}" > /dev/null 2>&1 || return 1 # Skip filesystems with canmount=off. The root fs should not have # canmount=off, but ignore it for backwards compatibility just in case. @@ -333,14 +327,14 @@ mount_fs() # Need the _original_ datasets mountpoint! mountpoint=$(get_fs_value "$fs" mountpoint) - if [ "$mountpoint" = "legacy" -o "$mountpoint" = "none" ]; then + if [ "$mountpoint" = "legacy" ] || [ "$mountpoint" = "none" ]; then # Can't use the mountpoint property. Might be one of our # clones. Check the 'org.zol:mountpoint' property set in # clone_snap() if that's usable. mountpoint=$(get_fs_value "$fs" org.zol:mountpoint) - if [ "$mountpoint" = "legacy" -o \ - "$mountpoint" = "none" -o \ - "$mountpoint" = "-" ] + if [ "$mountpoint" = "legacy" ] || + [ "$mountpoint" = "none" ] || + [ "$mountpoint" = "-" ] then if [ "$fs" != "${ZFS_BOOTFS}" ]; then # We don't have a proper mountpoint and this @@ -396,10 +390,10 @@ mount_fs() # Unlock a ZFS native encrypted filesystem. decrypt_fs() { - local fs="$1" - + fs="$1" + # If pool encryption is active and the zfs command understands '-o encryption' - if [ "$(zpool list -H -o feature@encryption $(echo "${fs}" | awk -F\/ '{print $1}'))" = 'active' ]; then + if [ "$(zpool list -H -o feature@encryption "$(echo "${fs}" | awk -F/ '{print $1}')")" = 'active' ]; then # Determine dataset that holds key for root dataset ENCRYPTIONROOT="$(get_fs_value "${fs}" encryptionroot)" @@ -427,7 +421,7 @@ decrypt_fs() TRY_COUNT=$((TRY_COUNT - 1)) done - # Prompt with systemd, if active + # Prompt with systemd, if active elif [ -e /run/systemd/system ]; then echo "systemd-ask-password" > /run/zfs_console_askpwd_cmd while [ $TRY_COUNT -gt 0 ]; do @@ -454,7 +448,7 @@ decrypt_fs() # Destroy a given filesystem. destroy_fs() { - local fs="$1" + fs="$1" [ "$quiet" != "y" ] && \ zfs_log_begin_msg "Destroying '$fs'" @@ -489,9 +483,9 @@ destroy_fs() # mounted with a 'zfs mount -a' in the init/systemd scripts). clone_snap() { - local snap="$1" - local destfs="$2" - local mountpoint="$3" + snap="$1" + destfs="$2" + mountpoint="$3" [ "$quiet" != "y" ] && zfs_log_begin_msg "Cloning '$snap' to '$destfs'" @@ -529,7 +523,7 @@ clone_snap() # Rollback a given snapshot. rollback_snap() { - local snap="$1" + snap="$1" [ "$quiet" != "y" ] && zfs_log_begin_msg "Rollback $snap" @@ -559,9 +553,8 @@ rollback_snap() # to the user to choose from. ask_user_snap() { - local fs="$1" - local i=1 - local SNAP snapnr snap debug + fs="$1" + i=1 # We need to temporarily disable debugging. Set 'debug' so we # remember to enabled it again. @@ -574,16 +567,16 @@ ask_user_snap() # Because we need the resulting snapshot, which is sent on # stdout to the caller, we use stderr for our questions. echo "What snapshot do you want to boot from?" > /dev/stderr - while read snap; do + while read -r snap; do echo " $i: ${snap}" > /dev/stderr - eval `echo SNAP_$i=$snap` + eval "$(echo SNAP_$i=$snap)" i=$((i + 1)) done <<EOT $("${ZFS}" list -H -oname -tsnapshot -r "${fs}") EOT - echo -n " Snap nr [1-$((i-1))]? " > /dev/stderr - read snapnr + echo "%s" " Snap nr [1-$((i-1))]? " > /dev/stderr + read -r snapnr # Re-enable debugging. if [ -n "${debug}" ]; then @@ -591,16 +584,16 @@ EOT set -x fi - echo "$(eval echo "$"SNAP_$snapnr)" + echo "$(eval echo '$SNAP_'$snapnr)" } setup_snapshot_booting() { - local snap="$1" - local s destfs subfs mountpoint retval=0 filesystems fs + snap="$1" + retval=0 - # Make sure that the snapshot specified actually exist. - if [ ! $(get_fs_value "${snap}" type) ] + # Make sure that the snapshot specified actually exists. + if [ ! "$(get_fs_value "${snap}" type)" ] then # Snapshot does not exist (...@<null> ?) # ask the user for a snapshot to use. @@ -617,7 +610,7 @@ setup_snapshot_booting() then # If the destination dataset for the clone # already exists, destroy it. Recursively - if [ $(get_fs_value "${rootfs}_${snapname}" type) ]; then + if [ "$(get_fs_value "${rootfs}_${snapname}" type)" ]; then filesystems=$("${ZFS}" list -oname -tfilesystem -H \ -r -Sname "${ZFS_BOOTFS}") for fs in $filesystems; do @@ -652,8 +645,8 @@ setup_snapshot_booting() # with clone_snap(). If legacy or none, then use # the sub fs value. mountpoint=$(get_fs_value "${s%%@*}" mountpoint) - if [ "$mountpoint" = "legacy" -o \ - "$mountpoint" = "none" ] + if [ "$mountpoint" = "legacy" ] || \ + [ "$mountpoint" = "none" ] then if [ -n "${subfs}" ]; then mountpoint="${subfs}" @@ -678,8 +671,6 @@ setup_snapshot_booting() # This is the main function. mountroot() { - local snaporig snapsub destfs pool POOLS - # ---------------------------------------------------------------- # I N I T I A L S E T U P @@ -742,7 +733,7 @@ mountroot() # No longer set in the defaults file, but it could have been set in # get_pools() in some circumstances. If it's something, but not 'yes', # it's no good to us. - [ -n "$USE_DISK_BY_ID" -a "$USE_DISK_BY_ID" != 'yes' ] && \ + [ -n "$USE_DISK_BY_ID" ] && [ "$USE_DISK_BY_ID" != 'yes' ] && \ unset USE_DISK_BY_ID # ---------------------------------------------------------------- @@ -788,12 +779,12 @@ mountroot() # ------------ # If we have 'ROOT' (see above), but not 'ZFS_BOOTFS', then use # 'ROOT' - [ -n "$ROOT" -a -z "${ZFS_BOOTFS}" ] && ZFS_BOOTFS="$ROOT" + [ -n "$ROOT" ] && [ -z "${ZFS_BOOTFS}" ] && ZFS_BOOTFS="$ROOT" # ------------ # Check for the `-B zfs-bootfs=%s/%u,...` kind of parameter. # NOTE: Only use the pool name and dataset. The rest is not - # supported by ZoL (whatever it's for). + # supported by OpenZFS (whatever it's for). if [ -z "$ZFS_RPOOL" ] then # The ${zfs-bootfs} variable is set at the kernel command @@ -809,11 +800,11 @@ mountroot() # ------------ # No root fs or pool specified - do auto detect. - if [ -z "$ZFS_RPOOL" -a -z "${ZFS_BOOTFS}" ] + if [ -z "$ZFS_RPOOL" ] && [ -z "${ZFS_BOOTFS}" ] then # Do auto detect. Do this by 'cheating' - set 'root=zfs:AUTO' # which will be caught later - ROOT=zfs:AUTO + ROOT='zfs:AUTO' fi # ---------------------------------------------------------------- @@ -858,7 +849,7 @@ mountroot() fi # Import the pool (if not already done so in the AUTO check above). - if [ -n "$ZFS_RPOOL" -a -z "${POOL_IMPORTED}" ] + if [ -n "$ZFS_RPOOL" ] && [ -z "${POOL_IMPORTED}" ] then [ "$quiet" != "y" ] && \ zfs_log_begin_msg "Importing ZFS root pool '$ZFS_RPOOL'" @@ -971,7 +962,7 @@ mountroot() touch /run/zfs_unlock_complete if [ -e /run/zfs_unlock_complete_notify ]; then - read zfs_unlock_complete_notify < /run/zfs_unlock_complete_notify + read -r zfs_unlock_complete_notify < /run/zfs_unlock_complete_notify fi # ------------ @@ -989,8 +980,8 @@ mountroot() echo echo "=> waiting for ENTER before continuing because of 'zfsdebug=1'. " - echo -n " 'c' for shell, 'r' for reboot, 'ENTER' to continue. " - read b + printf "%s" " 'c' for shell, 'r' for reboot, 'ENTER' to continue. " + read -r b [ "$b" = "c" ] && /bin/sh [ "$b" = "r" ] && reboot -f @@ -1000,12 +991,14 @@ mountroot() # ------------ # Run local bottom script - if type run_scripts > /dev/null 2>&1 && \ - [ -f "/scripts/local-bottom" -o -d "/scripts/local-bottom" ] + if command -v run_scripts > /dev/null 2>&1 then - [ "$quiet" != "y" ] && \ - zfs_log_begin_msg "Running /scripts/local-bottom" - run_scripts /scripts/local-bottom - [ "$quiet" != "y" ] && zfs_log_end_msg + if [ -f "/scripts/local-bottom" ] || [ -d "/scripts/local-bottom" ] + then + [ "$quiet" != "y" ] && \ + zfs_log_begin_msg "Running /scripts/local-bottom" + run_scripts /scripts/local-bottom + [ "$quiet" != "y" ] && zfs_log_end_msg + fi fi } diff --git a/contrib/intel_qat/patch/0001-cryptohash.diff b/contrib/intel_qat/patch/0001-cryptohash.diff new file mode 100644 index 000000000000..2d87c8f36256 --- /dev/null +++ b/contrib/intel_qat/patch/0001-cryptohash.diff @@ -0,0 +1,17 @@ +cryptohash.h was dropped and merged with crypto/sha.sh in 5.8 kernel. Details in: +https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=228c4f265c6eb60eaa4ed0edb3bf7c113173576c + +--- +diff --git a/quickassist/utilities/osal/src/linux/kernel_space/OsalCryptoInterface.c b/quickassist/utilities/osal/src/linux/kernel_space/OsalCryptoInterface.c +index 4c389da..e602377 100644 +--- a/quickassist/utilities/osal/src/linux/kernel_space/OsalCryptoInterface.c ++++ b/quickassist/utilities/osal/src/linux/kernel_space/OsalCryptoInterface.c +@@ -66,7 +66,7 @@ + + #include "Osal.h" + #include <linux/crypto.h> +-#include <linux/cryptohash.h> ++#include <crypto/sha.h> + #include <linux/version.h> + #if (LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,29)) + #include <crypto/internal/hash.h> diff --git a/contrib/intel_qat/patch/0001-pci_aer.diff b/contrib/intel_qat/patch/0001-pci_aer.diff new file mode 100644 index 000000000000..7516ac4fee75 --- /dev/null +++ b/contrib/intel_qat/patch/0001-pci_aer.diff @@ -0,0 +1,20 @@ +In kernel 5.7 the pci_cleanup_aer_uncorrect_error_status() function was +renamed with the following commit: + +git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=894020fdd88c1e9a74c60b67c0f19f1c7696ba2f + +This simply updates the function call with the proper name (pci_aer_clear_nonfatal_status()). + +--- +diff --git a/quickassist/qat/drivers/crypto/qat/qat_common/adf_aer.c b/quickassist/qat/drivers/crypto/qat/qat_common/adf_aer.c +index a6ce6df..545bb79 100644 +--- a/quickassist/qat/drivers/crypto/qat/qat_common/adf_aer.c ++++ b/quickassist/qat/drivers/crypto/qat/qat_common/adf_aer.c +@@ -304,7 +304,7 @@ static pci_ers_result_t adf_slot_reset(struct pci_dev *pdev) + pr_err("QAT: Can't find acceleration device\n"); + return PCI_ERS_RESULT_DISCONNECT; + } +- pci_cleanup_aer_uncorrect_error_status(pdev); ++ pci_aer_clear_nonfatal_status(pdev); + if (adf_dev_aer_schedule_reset(accel_dev, ADF_DEV_RESET_SYNC)) + return PCI_ERS_RESULT_DISCONNECT; diff --git a/contrib/intel_qat/patch/0001-timespec.diff b/contrib/intel_qat/patch/0001-timespec.diff new file mode 100644 index 000000000000..04fb053e1f8c --- /dev/null +++ b/contrib/intel_qat/patch/0001-timespec.diff @@ -0,0 +1,35 @@ +This patch attempts to expose timespec and getnstimeofday which were +explicitly hidden in the 5.6 kernel with the introduction of the +following commits: + +git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c766d1472c70d25ad475cf56042af1652e792b23 +git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=412c53a680a97cb1ae2c0ab60230e193bee86387 + +Code received from users@dpdk.org, issue tracked under QATE-59888. + +--- +diff --git a/quickassist/lookaside/access_layer/src/sample_code/performance/framework/linux/kernel_space/cpa_sample_code_utils.c b/quickassist/lookaside/access_layer/src/sample_code/performance/framework/linux/kernel_space/cpa_sample_code_utils.c +index 4639834..523e376 100644 +--- a/quickassist/lookaside/access_layer/src/sample_code/performance/framework/linux/kernel_space/cpa_sample_code_utils.c ++++ b/quickassist/lookaside/access_layer/src/sample_code/performance/framework/linux/kernel_space/cpa_sample_code_utils.c +@@ -107,6 +107,8 @@ atomic_t arrived; + extern struct device perf_device; + #endif + ++#define timespec timespec64 ++#define getnstimeofday ktime_get_real_ts64 + + /* Define a number for timeout */ + #define SAMPLE_CODE_MAX_LONG (0x7FFFFFFF) +diff --git a/quickassist/qat/compat/qat_compat.h b/quickassist/qat/compat/qat_compat.h +index 2a02eaf..3515092 100644 +--- a/quickassist/qat/compat/qat_compat.h ++++ b/quickassist/qat/compat/qat_compat.h +@@ -466,4 +466,7 @@ static inline void pci_ignore_hotplug(struct pci_dev *dev) + #if (RHEL_RELEASE_CODE && RHEL_RELEASE_VERSION(7, 3) <= RHEL_RELEASE_CODE) + #define QAT_KPT_CAP_DISCOVERY + #endif ++ ++#define timespec timespec64 ++#define getnstimeofday ktime_get_real_ts64 + #endif /* _QAT_COMPAT_H_ */ diff --git a/contrib/intel_qat/patch/LICENSE b/contrib/intel_qat/patch/LICENSE new file mode 100644 index 000000000000..8e12726c0a9a --- /dev/null +++ b/contrib/intel_qat/patch/LICENSE @@ -0,0 +1,30 @@ +BSD LICENSE + +Copyright (c) Intel Corporation. +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in + the documentation and/or other materials provided with the + distribution. + * Neither the name of Intel Corporation nor the names of its + contributors may be used to endorse or promote products derived + from this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/contrib/intel_qat/readme.md b/contrib/intel_qat/readme.md new file mode 100644 index 000000000000..7e45d395bb80 --- /dev/null +++ b/contrib/intel_qat/readme.md @@ -0,0 +1,27 @@ +# Intel_QAT easy install script + +This contrib contains community compatibility patches to get Intel QAT working on the following kernel versions: +- 5.6 +- 5.7 +- 5.8 + +These patches are based on the following Intel QAT version: +[1.7.l.4.10.0-00014](https://01.org/sites/default/files/downloads/qat1.7.l.4.10.0-00014.tar.gz) + +When using QAT with above kernels versions, the following patches needs to be applied using: +patch -p1 < _$PATCH_ +_Where $PATCH refers to the path of the patch in question_ + +### 5.6 +/patch/0001-timespec.diff + +### 5.7 +/patch/0001-pci_aer.diff + +### 5.8 +/patch/0001-cryptohash.diff + + +_Patches are supplied by [Storage Performance Development Kit (SPDK)](https://github.com/spdk/spdk)_ + + diff --git a/include/os/freebsd/spl/sys/kstat.h b/include/os/freebsd/spl/sys/kstat.h index 74c3da8ec376..5ceb88b297d9 100644 --- a/include/os/freebsd/spl/sys/kstat.h +++ b/include/os/freebsd/spl/sys/kstat.h @@ -83,6 +83,14 @@ typedef struct kstat_s kstat_t; typedef int kid_t; /* unique kstat id */ typedef int kstat_update_t(struct kstat_s *, int); /* dynamic update cb */ +struct seq_file { + char *sf_buf; + size_t sf_size; +}; + +void seq_printf(struct seq_file *m, const char *fmt, ...); + + typedef struct kstat_module { char ksm_name[KSTAT_STRLEN+1]; /* module name */ struct list_head ksm_module_list; /* module linkage */ @@ -92,6 +100,7 @@ typedef struct kstat_module { typedef struct kstat_raw_ops { int (*headers)(char *buf, size_t size); + int (*seq_headers)(struct seq_file *); int (*data)(char *buf, size_t size, void *data); void *(*addr)(kstat_t *ksp, loff_t index); } kstat_raw_ops_t; @@ -112,6 +121,7 @@ struct kstat_s { size_t ks_data_size; /* size of kstat data section */ kstat_update_t *ks_update; /* dynamic updates */ void *ks_private; /* private data */ + void *ks_private1; /* private data */ kmutex_t ks_private_lock; /* kstat private data lock */ kmutex_t *ks_lock; /* kstat data lock */ struct list_head ks_list; /* kstat linkage */ @@ -185,6 +195,12 @@ extern void __kstat_set_raw_ops(kstat_t *ksp, int (*data)(char *buf, size_t size, void *data), void* (*addr)(kstat_t *ksp, loff_t index)); +extern void __kstat_set_seq_raw_ops(kstat_t *ksp, + int (*headers)(struct seq_file *), + int (*data)(char *buf, size_t size, void *data), + void* (*addr)(kstat_t *ksp, loff_t index)); + + extern kstat_t *__kstat_create(const char *ks_module, int ks_instance, const char *ks_name, const char *ks_class, uchar_t ks_type, uint_t ks_ndata, uchar_t ks_flags); @@ -196,6 +212,8 @@ extern void kstat_waitq_exit(kstat_io_t *); extern void kstat_runq_enter(kstat_io_t *); extern void kstat_runq_exit(kstat_io_t *); +#define kstat_set_seq_raw_ops(k, h, d, a) \ + __kstat_set_seq_raw_ops(k, h, d, a) #define kstat_set_raw_ops(k, h, d, a) \ __kstat_set_raw_ops(k, h, d, a) #define kstat_create(m, i, n, c, t, s, f) \ diff --git a/include/os/freebsd/spl/sys/procfs_list.h b/include/os/freebsd/spl/sys/procfs_list.h index 5d623c369c4c..da13f0387cb5 100644 --- a/include/os/freebsd/spl/sys/procfs_list.h +++ b/include/os/freebsd/spl/sys/procfs_list.h @@ -33,16 +33,18 @@ * procfs list manipulation */ -struct seq_file { }; -void seq_printf(struct seq_file *m, const char *fmt, ...); - -typedef struct procfs_list { +typedef struct procfs_list procfs_list_t; +struct procfs_list { void *pl_private; + void *pl_next_data; kmutex_t pl_lock; list_t pl_list; uint64_t pl_next_id; + int (*pl_show)(struct seq_file *f, void *p); + int (*pl_show_header)(struct seq_file *f); + int (*pl_clear)(procfs_list_t *procfs_list); size_t pl_node_offset; -} procfs_list_t; +}; typedef struct procfs_list_node { list_node_t pln_link; @@ -50,6 +52,7 @@ typedef struct procfs_list_node { } procfs_list_node_t; void procfs_list_install(const char *module, + const char *submodule, const char *name, mode_t mode, procfs_list_t *procfs_list, diff --git a/include/os/freebsd/spl/sys/simd_x86.h b/include/os/freebsd/spl/sys/simd_x86.h index a35e205d5a3b..63d6017b79e0 100644 --- a/include/os/freebsd/spl/sys/simd_x86.h +++ b/include/os/freebsd/spl/sys/simd_x86.h @@ -29,6 +29,7 @@ #include <sys/cdefs.h> #include <sys/types.h> #include <sys/systm.h> +#include <sys/proc.h> #ifdef __i386__ #include <x86/fpu.h> #else @@ -42,16 +43,15 @@ #define kfpu_allowed() 1 #define kfpu_initialize(tsk) do {} while (0) -#define kfpu_begin() { \ - critical_enter(); \ - fpu_kern_enter(curthread, NULL, FPU_KERN_NOCTX); \ +#define kfpu_begin() { \ + if (__predict_false(!is_fpu_kern_thread(0))) \ + fpu_kern_enter(curthread, NULL, FPU_KERN_NOCTX);\ } -#define kfpu_end() \ - { \ - fpu_kern_leave(curthread, NULL); \ - critical_exit(); \ - } +#define kfpu_end() { \ + if (__predict_false(curpcb->pcb_flags & PCB_FPUNOSAVE)) \ + fpu_kern_leave(curthread, NULL); \ +} /* * Check if OS supports AVX and AVX2 by checking XCR0 diff --git a/include/os/linux/spl/sys/procfs_list.h b/include/os/linux/spl/sys/procfs_list.h index eb1519c0ad63..9bb437f55cf7 100644 --- a/include/os/linux/spl/sys/procfs_list.h +++ b/include/os/linux/spl/sys/procfs_list.h @@ -57,6 +57,7 @@ typedef struct procfs_list_node { } procfs_list_node_t; void procfs_list_install(const char *module, + const char *submodule, const char *name, mode_t mode, procfs_list_t *procfs_list, diff --git a/include/sys/frame.h b/include/sys/frame.h index 2865dbb57dc3..b6bbaa79b2f8 100644 --- a/include/sys/frame.h +++ b/include/sys/frame.h @@ -23,7 +23,8 @@ extern "C" { #endif -#if defined(__KERNEL__) && defined(HAVE_STACK_FRAME_NON_STANDARD) +#if defined(__KERNEL__) && defined(HAVE_KERNEL_OBJTOOL) && \ + defined(HAVE_STACK_FRAME_NON_STANDARD) #include <linux/frame.h> #else #define STACK_FRAME_NON_STANDARD(func) diff --git a/include/sys/lua/luaconf.h b/include/sys/lua/luaconf.h index ce99f339fe34..83202d71c27b 100644 --- a/include/sys/lua/luaconf.h +++ b/include/sys/lua/luaconf.h @@ -368,11 +368,7 @@ extern int lcompat_hashnum(int64_t); @@ LUAL_BUFFERSIZE is the buffer size used by the lauxlib buffer system. ** CHANGE it if it uses too much C-stack space. */ -#ifdef __linux__ #define LUAL_BUFFERSIZE 512 -#else -#define LUAL_BUFFERSIZE 1024 -#endif /* diff --git a/include/sys/zfs_context.h b/include/sys/zfs_context.h index e33f52c176a8..d9f5ed580ef1 100644 --- a/include/sys/zfs_context.h +++ b/include/sys/zfs_context.h @@ -386,6 +386,7 @@ typedef struct procfs_list_node { } procfs_list_node_t; void procfs_list_install(const char *module, + const char *submodule, const char *name, mode_t mode, procfs_list_t *procfs_list, diff --git a/include/sys/zstd/zstd.h b/include/sys/zstd/zstd.h index f965df31988c..e42e44c236ab 100644 --- a/include/sys/zstd/zstd.h +++ b/include/sys/zstd/zstd.h @@ -90,6 +90,7 @@ int zfs_zstd_decompress_level(void *s_start, void *d_start, size_t s_len, size_t d_len, uint8_t *level); int zfs_zstd_decompress(void *s_start, void *d_start, size_t s_len, size_t d_len, int n); +void zfs_zstd_cache_reap_now(void); #ifdef __cplusplus } diff --git a/lib/libshare/os/freebsd/nfs.c b/lib/libshare/os/freebsd/nfs.c index 65f3b11bf9b3..5951b9eafa2b 100644 --- a/lib/libshare/os/freebsd/nfs.c +++ b/lib/libshare/os/freebsd/nfs.c @@ -228,21 +228,33 @@ nfs_copy_entries(char *filename, const char *mountpoint) int error = SA_OK; char *line; - /* - * If the file doesn't exist then there is nothing more - * we need to do. - */ FILE *oldfp = fopen(ZFS_EXPORTS_FILE, "r"); - if (oldfp == NULL) - return (SA_OK); - FILE *newfp = fopen(filename, "w+"); + if (newfp == NULL) { + fprintf(stderr, "failed to open %s file: %s", filename, + strerror(errno)); + fclose(oldfp); + return (SA_SYSTEM_ERR); + } fputs(FILE_HEADER, newfp); - while ((line = zgetline(oldfp, mountpoint)) != NULL) - fprintf(newfp, "%s\n", line); - if (ferror(oldfp) != 0) { - error = ferror(oldfp); + + /* + * The ZFS_EXPORTS_FILE may not exist yet. If that's the + * case then just write out the new file. + */ + if (oldfp != NULL) { + while ((line = zgetline(oldfp, mountpoint)) != NULL) + fprintf(newfp, "%s\n", line); + if (ferror(oldfp) != 0) { + error = ferror(oldfp); + } + if (fclose(oldfp) != 0) { + fprintf(stderr, "Unable to close file %s: %s\n", + filename, strerror(errno)); + error = error != 0 ? error : SA_SYSTEM_ERR; + } } + if (error == 0 && ferror(newfp) != 0) { error = ferror(newfp); } @@ -252,8 +264,6 @@ nfs_copy_entries(char *filename, const char *mountpoint) filename, strerror(errno)); error = error != 0 ? error : SA_SYSTEM_ERR; } - fclose(oldfp); - return (error); } diff --git a/lib/libshare/os/linux/nfs.c b/lib/libshare/os/linux/nfs.c index a6a9b33d7655..1efa321b7bc6 100644 --- a/lib/libshare/os/linux/nfs.c +++ b/lib/libshare/os/linux/nfs.c @@ -393,6 +393,14 @@ static char * nfs_init_tmpfile(void) { char *tmpfile = NULL; + struct stat sb; + + if (stat(ZFS_EXPORTS_DIR, &sb) < 0 && + mkdir(ZFS_EXPORTS_DIR, 0755) < 0) { + fprintf(stderr, "failed to create %s: %s\n", + ZFS_EXPORTS_DIR, strerror(errno)); + return (NULL); + } if (asprintf(&tmpfile, "%s%s", ZFS_EXPORTS_FILE, ".XXXXXXXX") == -1) { fprintf(stderr, "Unable to allocate temporary file\n"); @@ -481,36 +489,49 @@ nfs_copy_entries(char *filename, const char *mountpoint) size_t buflen = 0; int error = SA_OK; - /* - * If the file doesn't exist then there is nothing more - * we need to do. - */ FILE *oldfp = fopen(ZFS_EXPORTS_FILE, "r"); - if (oldfp == NULL) - return (SA_OK); - FILE *newfp = fopen(filename, "w+"); + if (newfp == NULL) { + fprintf(stderr, "failed to open %s file: %s", filename, + strerror(errno)); + fclose(oldfp); + return (SA_SYSTEM_ERR); + } fputs(FILE_HEADER, newfp); - while ((getline(&buf, &buflen, oldfp)) != -1) { - char *space = NULL; - if (buf[0] == '\n' || buf[0] == '#') - continue; - - if ((space = strchr(buf, ' ')) != NULL) { - int mountpoint_len = strlen(mountpoint); + /* + * The ZFS_EXPORTS_FILE may not exist yet. If that's the + * case then just write out the new file. + */ + if (oldfp != NULL) { + while (getline(&buf, &buflen, oldfp) != -1) { + char *space = NULL; - if (space - buf == mountpoint_len && - strncmp(mountpoint, buf, mountpoint_len) == 0) { + if (buf[0] == '\n' || buf[0] == '#') continue; + + if ((space = strchr(buf, ' ')) != NULL) { + int mountpoint_len = strlen(mountpoint); + + if (space - buf == mountpoint_len && + strncmp(mountpoint, buf, + mountpoint_len) == 0) { + continue; + } } + fputs(buf, newfp); } - fputs(buf, newfp); - } - if (oldfp != NULL && ferror(oldfp) != 0) { - error = ferror(oldfp); + if (ferror(oldfp) != 0) { + error = ferror(oldfp); + } + if (fclose(oldfp) != 0) { + fprintf(stderr, "Unable to close file %s: %s\n", + filename, strerror(errno)); + error = error != 0 ? error : SA_SYSTEM_ERR; + } } + if (error == 0 && ferror(newfp) != 0) { error = ferror(newfp); } @@ -521,8 +542,6 @@ nfs_copy_entries(char *filename, const char *mountpoint) filename, strerror(errno)); error = error != 0 ? error : SA_SYSTEM_ERR; } - fclose(oldfp); - return (error); } @@ -701,13 +720,5 @@ static const sa_share_ops_t nfs_shareops = { void libshare_nfs_init(void) { - struct stat sb; - nfs_fstype = register_fstype("nfs", &nfs_shareops); - - if (stat(ZFS_EXPORTS_DIR, &sb) < 0 && - mkdir(ZFS_EXPORTS_DIR, 0755) < 0) { - fprintf(stderr, "failed to create %s: %s\n", - ZFS_EXPORTS_DIR, strerror(errno)); - } } diff --git a/lib/libzpool/kernel.c b/lib/libzpool/kernel.c index 145b21d40f99..ca357899367a 100644 --- a/lib/libzpool/kernel.c +++ b/lib/libzpool/kernel.c @@ -444,6 +444,7 @@ seq_printf(struct seq_file *m, const char *fmt, ...) void procfs_list_install(const char *module, + const char *submodule, const char *name, mode_t mode, procfs_list_t *procfs_list, diff --git a/man/man8/zfs-userspace.8 b/man/man8/zfs-userspace.8 index a8477d16a1ae..9c103bf48c04 100644 --- a/man/man8/zfs-userspace.8 +++ b/man/man8/zfs-userspace.8 @@ -44,7 +44,7 @@ .Oo Fl s Ar field Oc Ns ... .Oo Fl S Ar field Oc Ns ... .Oo Fl t Ar type Ns Oo , Ns Ar type Oc Ns ... Oc -.Ar filesystem Ns | Ns Ar snapshot +.Ar filesystem Ns | Ns Ar snapshot Ns | Ns Ar path .Nm .Cm groupspace .Op Fl Hinp @@ -52,14 +52,14 @@ .Oo Fl s Ar field Oc Ns ... .Oo Fl S Ar field Oc Ns ... .Oo Fl t Ar type Ns Oo , Ns Ar type Oc Ns ... Oc -.Ar filesystem Ns | Ns Ar snapshot +.Ar filesystem Ns | Ns Ar snapshot Ns | Ns Ar path .Nm .Cm projectspace .Op Fl Hp .Oo Fl o Ar field Ns Oo , Ns Ar field Oc Ns ... Oc .Oo Fl s Ar field Oc Ns ... .Oo Fl S Ar field Oc Ns ... -.Ar filesystem Ns | Ns Ar snapshot +.Ar filesystem Ns | Ns Ar snapshot Ns | Ns Ar path .Sh DESCRIPTION .Bl -tag -width "" .It Xo @@ -70,10 +70,11 @@ .Oo Fl s Ar field Oc Ns ... .Oo Fl S Ar field Oc Ns ... .Oo Fl t Ar type Ns Oo , Ns Ar type Oc Ns ... Oc -.Ar filesystem Ns | Ns Ar snapshot +.Ar filesystem Ns | Ns Ar snapshot Ns | Ns Ar path .Xc -Displays space consumed by, and quotas on, each user in the specified filesystem -or snapshot. +Displays space consumed by, and quotas on, each user in the specified filesystem, +snapshot, or path. +If a path is given, the filesystem that contains that path will be used. This corresponds to the .Sy userused@ Ns Em user , .Sy userobjused@ Ns Em user , @@ -167,7 +168,7 @@ except that the default types to display are .Oo Fl o Ar field Ns Oo , Ns Ar field Oc Ns ... Oc .Oo Fl s Ar field Oc Ns ... .Oo Fl S Ar field Oc Ns ... -.Ar filesystem Ns | Ns Ar snapshot +.Ar filesystem Ns | Ns Ar snapshot Ns | Ns Ar path .Xc Displays space consumed by, and quotas on, each project in the specified filesystem or snapshot. This subcommand is identical to diff --git a/man/man8/zpool-remove.8 b/man/man8/zpool-remove.8 index 055f852fc7cb..cd394f107e8d 100644 --- a/man/man8/zpool-remove.8 +++ b/man/man8/zpool-remove.8 @@ -55,6 +55,8 @@ This command supports removing hot spare, cache, log, and both mirrored and non-redundant primary top-level vdevs, including dedup and special vdevs. When the primary pool storage includes a top-level raidz vdev only hot spare, cache, and log devices can be removed. +Note that keys for all encrypted datasets must be loaded for top-level vdevs +to be removed. .sp Removing a top-level vdev reduces the total amount of space in the storage pool. The specified device will be evacuated by copying all allocated space from it to diff --git a/module/lua/llimits.h b/module/lua/llimits.h index 25466f14edca..177092fbc228 100644 --- a/module/lua/llimits.h +++ b/module/lua/llimits.h @@ -126,16 +126,7 @@ typedef LUAI_UACNUMBER l_uacNumber; * Minimum amount of available stack space (in bytes) to make a C call. With * gsub() recursion, the stack space between each luaD_call() is 1256 bytes. */ -#if defined(__FreeBSD__) -/* - * FreeBSD needs a few extra bytes in unoptimized debug builds to avoid a - * double-fault handling the error when the max call depth is exceeded just - * before the C stack runs out. 64 bytes seems to do the trick. - */ -#define LUAI_MINCSTACK 4160 -#else #define LUAI_MINCSTACK 4096 -#endif /* ** maximum number of upvalues in a closure (both C and Lua). (Value diff --git a/module/os/freebsd/spl/spl_kstat.c b/module/os/freebsd/spl/spl_kstat.c index df2da2d602b6..4cc77e20a4eb 100644 --- a/module/os/freebsd/spl/spl_kstat.c +++ b/module/os/freebsd/spl/spl_kstat.c @@ -55,6 +55,17 @@ __kstat_set_raw_ops(kstat_t *ksp, ksp->ks_raw_ops.addr = addr; } +void +__kstat_set_seq_raw_ops(kstat_t *ksp, + int (*headers)(struct seq_file *f), + int (*data)(char *buf, size_t size, void *data), + void *(*addr)(kstat_t *ksp, loff_t index)) +{ + ksp->ks_raw_ops.seq_headers = headers; + ksp->ks_raw_ops.data = data; + ksp->ks_raw_ops.addr = addr; +} + static int kstat_default_update(kstat_t *ksp, int rw) { @@ -160,7 +171,7 @@ kstat_sysctl_raw(SYSCTL_HANDLER_ARGS) void *data; kstat_t *ksp = arg1; void *(*addr_op)(kstat_t *ksp, loff_t index); - int n, rc = 0; + int n, has_header, rc = 0; sb = sbuf_new_auto(); if (sb == NULL) @@ -180,14 +191,25 @@ kstat_sysctl_raw(SYSCTL_HANDLER_ARGS) ksp->ks_raw_buf = malloc(PAGE_SIZE, M_TEMP, M_WAITOK); n = 0; + has_header = (ksp->ks_raw_ops.headers || + ksp->ks_raw_ops.seq_headers); + restart_headers: if (ksp->ks_raw_ops.headers) { rc = ksp->ks_raw_ops.headers( ksp->ks_raw_buf, ksp->ks_raw_bufsize); + } else if (ksp->ks_raw_ops.seq_headers) { + struct seq_file f; + + f.sf_buf = ksp->ks_raw_buf; + f.sf_size = ksp->ks_raw_bufsize; + rc = ksp->ks_raw_ops.seq_headers(&f); + } + if (has_header) { if (rc == ENOMEM && !kstat_resize_raw(ksp)) goto restart_headers; if (rc == 0) - sbuf_printf(sb, "%s", ksp->ks_raw_buf); + sbuf_printf(sb, "\n%s", ksp->ks_raw_buf); } while ((data = addr_op(ksp, n)) != NULL) { @@ -220,16 +242,21 @@ kstat_t * __kstat_create(const char *module, int instance, const char *name, const char *class, uchar_t ks_type, uint_t ks_ndata, uchar_t flags) { + char buf[KSTAT_STRLEN]; struct sysctl_oid *root; kstat_t *ksp; + char *pool; KASSERT(instance == 0, ("instance=%d", instance)); if ((ks_type == KSTAT_TYPE_INTR) || (ks_type == KSTAT_TYPE_IO)) ASSERT(ks_ndata == 1); + if (class == NULL) + class = "misc"; + /* - * Allocate the main structure. We don't need to copy module/class/name - * stuff in here, because it is only used for sysctl node creation + * Allocate the main structure. We don't need to keep a copy of + * module in here, because it is only used for sysctl node creation * done in this function. */ ksp = malloc(sizeof (*ksp), M_KSTAT, M_WAITOK|M_ZERO); @@ -237,8 +264,8 @@ __kstat_create(const char *module, int instance, const char *name, ksp->ks_crtime = gethrtime(); ksp->ks_snaptime = ksp->ks_crtime; ksp->ks_instance = instance; - strncpy(ksp->ks_name, name, KSTAT_STRLEN); - strncpy(ksp->ks_class, class, KSTAT_STRLEN); + (void) strlcpy(ksp->ks_name, name, KSTAT_STRLEN); + (void) strlcpy(ksp->ks_class, class, KSTAT_STRLEN); ksp->ks_type = ks_type; ksp->ks_flags = flags; ksp->ks_update = kstat_default_update; @@ -247,28 +274,28 @@ __kstat_create(const char *module, int instance, const char *name, ksp->ks_lock = &ksp->ks_private_lock; switch (ksp->ks_type) { - case KSTAT_TYPE_RAW: - ksp->ks_ndata = 1; - ksp->ks_data_size = ks_ndata; - break; - case KSTAT_TYPE_NAMED: - ksp->ks_ndata = ks_ndata; - ksp->ks_data_size = ks_ndata * sizeof (kstat_named_t); - break; - case KSTAT_TYPE_INTR: - ksp->ks_ndata = ks_ndata; - ksp->ks_data_size = ks_ndata * sizeof (kstat_intr_t); - break; - case KSTAT_TYPE_IO: - ksp->ks_ndata = ks_ndata; - ksp->ks_data_size = ks_ndata * sizeof (kstat_io_t); - break; - case KSTAT_TYPE_TIMER: - ksp->ks_ndata = ks_ndata; - ksp->ks_data_size = ks_ndata * sizeof (kstat_timer_t); - break; - default: - panic("Undefined kstat type %d\n", ksp->ks_type); + case KSTAT_TYPE_RAW: + ksp->ks_ndata = 1; + ksp->ks_data_size = ks_ndata; + break; + case KSTAT_TYPE_NAMED: + ksp->ks_ndata = ks_ndata; + ksp->ks_data_size = ks_ndata * sizeof (kstat_named_t); + break; + case KSTAT_TYPE_INTR: + ksp->ks_ndata = ks_ndata; + ksp->ks_data_size = ks_ndata * sizeof (kstat_intr_t); + break; + case KSTAT_TYPE_IO: + ksp->ks_ndata = ks_ndata; + ksp->ks_data_size = ks_ndata * sizeof (kstat_io_t); + break; + case KSTAT_TYPE_TIMER: + ksp->ks_ndata = ks_ndata; + ksp->ks_data_size = ks_ndata * sizeof (kstat_timer_t); + break; + default: + panic("Undefined kstat type %d\n", ksp->ks_type); } if (ksp->ks_flags & KSTAT_FLAG_VIRTUAL) { @@ -280,10 +307,22 @@ __kstat_create(const char *module, int instance, const char *name, ksp = NULL; } } + + /* + * Some kstats use a module name like "zfs/poolname" to distinguish a + * set of kstats belonging to a specific pool. Split on '/' to add an + * extra node for the pool name if needed. + */ + (void) strlcpy(buf, module, KSTAT_STRLEN); + module = buf; + pool = strchr(module, '/'); + if (pool != NULL) + *pool++ = '\0'; + /* * Create sysctl tree for those statistics: * - * kstat.<module>.<class>.<name>. + * kstat.<module>[.<pool>].<class>.<name> */ sysctl_ctx_init(&ksp->ks_sysctl_ctx); root = SYSCTL_ADD_NODE(&ksp->ks_sysctl_ctx, @@ -295,11 +334,26 @@ __kstat_create(const char *module, int instance, const char *name, free(ksp, M_KSTAT); return (NULL); } + if (pool != NULL) { + root = SYSCTL_ADD_NODE(&ksp->ks_sysctl_ctx, + SYSCTL_CHILDREN(root), OID_AUTO, pool, CTLFLAG_RW, 0, ""); + if (root == NULL) { + printf("%s: Cannot create kstat.%s.%s tree!\n", + __func__, module, pool); + sysctl_ctx_free(&ksp->ks_sysctl_ctx); + free(ksp, M_KSTAT); + return (NULL); + } + } root = SYSCTL_ADD_NODE(&ksp->ks_sysctl_ctx, SYSCTL_CHILDREN(root), OID_AUTO, class, CTLFLAG_RW, 0, ""); if (root == NULL) { - printf("%s: Cannot create kstat.%s.%s tree!\n", __func__, - module, class); + if (pool != NULL) + printf("%s: Cannot create kstat.%s.%s.%s tree!\n", + __func__, module, pool, class); + else + printf("%s: Cannot create kstat.%s.%s tree!\n", + __func__, module, class); sysctl_ctx_free(&ksp->ks_sysctl_ctx); free(ksp, M_KSTAT); return (NULL); @@ -309,8 +363,13 @@ __kstat_create(const char *module, int instance, const char *name, SYSCTL_CHILDREN(root), OID_AUTO, name, CTLFLAG_RW, 0, ""); if (root == NULL) { - printf("%s: Cannot create kstat.%s.%s.%s tree!\n", - __func__, module, class, name); + if (pool != NULL) + printf("%s: Cannot create kstat.%s.%s.%s.%s " + "tree!\n", __func__, module, pool, class, + name); + else + printf("%s: Cannot create kstat.%s.%s.%s " + "tree!\n", __func__, module, class, name); sysctl_ctx_free(&ksp->ks_sysctl_ctx); free(ksp, M_KSTAT); return (NULL); @@ -342,64 +401,62 @@ kstat_install_named(kstat_t *ksp) namelast = ksent->name; } switch (typelast) { - case KSTAT_DATA_CHAR: - /* Not Implemented */ - break; - case KSTAT_DATA_INT32: - SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, - SYSCTL_CHILDREN(ksp->ks_sysctl_root), - OID_AUTO, namelast, - CTLTYPE_S32 | CTLFLAG_RD, ksp, i, - kstat_sysctl, "I", namelast); - break; - case KSTAT_DATA_UINT32: - SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, - SYSCTL_CHILDREN(ksp->ks_sysctl_root), - OID_AUTO, namelast, - CTLTYPE_U32 | CTLFLAG_RD, ksp, i, - kstat_sysctl, "IU", namelast); - break; - case KSTAT_DATA_INT64: - SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, - SYSCTL_CHILDREN(ksp->ks_sysctl_root), - OID_AUTO, namelast, - CTLTYPE_S64 | CTLFLAG_RD, ksp, i, - kstat_sysctl, "Q", namelast); - break; - case KSTAT_DATA_UINT64: - SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, - SYSCTL_CHILDREN(ksp->ks_sysctl_root), - OID_AUTO, namelast, - CTLTYPE_U64 | CTLFLAG_RD, ksp, i, - kstat_sysctl, "QU", namelast); - break; - case KSTAT_DATA_LONG: - SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, - SYSCTL_CHILDREN(ksp->ks_sysctl_root), - OID_AUTO, namelast, - CTLTYPE_LONG | CTLFLAG_RD, ksp, i, - kstat_sysctl, "L", namelast); - break; - case KSTAT_DATA_ULONG: - SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, - SYSCTL_CHILDREN(ksp->ks_sysctl_root), - OID_AUTO, namelast, - CTLTYPE_ULONG | CTLFLAG_RD, ksp, i, - kstat_sysctl, "LU", namelast); - break; - case KSTAT_DATA_STRING: - SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, - SYSCTL_CHILDREN(ksp->ks_sysctl_root), - OID_AUTO, namelast, - CTLTYPE_STRING | CTLFLAG_RD, ksp, i, - kstat_sysctl_string, "A", namelast); - break; - default: - panic("unsupported type: %d", typelast); + case KSTAT_DATA_CHAR: + /* Not Implemented */ + break; + case KSTAT_DATA_INT32: + SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, + SYSCTL_CHILDREN(ksp->ks_sysctl_root), + OID_AUTO, namelast, + CTLTYPE_S32 | CTLFLAG_RD | CTLFLAG_MPSAFE, + ksp, i, kstat_sysctl, "I", namelast); + break; + case KSTAT_DATA_UINT32: + SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, + SYSCTL_CHILDREN(ksp->ks_sysctl_root), + OID_AUTO, namelast, + CTLTYPE_U32 | CTLFLAG_RD | CTLFLAG_MPSAFE, + ksp, i, kstat_sysctl, "IU", namelast); + break; + case KSTAT_DATA_INT64: + SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, + SYSCTL_CHILDREN(ksp->ks_sysctl_root), + OID_AUTO, namelast, + CTLTYPE_S64 | CTLFLAG_RD | CTLFLAG_MPSAFE, + ksp, i, kstat_sysctl, "Q", namelast); + break; + case KSTAT_DATA_UINT64: + SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, + SYSCTL_CHILDREN(ksp->ks_sysctl_root), + OID_AUTO, namelast, + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_MPSAFE, + ksp, i, kstat_sysctl, "QU", namelast); + break; + case KSTAT_DATA_LONG: + SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, + SYSCTL_CHILDREN(ksp->ks_sysctl_root), + OID_AUTO, namelast, + CTLTYPE_LONG | CTLFLAG_RD | CTLFLAG_MPSAFE, + ksp, i, kstat_sysctl, "L", namelast); + break; + case KSTAT_DATA_ULONG: + SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, + SYSCTL_CHILDREN(ksp->ks_sysctl_root), + OID_AUTO, namelast, + CTLTYPE_ULONG | CTLFLAG_RD | CTLFLAG_MPSAFE, + ksp, i, kstat_sysctl, "LU", namelast); + break; + case KSTAT_DATA_STRING: + SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, + SYSCTL_CHILDREN(ksp->ks_sysctl_root), + OID_AUTO, namelast, + CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, + ksp, i, kstat_sysctl_string, "A", namelast); + break; + default: + panic("unsupported type: %d", typelast); } - } - } void @@ -411,39 +468,37 @@ kstat_install(kstat_t *ksp) VERIFY(ksp->ks_type == KSTAT_TYPE_RAW); switch (ksp->ks_type) { - case KSTAT_TYPE_NAMED: - return (kstat_install_named(ksp)); - break; - case KSTAT_TYPE_RAW: - if (ksp->ks_raw_ops.data) { - root = SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, - SYSCTL_CHILDREN(ksp->ks_sysctl_root), - OID_AUTO, ksp->ks_name, - CTLTYPE_STRING | CTLFLAG_RD, ksp, 0, - kstat_sysctl_raw, "A", ksp->ks_name); - } else { - root = SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, - SYSCTL_CHILDREN(ksp->ks_sysctl_root), - OID_AUTO, ksp->ks_name, - CTLTYPE_OPAQUE | CTLFLAG_RD, ksp, 0, - kstat_sysctl_raw, "", ksp->ks_name); - } - VERIFY(root != NULL); - break; - case KSTAT_TYPE_IO: + case KSTAT_TYPE_NAMED: + return (kstat_install_named(ksp)); + case KSTAT_TYPE_RAW: + if (ksp->ks_raw_ops.data) { root = SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, SYSCTL_CHILDREN(ksp->ks_sysctl_root), OID_AUTO, ksp->ks_name, - CTLTYPE_STRING | CTLFLAG_RD, ksp, 0, - kstat_sysctl_io, "A", ksp->ks_name); - break; - case KSTAT_TYPE_TIMER: - case KSTAT_TYPE_INTR: - default: - panic("unsupported kstat type %d\n", ksp->ks_type); + CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, + ksp, 0, kstat_sysctl_raw, "A", ksp->ks_name); + } else { + root = SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, + SYSCTL_CHILDREN(ksp->ks_sysctl_root), + OID_AUTO, ksp->ks_name, + CTLTYPE_OPAQUE | CTLFLAG_RD | CTLFLAG_MPSAFE, + ksp, 0, kstat_sysctl_raw, "", ksp->ks_name); + } + break; + case KSTAT_TYPE_IO: + root = SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx, + SYSCTL_CHILDREN(ksp->ks_sysctl_root), + OID_AUTO, ksp->ks_name, + CTLTYPE_STRING | CTLFLAG_RD | CTLFLAG_MPSAFE, + ksp, 0, kstat_sysctl_io, "A", ksp->ks_name); + break; + case KSTAT_TYPE_TIMER: + case KSTAT_TYPE_INTR: + default: + panic("unsupported kstat type %d\n", ksp->ks_type); } + VERIFY(root != NULL); ksp->ks_sysctl_root = root; - } void diff --git a/module/os/freebsd/spl/spl_procfs_list.c b/module/os/freebsd/spl/spl_procfs_list.c index 7b4ae9d0e357..e8448ce00686 100644 --- a/module/os/freebsd/spl/spl_procfs_list.c +++ b/module/os/freebsd/spl/spl_procfs_list.c @@ -32,12 +32,74 @@ __FBSDID("$FreeBSD$"); #include <sys/mutex.h> #include <sys/procfs_list.h> +typedef struct procfs_list_iter { + procfs_list_t *pli_pl; + void *pli_elt; +} pli_t; + void -seq_printf(struct seq_file *m, const char *fmt, ...) -{} +seq_printf(struct seq_file *f, const char *fmt, ...) +{ + va_list adx; + + va_start(adx, fmt); + (void) vsnprintf(f->sf_buf, f->sf_size, fmt, adx); + va_end(adx); +} + +static int +procfs_list_update(kstat_t *ksp, int rw) +{ + procfs_list_t *pl = ksp->ks_private; + + if (rw == KSTAT_WRITE) + pl->pl_clear(pl); + + return (0); +} + +static int +procfs_list_data(char *buf, size_t size, void *data) +{ + pli_t *p; + void *elt; + procfs_list_t *pl; + struct seq_file f; + + p = data; + pl = p->pli_pl; + elt = p->pli_elt; + free(p, M_TEMP); + f.sf_buf = buf; + f.sf_size = size; + return (pl->pl_show(&f, elt)); +} + +static void * +procfs_list_addr(kstat_t *ksp, loff_t n) +{ + procfs_list_t *pl = ksp->ks_private; + void *elt = ksp->ks_private1; + pli_t *p = NULL; + + + if (n == 0) + ksp->ks_private1 = list_head(&pl->pl_list); + else if (elt) + ksp->ks_private1 = list_next(&pl->pl_list, elt); + + if (ksp->ks_private1) { + p = malloc(sizeof (*p), M_TEMP, M_WAITOK); + p->pli_pl = pl; + p->pli_elt = ksp->ks_private1; + } + + return (p); +} void procfs_list_install(const char *module, + const char *submodule, const char *name, mode_t mode, procfs_list_t *procfs_list, @@ -46,12 +108,31 @@ procfs_list_install(const char *module, int (*clear)(procfs_list_t *procfs_list), size_t procfs_list_node_off) { + kstat_t *procfs_kstat; + mutex_init(&procfs_list->pl_lock, NULL, MUTEX_DEFAULT, NULL); list_create(&procfs_list->pl_list, procfs_list_node_off + sizeof (procfs_list_node_t), procfs_list_node_off + offsetof(procfs_list_node_t, pln_link)); + procfs_list->pl_show = show; + procfs_list->pl_show_header = show_header; + procfs_list->pl_clear = clear; procfs_list->pl_next_id = 1; procfs_list->pl_node_offset = procfs_list_node_off; + + procfs_kstat = kstat_create(module, 0, name, submodule, + KSTAT_TYPE_RAW, 0, KSTAT_FLAG_VIRTUAL); + + if (procfs_kstat) { + procfs_kstat->ks_lock = &procfs_list->pl_lock; + procfs_kstat->ks_ndata = UINT32_MAX; + procfs_kstat->ks_private = procfs_list; + procfs_kstat->ks_update = procfs_list_update; + kstat_set_seq_raw_ops(procfs_kstat, show_header, + procfs_list_data, procfs_list_addr); + kstat_install(procfs_kstat); + procfs_list->pl_private = procfs_kstat; + } } void @@ -62,6 +143,7 @@ void procfs_list_destroy(procfs_list_t *procfs_list) { ASSERT(list_is_empty(&procfs_list->pl_list)); + kstat_delete(procfs_list->pl_private); list_destroy(&procfs_list->pl_list); mutex_destroy(&procfs_list->pl_lock); } diff --git a/module/os/freebsd/spl/spl_taskq.c b/module/os/freebsd/spl/spl_taskq.c index 049e889cf304..cc025de959e3 100644 --- a/module/os/freebsd/spl/spl_taskq.c +++ b/module/os/freebsd/spl/spl_taskq.c @@ -169,6 +169,10 @@ taskq_tsd_set(void *context) { taskq_t *tq = context; +#if defined(__amd64__) || defined(__i386__) || defined(__aarch64__) + if (context != NULL && tsd_get(taskq_tsd) == NULL) + fpu_kern_thread(FPU_KERN_NORMAL); +#endif tsd_set(taskq_tsd, tq); } diff --git a/module/os/freebsd/zfs/kmod_core.c b/module/os/freebsd/zfs/kmod_core.c index 4c696129857a..3a13271aac6f 100644 --- a/module/os/freebsd/zfs/kmod_core.c +++ b/module/os/freebsd/zfs/kmod_core.c @@ -86,6 +86,7 @@ __FBSDID("$FreeBSD$"); #include <sys/zio_checksum.h> #include <sys/vdev_removal.h> #include <sys/dsl_crypt.h> +#include <sys/zfs_context.h> #include <sys/zfs_ioctl_compat.h> #include <sys/zfs_ioctl_impl.h> @@ -98,7 +99,7 @@ __FBSDID("$FreeBSD$"); SYSCTL_DECL(_vfs_zfs); SYSCTL_DECL(_vfs_zfs_vdev); - +extern uint_t rrw_tsd_key; static int zfs_version_ioctl = ZFS_IOCVER_OZFS; SYSCTL_DECL(_vfs_zfs_version); SYSCTL_INT(_vfs_zfs_version, OID_AUTO, ioctl, CTLFLAG_RD, &zfs_version_ioctl, @@ -180,6 +181,7 @@ out: if (zcl) kmem_free(zcl, sizeof (zfs_cmd_legacy_t)); kmem_free(zc, sizeof (zfs_cmd_t)); + MPASS(tsd_get(rrw_tsd_key) == NULL); return (error); } diff --git a/module/os/freebsd/zfs/sysctl_os.c b/module/os/freebsd/zfs/sysctl_os.c index b3cb7e7e4374..c9b350a540ea 100644 --- a/module/os/freebsd/zfs/sysctl_os.c +++ b/module/os/freebsd/zfs/sysctl_os.c @@ -243,8 +243,9 @@ sysctl_vfs_zfs_arc_no_grow_shift(SYSCTL_HANDLER_ARGS) return (0); } -SYSCTL_PROC(_vfs_zfs, OID_AUTO, arc_no_grow_shift, CTLTYPE_U32 | CTLFLAG_RWTUN, - 0, sizeof (uint32_t), sysctl_vfs_zfs_arc_no_grow_shift, "U", +SYSCTL_PROC(_vfs_zfs, OID_AUTO, arc_no_grow_shift, + CTLTYPE_U32 | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, 0, sizeof (uint32_t), + sysctl_vfs_zfs_arc_no_grow_shift, "U", "log2(fraction of ARC which must be free to allow growing)"); int @@ -275,10 +276,12 @@ param_set_arc_int(SYSCTL_HANDLER_ARGS) return (0); } -SYSCTL_PROC(_vfs_zfs, OID_AUTO, arc_min, CTLTYPE_ULONG | CTLFLAG_RWTUN, +SYSCTL_PROC(_vfs_zfs, OID_AUTO, arc_min, + CTLTYPE_ULONG | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, &zfs_arc_min, sizeof (zfs_arc_min), param_set_arc_long, "LU", "min arc size (LEGACY)"); -SYSCTL_PROC(_vfs_zfs, OID_AUTO, arc_max, CTLTYPE_ULONG | CTLFLAG_RWTUN, +SYSCTL_PROC(_vfs_zfs, OID_AUTO, arc_max, + CTLTYPE_ULONG | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, &zfs_arc_max, sizeof (zfs_arc_max), param_set_arc_long, "LU", "max arc size (LEGACY)"); @@ -558,11 +561,13 @@ param_set_max_auto_ashift(SYSCTL_HANDLER_ARGS) return (0); } -SYSCTL_PROC(_vfs_zfs, OID_AUTO, min_auto_ashift, CTLTYPE_U64 | CTLFLAG_RWTUN, +SYSCTL_PROC(_vfs_zfs, OID_AUTO, min_auto_ashift, + CTLTYPE_U64 | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, &zfs_vdev_min_auto_ashift, sizeof (zfs_vdev_min_auto_ashift), param_set_min_auto_ashift, "QU", "Min ashift used when creating new top-level vdev. (LEGACY)"); -SYSCTL_PROC(_vfs_zfs, OID_AUTO, max_auto_ashift, CTLTYPE_U64 | CTLFLAG_RWTUN, +SYSCTL_PROC(_vfs_zfs, OID_AUTO, max_auto_ashift, + CTLTYPE_U64 | CTLFLAG_RWTUN | CTLFLAG_MPSAFE, &zfs_vdev_max_auto_ashift, sizeof (zfs_vdev_max_auto_ashift), param_set_max_auto_ashift, "QU", "Max ashift used when optimizing for logical -> physical sector size on " diff --git a/module/os/freebsd/zfs/zfs_ioctl_compat.c b/module/os/freebsd/zfs/zfs_ioctl_compat.c index 8dec8644c06e..81967bed73f9 100644 --- a/module/os/freebsd/zfs/zfs_ioctl_compat.c +++ b/module/os/freebsd/zfs/zfs_ioctl_compat.c @@ -322,8 +322,10 @@ zfs_ioctl_ozfs_to_legacy(int request) if (request > ZFS_IOC_LAST) return (-1); - if (request > ZFS_IOC_PLATFORM) + if (request > ZFS_IOC_PLATFORM) { + request -= ZFS_IOC_PLATFORM + 1; return (zfs_ioctl_ozfs_to_legacy_platform_[request]); + } if (request >= sizeof (zfs_ioctl_ozfs_to_legacy_common_)/sizeof (long)) return (-1); return (zfs_ioctl_ozfs_to_legacy_common_[request]); diff --git a/module/os/freebsd/zfs/zfs_vfsops.c b/module/os/freebsd/zfs/zfs_vfsops.c index 77812ca8d400..ec8303283414 100644 --- a/module/os/freebsd/zfs/zfs_vfsops.c +++ b/module/os/freebsd/zfs/zfs_vfsops.c @@ -1532,8 +1532,12 @@ zfsvfs_teardown(zfsvfs_t *zfsvfs, boolean_t unmounting) * 'z_parent' is self referential for non-snapshots. */ #ifdef FREEBSD_NAMECACHE +#if __FreeBSD_version >= 1300117 + cache_purgevfs(zfsvfs->z_parent->z_vfs); +#else cache_purgevfs(zfsvfs->z_parent->z_vfs, true); #endif +#endif } /* diff --git a/module/os/freebsd/zfs/zio_crypt.c b/module/os/freebsd/zfs/zio_crypt.c index d89ef80edd66..fb88bc325d3c 100644 --- a/module/os/freebsd/zfs/zio_crypt.c +++ b/module/os/freebsd/zfs/zio_crypt.c @@ -1234,8 +1234,7 @@ zio_crypt_do_indirect_mac_checksum_abd(boolean_t generate, abd_t *abd, * accommodate some of the drivers, the authbuf needs to be logically before * the data. This means that we need to copy the source to the destination, * and set up an extra iovec_t at the beginning to handle the authbuf. - * It also means we'll only return one uio_t, which we do via the clumsy - * ifdef in the function declaration. + * It also means we'll only return one uio_t. */ /* ARGSUSED */ @@ -1245,52 +1244,46 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, uio_t *out_uio, uint_t *enc_len, uint8_t **authbuf, uint_t *auth_len, boolean_t *no_crypt) { - int ret; - uint64_t txtype, lr_len; - uint_t nr_src, nr_dst, crypt_len; - uint_t aad_len = 0, nr_iovecs = 0, total_len = 0; - iovec_t *src_iovecs = NULL, *dst_iovecs = NULL; + uint8_t *aadbuf = zio_buf_alloc(datalen); uint8_t *src, *dst, *slrp, *dlrp, *blkend, *aadp; + iovec_t *dst_iovecs; zil_chain_t *zilc; lr_t *lr; - uint8_t *aadbuf = zio_buf_alloc(datalen); + uint64_t txtype, lr_len; + uint_t crypt_len, nr_iovecs, vec; + uint_t aad_len = 0, total_len = 0; - /* cipherbuf always needs an extra iovec for the MAC */ if (encrypt) { src = plainbuf; dst = cipherbuf; - nr_src = 0; - nr_dst = 1; } else { src = cipherbuf; dst = plainbuf; - nr_src = 1; - nr_dst = 0; } - - /* - * We need at least two iovecs -- one for the AAD, - * one for the MAC. - */ bcopy(src, dst, datalen); - nr_dst = 2; - /* find the start and end record of the log block */ + /* Find the start and end record of the log block. */ zilc = (zil_chain_t *)src; slrp = src + sizeof (zil_chain_t); aadp = aadbuf; blkend = src + ((byteswap) ? BSWAP_64(zilc->zc_nused) : zilc->zc_nused); - /* calculate the number of encrypted iovecs we will need */ + /* + * Calculate the number of encrypted iovecs we will need. + */ + + /* We need at least two iovecs -- one for the AAD, one for the MAC. */ + nr_iovecs = 2; + for (; slrp < blkend; slrp += lr_len) { lr = (lr_t *)slrp; - if (!byteswap) { - txtype = lr->lrc_txtype; - lr_len = lr->lrc_reclen; - } else { + if (byteswap) { txtype = BSWAP_64(lr->lrc_txtype); lr_len = BSWAP_64(lr->lrc_reclen); + } else { + txtype = lr->lrc_txtype; + lr_len = lr->lrc_reclen; } nr_iovecs++; @@ -1298,27 +1291,7 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, nr_iovecs++; } - nr_src = 0; - nr_dst += nr_iovecs; - - /* allocate the iovec arrays */ - if (nr_src != 0) { - src_iovecs = kmem_alloc(nr_src * sizeof (iovec_t), KM_SLEEP); - if (src_iovecs == NULL) { - ret = SET_ERROR(ENOMEM); - goto error; - } - bzero(src_iovecs, nr_src * sizeof (iovec_t)); - } - - if (nr_dst != 0) { - dst_iovecs = kmem_alloc(nr_dst * sizeof (iovec_t), KM_SLEEP); - if (dst_iovecs == NULL) { - ret = SET_ERROR(ENOMEM); - goto error; - } - bzero(dst_iovecs, nr_dst * sizeof (iovec_t)); - } + dst_iovecs = kmem_alloc(nr_iovecs * sizeof (iovec_t), KM_SLEEP); /* * Copy the plain zil header over and authenticate everything except @@ -1326,18 +1299,20 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, * the embedded checksum will not have been calculated yet, so we don't * authenticate that. */ - bcopy(src, dst, sizeof (zil_chain_t)); bcopy(src, aadp, sizeof (zil_chain_t) - sizeof (zio_eck_t)); aadp += sizeof (zil_chain_t) - sizeof (zio_eck_t); aad_len += sizeof (zil_chain_t) - sizeof (zio_eck_t); - /* loop over records again, filling in iovecs */ - /* The first one will contain the authbuf */ - nr_iovecs = 1; - slrp = src + sizeof (zil_chain_t); dlrp = dst + sizeof (zil_chain_t); + /* + * Loop over records again, filling in iovecs. + */ + + /* The first iovec will contain the authbuf. */ + vec = 1; + for (; slrp < blkend; slrp += lr_len, dlrp += lr_len) { lr = (lr_t *)slrp; @@ -1355,8 +1330,6 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, aadp += sizeof (lr_t); aad_len += sizeof (lr_t); - ASSERT3P(dst_iovecs, !=, NULL); - /* * If this is a TX_WRITE record we want to encrypt everything * except the bp if exists. If the bp does exist we want to @@ -1365,9 +1338,9 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, if (txtype == TX_WRITE) { crypt_len = sizeof (lr_write_t) - sizeof (lr_t) - sizeof (blkptr_t); - dst_iovecs[nr_iovecs].iov_base = (char *)dlrp + + dst_iovecs[vec].iov_base = (char *)dlrp + sizeof (lr_t); - dst_iovecs[nr_iovecs].iov_len = crypt_len; + dst_iovecs[vec].iov_len = crypt_len; /* copy the bp now since it will not be encrypted */ bcopy(slrp + sizeof (lr_write_t) - sizeof (blkptr_t), @@ -1377,56 +1350,45 @@ zio_crypt_init_uios_zil(boolean_t encrypt, uint8_t *plainbuf, aadp, sizeof (blkptr_t)); aadp += sizeof (blkptr_t); aad_len += sizeof (blkptr_t); - nr_iovecs++; + vec++; total_len += crypt_len; if (lr_len != sizeof (lr_write_t)) { crypt_len = lr_len - sizeof (lr_write_t); - dst_iovecs[nr_iovecs].iov_base = (char *) + dst_iovecs[vec].iov_base = (char *) dlrp + sizeof (lr_write_t); - dst_iovecs[nr_iovecs].iov_len = crypt_len; - nr_iovecs++; + dst_iovecs[vec].iov_len = crypt_len; + vec++; total_len += crypt_len; } } else { crypt_len = lr_len - sizeof (lr_t); - dst_iovecs[nr_iovecs].iov_base = (char *)dlrp + + dst_iovecs[vec].iov_base = (char *)dlrp + sizeof (lr_t); - dst_iovecs[nr_iovecs].iov_len = crypt_len; - nr_iovecs++; + dst_iovecs[vec].iov_len = crypt_len; + vec++; total_len += crypt_len; } } - *no_crypt = (nr_iovecs == 0); - *enc_len = total_len; - *authbuf = aadbuf; - *auth_len = aad_len; + /* The last iovec will contain the MAC. */ + ASSERT3U(vec, ==, nr_iovecs - 1); + + /* AAD */ dst_iovecs[0].iov_base = aadbuf; dst_iovecs[0].iov_len = aad_len; + /* MAC */ + dst_iovecs[vec].iov_base = 0; + dst_iovecs[vec].iov_len = 0; + *no_crypt = (vec == 1); + *enc_len = total_len; + *authbuf = aadbuf; + *auth_len = aad_len; out_uio->uio_iov = dst_iovecs; - out_uio->uio_iovcnt = nr_dst; + out_uio->uio_iovcnt = nr_iovecs; return (0); - -error: - zio_buf_free(aadbuf, datalen); - if (src_iovecs != NULL) - kmem_free(src_iovecs, nr_src * sizeof (iovec_t)); - if (dst_iovecs != NULL) - kmem_free(dst_iovecs, nr_dst * sizeof (iovec_t)); - - *enc_len = 0; - *authbuf = NULL; - *auth_len = 0; - *no_crypt = B_FALSE; - puio->uio_iov = NULL; - puio->uio_iovcnt = 0; - out_uio->uio_iov = NULL; - out_uio->uio_iovcnt = 0; - - return (ret); } /* @@ -1438,29 +1400,22 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, uio_t *puio, uio_t *out_uio, uint_t *enc_len, uint8_t **authbuf, uint_t *auth_len, boolean_t *no_crypt) { - int ret; - uint_t nr_src, nr_dst, crypt_len; - uint_t aad_len = 0, nr_iovecs = 0, total_len = 0; - uint_t i, j, max_dnp = datalen >> DNODE_SHIFT; - iovec_t *src_iovecs = NULL, *dst_iovecs = NULL; + uint8_t *aadbuf = zio_buf_alloc(datalen); uint8_t *src, *dst, *aadp; dnode_phys_t *dnp, *adnp, *sdnp, *ddnp; - uint8_t *aadbuf = zio_buf_alloc(datalen); + iovec_t *dst_iovecs; + uint_t nr_iovecs, crypt_len, vec; + uint_t aad_len = 0, total_len = 0; + uint_t i, j, max_dnp = datalen >> DNODE_SHIFT; if (encrypt) { src = plainbuf; dst = cipherbuf; - nr_src = 0; - nr_dst = 1; } else { src = cipherbuf; dst = plainbuf; - nr_src = 1; - nr_dst = 0; } - bcopy(src, dst, datalen); - nr_dst = 2; sdnp = (dnode_phys_t *)src; ddnp = (dnode_phys_t *)dst; @@ -1470,6 +1425,10 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, * Count the number of iovecs we will need to do the encryption by * counting the number of bonus buffers that need to be encrypted. */ + + /* We need at least two iovecs -- one for the AAD, one for the MAC. */ + nr_iovecs = 2; + for (i = 0; i < max_dnp; i += sdnp[i].dn_extra_slots + 1) { /* * This block may still be byteswapped. However, all of the @@ -1484,34 +1443,17 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, } } - nr_src = 0; - nr_dst += nr_iovecs; - - if (nr_src != 0) { - src_iovecs = kmem_alloc(nr_src * sizeof (iovec_t), KM_SLEEP); - if (src_iovecs == NULL) { - ret = SET_ERROR(ENOMEM); - goto error; - } - bzero(src_iovecs, nr_src * sizeof (iovec_t)); - } - - if (nr_dst != 0) { - dst_iovecs = kmem_alloc(nr_dst * sizeof (iovec_t), KM_SLEEP); - if (dst_iovecs == NULL) { - ret = SET_ERROR(ENOMEM); - goto error; - } - bzero(dst_iovecs, nr_dst * sizeof (iovec_t)); - } - - nr_iovecs = 1; + dst_iovecs = kmem_alloc(nr_iovecs * sizeof (iovec_t), KM_SLEEP); /* * Iterate through the dnodes again, this time filling in the uios * we allocated earlier. We also concatenate any data we want to * authenticate onto aadbuf. */ + + /* The first iovec will contain the authbuf. */ + vec = 1; + for (i = 0; i < max_dnp; i += sdnp[i].dn_extra_slots + 1) { dnp = &sdnp[i]; @@ -1565,12 +1507,10 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, if (dnp->dn_type != DMU_OT_NONE && DMU_OT_IS_ENCRYPTED(dnp->dn_bonustype) && dnp->dn_bonuslen != 0) { - ASSERT3U(nr_iovecs, <, nr_dst); - ASSERT3P(dst_iovecs, !=, NULL); - dst_iovecs[nr_iovecs].iov_base = DN_BONUS(&ddnp[i]); - dst_iovecs[nr_iovecs].iov_len = crypt_len; + dst_iovecs[vec].iov_base = DN_BONUS(&ddnp[i]); + dst_iovecs[vec].iov_len = crypt_len; - nr_iovecs++; + vec++; total_len += crypt_len; } else { bcopy(DN_BONUS(dnp), DN_BONUS(&ddnp[i]), crypt_len); @@ -1580,33 +1520,24 @@ zio_crypt_init_uios_dnode(boolean_t encrypt, uint64_t version, } } - *no_crypt = (nr_iovecs == 0); - *enc_len = total_len; - *authbuf = aadbuf; - *auth_len = aad_len; + /* The last iovec will contain the MAC. */ + ASSERT3U(vec, ==, nr_iovecs - 1); + /* AAD */ dst_iovecs[0].iov_base = aadbuf; dst_iovecs[0].iov_len = aad_len; + /* MAC */ + dst_iovecs[vec].iov_base = 0; + dst_iovecs[vec].iov_len = 0; + + *no_crypt = (vec == 1); + *enc_len = total_len; + *authbuf = aadbuf; + *auth_len = aad_len; out_uio->uio_iov = dst_iovecs; - out_uio->uio_iovcnt = nr_dst; + out_uio->uio_iovcnt = nr_iovecs; return (0); - -error: - zio_buf_free(aadbuf, datalen); - if (src_iovecs != NULL) - kmem_free(src_iovecs, nr_src * sizeof (iovec_t)); - if (dst_iovecs != NULL) - kmem_free(dst_iovecs, nr_dst * sizeof (iovec_t)); - - *enc_len = 0; - *authbuf = NULL; - *auth_len = 0; - *no_crypt = B_FALSE; - out_uio->uio_iov = NULL; - out_uio->uio_iovcnt = 0; - - return (ret); } /* ARGSUSED */ diff --git a/module/os/linux/spl/spl-procfs-list.c b/module/os/linux/spl/spl-procfs-list.c index 189d6a7c6082..cae13228c62c 100644 --- a/module/os/linux/spl/spl-procfs-list.c +++ b/module/os/linux/spl/spl-procfs-list.c @@ -89,7 +89,17 @@ procfs_list_next_node(procfs_list_cursor_t *cursor, loff_t *pos) cursor->cached_node = next_node; cursor->cached_pos = NODE_ID(procfs_list, cursor->cached_node); *pos = cursor->cached_pos; + } else { + /* + * seq_read() expects ->next() to update the position even + * when there are no more entries. Advance the position to + * prevent a warning from being logged. + */ + cursor->cached_node = NULL; + cursor->cached_pos++; + *pos = cursor->cached_pos; } + return (next_node); } @@ -105,6 +115,8 @@ procfs_list_seq_start(struct seq_file *f, loff_t *pos) cursor->cached_node = SEQ_START_TOKEN; cursor->cached_pos = 0; return (SEQ_START_TOKEN); + } else if (cursor->cached_node == NULL) { + return (NULL); } /* @@ -207,6 +219,7 @@ static const kstat_proc_op_t procfs_list_operations = { */ void procfs_list_install(const char *module, + const char *submodule, const char *name, mode_t mode, procfs_list_t *procfs_list, @@ -215,6 +228,12 @@ procfs_list_install(const char *module, int (*clear)(procfs_list_t *procfs_list), size_t procfs_list_node_off) { + char *modulestr; + + if (submodule != NULL) + modulestr = kmem_asprintf("%s/%s", module, submodule); + else + modulestr = kmem_asprintf("%s", module); mutex_init(&procfs_list->pl_lock, NULL, MUTEX_DEFAULT, NULL); list_create(&procfs_list->pl_list, procfs_list_node_off + sizeof (procfs_list_node_t), @@ -225,9 +244,10 @@ procfs_list_install(const char *module, procfs_list->pl_clear = clear; procfs_list->pl_node_offset = procfs_list_node_off; - kstat_proc_entry_init(&procfs_list->pl_kstat_entry, module, name); + kstat_proc_entry_init(&procfs_list->pl_kstat_entry, modulestr, name); kstat_proc_entry_install(&procfs_list->pl_kstat_entry, mode, &procfs_list_operations, procfs_list); + kmem_strfree(modulestr); } EXPORT_SYMBOL(procfs_list_install); diff --git a/module/os/linux/zfs/vdev_disk.c b/module/os/linux/zfs/vdev_disk.c index 85daef43be40..a54961c76870 100644 --- a/module/os/linux/zfs/vdev_disk.c +++ b/module/os/linux/zfs/vdev_disk.c @@ -436,6 +436,16 @@ vdev_submit_bio_impl(struct bio *bio) #endif } +/* + * preempt_schedule_notrace is GPL-only which breaks the ZFS build, so + * replace it with preempt_schedule under the following condition: + */ +#if defined(CONFIG_ARM64) && \ + defined(CONFIG_PREEMPTION) && \ + defined(CONFIG_BLK_CGROUP) +#define preempt_schedule_notrace(x) preempt_schedule(x) +#endif + #ifdef HAVE_BIO_SET_DEV #if defined(CONFIG_BLK_CGROUP) && defined(HAVE_BIO_SET_DEV_GPL_ONLY) /* diff --git a/module/os/linux/zfs/zfs_debug.c b/module/os/linux/zfs/zfs_debug.c index d98463f1b7f7..8d7f04097da8 100644 --- a/module/os/linux/zfs/zfs_debug.c +++ b/module/os/linux/zfs/zfs_debug.c @@ -94,6 +94,7 @@ void zfs_dbgmsg_init(void) { procfs_list_install("zfs", + NULL, "dbgmsg", 0600, &zfs_dbgmsgs, diff --git a/module/zfs/arc.c b/module/zfs/arc.c index 7a499298f75c..3ba198380733 100644 --- a/module/zfs/arc.c +++ b/module/zfs/arc.c @@ -308,6 +308,7 @@ #include <sys/aggsum.h> #include <cityhash.h> #include <sys/vdev_trim.h> +#include <sys/zstd/zstd.h> #ifndef _KERNEL /* set with ZFS_DEBUG=watch, to enable watchpoints on frozen buffers */ @@ -4859,6 +4860,7 @@ static boolean_t arc_reap_cb_check(void *arg, zthr_t *zthr) { int64_t free_memory = arc_available_memory(); + static int reap_cb_check_counter = 0; /* * If a kmem reap is already active, don't schedule more. We must @@ -4883,6 +4885,14 @@ arc_reap_cb_check(void *arg, zthr_t *zthr) arc_no_grow = B_FALSE; } + /* + * Called unconditionally every 60 seconds to reclaim unused + * zstd compression and decompression context. This is done + * here to avoid the need for an independent thread. + */ + if (!((reap_cb_check_counter++) % 60)) + zfs_zstd_cache_reap_now(); + return (B_FALSE); } diff --git a/module/zfs/dbuf_stats.c b/module/zfs/dbuf_stats.c index a2f3c580ee6c..12bb568a08cc 100644 --- a/module/zfs/dbuf_stats.c +++ b/module/zfs/dbuf_stats.c @@ -134,7 +134,8 @@ dbuf_stats_hash_table_data(char *buf, size_t size, void *data) ASSERT3S(dsh->idx, >=, 0); ASSERT3S(dsh->idx, <=, h->hash_table_mask); - memset(buf, 0, size); + if (size) + buf[0] = 0; mutex_enter(DBUF_HASH_MUTEX(h, dsh->idx)); for (db = h->hash_table[dsh->idx]; db != NULL; db = db->db_hash_next) { diff --git a/module/zfs/dmu_send.c b/module/zfs/dmu_send.c index 33e99c2e02ab..9480c8b75497 100644 --- a/module/zfs/dmu_send.c +++ b/module/zfs/dmu_send.c @@ -643,7 +643,7 @@ dump_freeobjects(dmu_send_cookie_t *dscp, uint64_t firstobj, uint64_t numobjs) * receiving side. */ if (maxobj > 0) { - if (maxobj < firstobj) + if (maxobj <= firstobj) return (0); if (maxobj < firstobj + numobjs) @@ -663,8 +663,6 @@ dump_freeobjects(dmu_send_cookie_t *dscp, uint64_t firstobj, uint64_t numobjs) return (SET_ERROR(EINTR)); dscp->dsc_pending_op = PENDING_NONE; } - if (numobjs == 0) - numobjs = UINT64_MAX - firstobj; if (dscp->dsc_pending_op == PENDING_FREEOBJECTS) { /* @@ -2686,12 +2684,15 @@ dmu_send_obj(const char *pool, uint64_t tosnap, uint64_t fromsnap, bcopy(fromredact, dspp.fromredactsnaps, size); } - if (!dsl_dataset_is_before(dspp.to_ds, fromds, 0)) { + boolean_t is_before = + dsl_dataset_is_before(dspp.to_ds, fromds, 0); + dspp.is_clone = (dspp.to_ds->ds_dir != + fromds->ds_dir); + dsl_dataset_rele(fromds, FTAG); + if (!is_before) { + dsl_pool_rele(dspp.dp, FTAG); err = SET_ERROR(EXDEV); } else { - dspp.is_clone = (dspp.to_ds->ds_dir != - fromds->ds_dir); - dsl_dataset_rele(fromds, FTAG); err = dmu_send_impl(&dspp); } } else { diff --git a/module/zfs/dnode.c b/module/zfs/dnode.c index 30d20bfefa12..23364dbae897 100644 --- a/module/zfs/dnode.c +++ b/module/zfs/dnode.c @@ -1355,7 +1355,8 @@ dnode_hold_impl(objset_t *os, uint64_t object, int flag, int slots, * We do not need to decrypt to read the dnode so it doesn't matter * if we get the encrypted or decrypted version. */ - err = dbuf_read(db, NULL, DB_RF_CANFAIL | DB_RF_NO_DECRYPT); + err = dbuf_read(db, NULL, DB_RF_CANFAIL | + DB_RF_NO_DECRYPT | DB_RF_NOPREFETCH); if (err) { DNODE_STAT_BUMP(dnode_hold_dbuf_read); dbuf_rele(db, FTAG); @@ -2396,7 +2397,8 @@ dnode_next_offset_level(dnode_t *dn, int flags, uint64_t *offset, return (SET_ERROR(ESRCH)); } error = dbuf_read(db, NULL, - DB_RF_CANFAIL | DB_RF_HAVESTRUCT | DB_RF_NO_DECRYPT); + DB_RF_CANFAIL | DB_RF_HAVESTRUCT | + DB_RF_NO_DECRYPT | DB_RF_NOPREFETCH); if (error) { dbuf_rele(db, FTAG); return (error); diff --git a/module/zfs/dsl_crypt.c b/module/zfs/dsl_crypt.c index 33c21e86c4d7..26d4c2fe7e33 100644 --- a/module/zfs/dsl_crypt.c +++ b/module/zfs/dsl_crypt.c @@ -235,11 +235,7 @@ dsl_crypto_params_create_nvlist(dcp_cmd_t cmd, nvlist_t *props, return (0); error: - if (wkey != NULL) - dsl_wrapping_key_free(wkey); - if (dcp != NULL) - kmem_free(dcp, sizeof (dsl_crypto_params_t)); - + kmem_free(dcp, sizeof (dsl_crypto_params_t)); *dcp_out = NULL; return (ret); } diff --git a/module/zfs/spa_misc.c b/module/zfs/spa_misc.c index 41f0ddbde288..04210472886c 100644 --- a/module/zfs/spa_misc.c +++ b/module/zfs/spa_misc.c @@ -2169,6 +2169,7 @@ spa_import_progress_init(void) spa_import_progress_list; procfs_list_install("zfs", + NULL, "import_progress", 0644, &spa_import_progress_list->procfs_list, diff --git a/module/zfs/spa_stats.c b/module/zfs/spa_stats.c index cf0be3c45dc8..c3eacc14239e 100644 --- a/module/zfs/spa_stats.c +++ b/module/zfs/spa_stats.c @@ -122,14 +122,11 @@ static void spa_read_history_init(spa_t *spa) { spa_history_list_t *shl = &spa->spa_stats.read_history; - char *module; shl->size = 0; - - module = kmem_asprintf("zfs/%s", spa_name(spa)); - shl->procfs_list.pl_private = shl; - procfs_list_install(module, + procfs_list_install("zfs", + spa_name(spa), "reads", 0600, &shl->procfs_list, @@ -137,8 +134,6 @@ spa_read_history_init(spa_t *spa) spa_read_history_show_header, spa_read_history_clear, offsetof(spa_read_history_t, srh_node)); - - kmem_strfree(module); } static void @@ -293,14 +288,11 @@ static void spa_txg_history_init(spa_t *spa) { spa_history_list_t *shl = &spa->spa_stats.txg_history; - char *module; shl->size = 0; - - module = kmem_asprintf("zfs/%s", spa_name(spa)); - shl->procfs_list.pl_private = shl; - procfs_list_install(module, + procfs_list_install("zfs", + spa_name(spa), "txgs", 0644, &shl->procfs_list, @@ -308,8 +300,6 @@ spa_txg_history_init(spa_t *spa) spa_txg_history_show_header, spa_txg_history_clear, offsetof(spa_txg_history_t, sth_node)); - - kmem_strfree(module); } static void @@ -699,14 +689,12 @@ static void spa_mmp_history_init(spa_t *spa) { spa_history_list_t *shl = &spa->spa_stats.mmp_history; - char *module; shl->size = 0; - module = kmem_asprintf("zfs/%s", spa_name(spa)); - shl->procfs_list.pl_private = shl; - procfs_list_install(module, + procfs_list_install("zfs", + spa_name(spa), "multihost", 0644, &shl->procfs_list, @@ -714,8 +702,6 @@ spa_mmp_history_init(spa_t *spa) spa_mmp_history_show_header, spa_mmp_history_clear, offsetof(spa_mmp_history_t, smh_node)); - - kmem_strfree(module); } static void diff --git a/module/zfs/zfs_log.c b/module/zfs/zfs_log.c index 4eae855f4274..fb44007fefc3 100644 --- a/module/zfs/zfs_log.c +++ b/module/zfs/zfs_log.c @@ -584,15 +584,22 @@ zfs_log_write(zilog_t *zilog, dmu_tx_t *tx, int txtype, (wr_state == WR_COPIED ? len : 0)); lr = (lr_write_t *)&itx->itx_lr; - DB_DNODE_ENTER(db); - if (wr_state == WR_COPIED && dmu_read_by_dnode(DB_DNODE(db), - off, len, lr + 1, DMU_READ_NO_PREFETCH) != 0) { - zil_itx_destroy(itx); - itx = zil_itx_create(txtype, sizeof (*lr)); - lr = (lr_write_t *)&itx->itx_lr; - wr_state = WR_NEED_COPY; + /* + * For WR_COPIED records, copy the data into the lr_write_t. + */ + if (wr_state == WR_COPIED) { + int err; + DB_DNODE_ENTER(db); + err = dmu_read_by_dnode(DB_DNODE(db), off, len, lr + 1, + DMU_READ_NO_PREFETCH); + if (err != 0) { + zil_itx_destroy(itx); + itx = zil_itx_create(txtype, sizeof (*lr)); + lr = (lr_write_t *)&itx->itx_lr; + wr_state = WR_NEED_COPY; + } + DB_DNODE_EXIT(db); } - DB_DNODE_EXIT(db); itx->itx_wr_state = wr_state; lr->lr_foid = zp->z_id; diff --git a/module/zstd/zfs_zstd.c b/module/zstd/zfs_zstd.c index 431801d47c37..34c56b7a7fdf 100644 --- a/module/zstd/zfs_zstd.c +++ b/module/zstd/zfs_zstd.c @@ -238,7 +238,7 @@ zstd_mempool_alloc(struct zstd_pool *zstd_mempool, size_t size) * Check if objects fits the size, if so we take it and * update the timestamp. */ - if (!mem && pool->mem && size <= pool->size) { + if (size && !mem && pool->mem && size <= pool->size) { pool->timeout = gethrestime_sec() + ZSTD_POOL_TIMEOUT; mem = pool->mem; @@ -257,7 +257,7 @@ zstd_mempool_alloc(struct zstd_pool *zstd_mempool, size_t size) } } - if (mem) { + if (!size || mem) { return (mem); } @@ -688,6 +688,19 @@ zstd_mempool_deinit(void) zstd_mempool_cctx = NULL; } +/* release unused memory from pool */ + +void +zfs_zstd_cache_reap_now(void) +{ + /* + * calling alloc with zero size seeks + * and releases old unused objects + */ + zstd_mempool_alloc(zstd_mempool_cctx, 0); + zstd_mempool_alloc(zstd_mempool_dctx, 0); +} + extern int __init zstd_init(void) { @@ -729,10 +742,11 @@ module_init(zstd_init); module_exit(zstd_fini); ZFS_MODULE_DESCRIPTION("ZSTD Compression for ZFS"); -ZFS_MODULE_LICENSE("BSD"); +ZFS_MODULE_LICENSE("Dual BSD/GPL"); ZFS_MODULE_VERSION(ZSTD_VERSION_STRING); EXPORT_SYMBOL(zfs_zstd_compress); EXPORT_SYMBOL(zfs_zstd_decompress_level); EXPORT_SYMBOL(zfs_zstd_decompress); +EXPORT_SYMBOL(zfs_zstd_cache_reap_now); #endif diff --git a/tests/runfiles/common.run b/tests/runfiles/common.run index 725afe2f054a..e06281648e70 100644 --- a/tests/runfiles/common.run +++ b/tests/runfiles/common.run @@ -790,7 +790,7 @@ tests = ['recv_dedup', 'recv_dedup_encrypted_zvol', 'rsend_001_pos', 'send_freeobjects', 'send_realloc_files', 'send_realloc_encrypted_files', 'send_spill_block', 'send_holds', 'send_hole_birth', 'send_mixed_raw', 'send-wR_encrypted_zvol', - 'send_partial_dataset'] + 'send_partial_dataset', 'send_invalid'] tags = ['functional', 'rsend'] [tests/functional/scrub_mirror] diff --git a/tests/zfs-tests/cmd/Makefile.am b/tests/zfs-tests/cmd/Makefile.am index 20e85cf6bb3f..bf54c1d45710 100644 --- a/tests/zfs-tests/cmd/Makefile.am +++ b/tests/zfs-tests/cmd/Makefile.am @@ -1,6 +1,7 @@ EXTRA_DIST = file_common.h SUBDIRS = \ + badsend \ btree_test \ chg_usr_exec \ devname2devid \ diff --git a/tests/zfs-tests/cmd/badsend/.gitignore b/tests/zfs-tests/cmd/badsend/.gitignore new file mode 100644 index 000000000000..d2efa627aa29 --- /dev/null +++ b/tests/zfs-tests/cmd/badsend/.gitignore @@ -0,0 +1 @@ +/badsend diff --git a/tests/zfs-tests/cmd/badsend/Makefile.am b/tests/zfs-tests/cmd/badsend/Makefile.am new file mode 100644 index 000000000000..5a8946f0d4bf --- /dev/null +++ b/tests/zfs-tests/cmd/badsend/Makefile.am @@ -0,0 +1,11 @@ +include $(top_srcdir)/config/Rules.am + +pkgexecdir = $(datadir)/@PACKAGE@/zfs-tests/bin + +pkgexec_PROGRAMS = badsend + +badsend_SOURCES = badsend.c +badsend_LDADD = \ + $(abs_top_builddir)/lib/libzfs_core/libzfs_core.la \ + $(abs_top_builddir)/lib/libzfs/libzfs.la \ + $(abs_top_builddir)/lib/libnvpair/libnvpair.la diff --git a/tests/zfs-tests/cmd/badsend/badsend.c b/tests/zfs-tests/cmd/badsend/badsend.c new file mode 100644 index 000000000000..af17bc7255b4 --- /dev/null +++ b/tests/zfs-tests/cmd/badsend/badsend.c @@ -0,0 +1,136 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Portions Copyright 2020 iXsystems, Inc. + */ + +/* + * Test some invalid send operations with libzfs/libzfs_core. + * + * Specifying the to and from snaps in the wrong order should return EXDEV. + * We are checking that the early return doesn't accidentally leave any + * references held, so this test is designed to trigger a panic when asserts + * are verified with the bug present. + */ + +#include <libzfs.h> +#include <libzfs_core.h> + +#include <fcntl.h> +#include <stdlib.h> +#include <string.h> +#include <unistd.h> +#include <sysexits.h> +#include <err.h> + +static void +usage(const char *name) +{ + fprintf(stderr, "usage: %s snap0 snap1\n", name); + exit(EX_USAGE); +} + +int +main(int argc, char const * const argv[]) +{ + sendflags_t flags = { 0 }; + libzfs_handle_t *zhdl; + zfs_handle_t *zhp; + const char *fromfull, *tofull, *fsname, *fromsnap, *tosnap, *p; + uint64_t size; + int fd, error; + + if (argc != 3) + usage(argv[0]); + + fromfull = argv[1]; + tofull = argv[2]; + + p = strchr(fromfull, '@'); + if (p == NULL) + usage(argv[0]); + fromsnap = p + 1; + + p = strchr(tofull, '@'); + if (p == NULL) + usage(argv[0]); + tosnap = p + 1; + + fsname = strndup(tofull, p - tofull); + if (strncmp(fsname, fromfull, p - tofull) != 0) + usage(argv[0]); + + fd = open("/dev/null", O_WRONLY); + if (fd == -1) + err(EX_OSERR, "open(\"/dev/null\", O_WRONLY)"); + + zhdl = libzfs_init(); + if (zhdl == NULL) + errx(EX_OSERR, "libzfs_init(): %s", libzfs_error_init(errno)); + + zhp = zfs_open(zhdl, fsname, ZFS_TYPE_FILESYSTEM); + if (zhp == NULL) + err(EX_OSERR, "zfs_open(\"%s\")", fsname); + + /* + * Exercise EXDEV in dmu_send_obj. The error gets translated to + * EZFS_CROSSTARGET in libzfs. + */ + error = zfs_send(zhp, tosnap, fromsnap, &flags, fd, NULL, NULL, NULL); + if (error == 0 || libzfs_errno(zhdl) != EZFS_CROSSTARGET) + errx(EX_OSERR, "zfs_send(\"%s\", \"%s\") should have failed " + "with EZFS_CROSSTARGET, not %d", + tofull, fromfull, libzfs_errno(zhdl)); + printf("zfs_send(\"%s\", \"%s\"): %s\n", + tofull, fromfull, libzfs_error_description(zhdl)); + + zfs_close(zhp); + + /* + * Exercise EXDEV in dmu_send. + */ + error = lzc_send_resume_redacted(fromfull, tofull, fd, 0, 0, 0, NULL); + if (error != EXDEV) + errx(EX_OSERR, "lzc_send_resume_redacted(\"%s\", \"%s\")" + " should have failed with EXDEV, not %d", + fromfull, tofull, error); + printf("lzc_send_resume_redacted(\"%s\", \"%s\"): %s\n", + fromfull, tofull, strerror(error)); + + /* + * Exercise EXDEV in dmu_send_estimate_fast. + */ + error = lzc_send_space_resume_redacted(fromfull, tofull, 0, 0, 0, 0, + NULL, fd, &size); + if (error != EXDEV) + errx(EX_OSERR, "lzc_send_space_resume_redacted(\"%s\", \"%s\")" + " should have failed with EXDEV, not %d", + fromfull, tofull, error); + printf("lzc_send_space_resume_redacted(\"%s\", \"%s\"): %s\n", + fromfull, tofull, strerror(error)); + + close(fd); + libzfs_fini(zhdl); + free((void *)fsname); + + return (EXIT_SUCCESS); +} diff --git a/tests/zfs-tests/include/commands.cfg b/tests/zfs-tests/include/commands.cfg index 4c11bf146378..5a507b94ab6c 100644 --- a/tests/zfs-tests/include/commands.cfg +++ b/tests/zfs-tests/include/commands.cfg @@ -190,7 +190,8 @@ export ZFS_FILES='zdb zstreamdump zfs_ids_to_path' -export ZFSTEST_FILES='btree_test +export ZFSTEST_FILES='badsend + btree_test chg_usr_exec devname2devid dir_rd_update diff --git a/tests/zfs-tests/tests/functional/rsend/Makefile.am b/tests/zfs-tests/tests/functional/rsend/Makefile.am index cf6c727dfa16..61be2ec1889d 100644 --- a/tests/zfs-tests/tests/functional/rsend/Makefile.am +++ b/tests/zfs-tests/tests/functional/rsend/Makefile.am @@ -51,6 +51,7 @@ dist_pkgdata_SCRIPTS = \ send_spill_block.ksh \ send_holds.ksh \ send_hole_birth.ksh \ + send_invalid.ksh \ send_mixed_raw.ksh \ send-wR_encrypted_zvol.ksh diff --git a/tests/zfs-tests/tests/functional/rsend/send_invalid.ksh b/tests/zfs-tests/tests/functional/rsend/send_invalid.ksh new file mode 100644 index 000000000000..a0abe64b4cca --- /dev/null +++ b/tests/zfs-tests/tests/functional/rsend/send_invalid.ksh @@ -0,0 +1,52 @@ +#!/bin/ksh + +# +# This file and its contents are supplied under the terms of the +# Common Development and Distribution License ("CDDL"), version a.0. +# You may only use this file in accordance with the terms of version +# a.0 of the CDDL. +# +# A full copy of the text of the CDDL should have accompanied this +# source. A copy of the CDDL is also available via the Internet at +# http://www.illumos.org/license/CDDL. +# + +# +# Portions Copyright 2020 iXsystems, Inc. +# + +. $STF_SUITE/include/libtest.shlib +. $STF_SUITE/tests/functional/rsend/rsend.kshlib + +# +# Description: +# Verify that send with invalid options will fail gracefully. +# +# Strategy: +# 1. Perform zfs send on the cli with the order of the snapshots reversed +# 2. Perform zfs send using libzfs with the order of the snapshots reversed +# + +verify_runnable "both" + +log_assert "Verify that send with invalid options will fail gracefully." + +function cleanup +{ + datasetexists $testfs && destroy_dataset $testfs -r +} +log_onexit cleanup + +testfs=$POOL/fs + +log_must zfs create $testfs +log_must zfs snap $testfs@snap0 +log_must zfs snap $testfs@snap1 + +# Test bad send with the CLI +log_mustnot eval "zfs send -i $testfs@snap1 $testfs@snap0 >/dev/null" + +# Test bad send with libzfs/libzfs_core +log_must badsend $testfs@snap0 $testfs@snap1 + +log_pass "Send with invalid options fails gracefully." |