src/tests, branch releng/12.2

MF stable/12 r366190:

2020-09-28T00:23:59Z

fusefs: fix mmap'd writes in direct_io mode If a FUSE server returns FOPEN_DIRECT_IO in response to FUSE_OPEN, that instructs the kernel to bypass the page cache for that file. This feature is also known by libfuse's name: "direct_io". However, when accessing a file via mmap, there is no possible way to bypass the cache completely. This change fixes a deadlock that would happen when an mmap'd write tried to invalidate a portion of the cache, wrongly assuming that a write couldn't possibly come from cache if direct_io were set. Arguably, we could instead disable mmap for files with FOPEN_DIRECT_IO set. But allowing it is less likely to cause user complaints, and is more in keeping with the spirit of open(2), where O_DIRECT instructs the kernel to "reduce", not "eliminate" cache effects. PR: 247276 Approved by: re (gjb) Reported by: trapexit@spawn.link Reviewed by: cem Differential Revision: https://reviews.freebsd.org/D26485

MFS r365689,r365808,r365860

2020-09-19T20:46:56Z

MFOpenZFS: Introduce read/write kstats per dataset The following patch introduces a few statistics on reads and writes grouped by dataset. These statistics are implemented as kstats (backed by aggregate sums for performance) and can be retrieved by using the dataset objset ID number. The motivation for this change is to provide some preliminary analytics on dataset usage/performance. Reviewed-by: Richard Elling Reviewed-by: Brian Behlendorf Reviewed by: Matthew Ahrens Signed-off-by: Serapheim Dimitropoulos openzfs/zfs@a448a2557ec4938ed6944c7766fe0b8e6e5f6456 Also contains parts of: MFOpenZFS: Connect dataset_kstats for FreeBSD Example output: kstat.zfs.mypool.dataset.objset-0x10b.nread: 150528 kstat.zfs.mypool.dataset.objset-0x10b.reads: 48 kstat.zfs.mypool.dataset.objset-0x10b.nwritten: 134217728 kstat.zfs.mypool.dataset.objset-0x10b.writes: 1024 kstat.zfs.mypool.dataset.objset-0x10b.dataset_name: mypool/datasetname Reviewed-by: Ryan Moeller Reviewed by: Sean Eric Fagan Reviewed-by: Serapheim Dimitropoulos Reviewed-by: Brian Behlendorf Signed-off-by: Allan Jude openzfs/zfs@4547fc4e071ceb1818b3a46c3035b923e06e5390 Approved by: re (gjb) Relnotes: yes Sponsored by: Klara Inc.

MFC r360509: geli tests require OpenSSL - exclude them under WITHOUT_OPENSSL

2020-08-17T17:48:28Z

PR: 245931 Submitted by: The FreeBSD Foundation

MFC r359836-r359837, r359891, r360236-r360237: close_range(2)

2020-08-05T03:53:57Z

This commit diverges from our usual procedure of committing generated files separately because the original commit, r359836, conflated userland use (in tests) with the implementation. __FreeBSD_version was already bumped ~15 hours prior to this MFC, 1201522 marks the introduction of close_range. CPython will use this syscall for some pretty sizable performance gains. r359836: Implement a close_range(2) syscall close_range(min, max, flags) allows for a range of descriptors to be closed. The Python folk have indicated that they would much prefer this interface to closefrom(2), as the case may be that they/someone have special fds dup'd to higher in the range and they can't necessarily closefrom(min) because they don't want to hit the upper range, but relocating them to lower isn't necessarily feasible. sys_closefrom has been rewritten to use kern_close_range() using ~0U to indicate closing to the end of the range. This was chosen rather than requiring callers of kern_close_range() to hold FILEDESC_SLOCK across the call to kern_close_range for simplicity. The flags argument of close_range(2) is currently unused, so any flags set is currently EINVAL. It was added to the interface in Linux so that future flags could be added for, e.g., "halt on first error" and things of this nature. This patch is based on a syscall of the same design that is expected to be merged into Linux. r359837: sysent: re-roll after introduction of close_range in r359836 r359891: close_range/closefrom: fix regression from close_range introduction close_range will clamp the range between [0, fdp->fd_lastfile], but failed to take into account that fdp->fd_lastfile can become -1 if all fds are closed. =-( In this scenario, just return because there's nothing further we can do at the moment. Add a test case for this, fork() and simply closefrom(0) twice in the child; on the second invocation, fdp->fd_lastfile == -1 and will trigger a panic before this change. r360236: close_range(2): use newly assigned AUE_CLOSERANGE r360237: sysent: re-roll after 360236 (AUE_CLOSERANGE used)

MFC r363014:

2020-07-24T18:19:25Z

geli: enable direct dispatch geli does all of its crypto operations in a separate thread pool, so g_eli_start, g_eli_read_done, and g_eli_write_done don't actually do very much work. Enabling direct dispatch eliminates the g_up/g_down bottlenecks, doubling IOPs on my system. This change does not affect the thread pool. Reviewed by: markj Sponsored by: Axcient Differential Revision: https://reviews.freebsd.org/D25587

MFC r363181:

2020-07-22T14:24:41Z

Add safexcel(4) to cryptotest.

MFC r361401:

2020-06-12T20:32:26Z

Fix issues with FUSE_ACCESS when default_permissions is disabled This patch fixes two issues relating to FUSE_ACCESS when the default_permissions mount option is disabled: * VOP_ACCESS() calls with VADMIN set should never be sent to a fuse server in the form of FUSE_ACCESS operations. The FUSE protocol has no equivalent of VADMIN, so we must evaluate such things kernel-side, regardless of the default_permissions setting. * The FUSE protocol only requires FUSE_ACCESS to be sent for two purposes: for the access(2) syscall and to check directory permissions for searchability during lookup. FreeBSD sends it much more frequently, due to differences between our VFS and Linux's, for which FUSE was designed. But this patch does eliminate several cases not required by the FUSE protocol: * for any FUSE_*XATTR operation * when creating a new file * when deleting a file * when setting timestamps, such as by utimensat(2). * Additionally, when default_permissions is disabled, this patch removes one FUSE_GETATTR operation when deleting a file. PR: 245689 Reported by: MooseFS FreeBSD Team Reviewed by: cem Differential Revision: https://reviews.freebsd.org/D24777

MFC r361223:

2020-06-12T20:11:25Z

fusefs: fix intermittency in some ENOENT tests When a FUSE operation other than LOOKUP returns ENOENT, the kernel will reclaim that vnode, resuling in a FUSE_FORGET being sent a short while later. Many of the ENOENT tests weren't expecting those FUSE_FORGET operations. They usually passed by luck since FUSE_FORGET is often delayed. This commit adds appropriate expectations.

MFC r361286:

2020-05-27T15:06:03Z

Avoid hard-coding pipe buffer sizes in the pipe and fifo kqueue tests.

Fix a mismerge in r361454.

2020-05-25T13:14:40Z

This is a direct commit to stable/12. Reported by: Jenkins