summaryrefslogtreecommitdiff
path: root/sys/vm/vm_fault.c
Commit message (Collapse)AuthorAgeFilesLines
* Make MAXPHYS tunable. Bump MAXPHYS to 1M.Konstantin Belousov2020-11-281-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace MAXPHYS by runtime variable maxphys. It is initialized from MAXPHYS by default, but can be also adjusted with the tunable kern.maxphys. Make b_pages[] array in struct buf flexible. Size b_pages[] for buffer cache buffers exactly to atop(maxbcachebuf) (currently it is sized to atop(MAXPHYS)), and b_pages[] for pbufs is sized to atop(maxphys) + 1. The +1 for pbufs allow several pbuf consumers, among them vmapbuf(), to use unaligned buffers still sized to maxphys, esp. when such buffers come from userspace (*). Overall, we save significant amount of otherwise wasted memory in b_pages[] for buffer cache buffers, while bumping MAXPHYS to desired high value. Eliminate all direct uses of the MAXPHYS constant in kernel and driver sources, except a place which initialize maxphys. Some random (and arguably weird) uses of MAXPHYS, e.g. in linuxolator, are converted straight. Some drivers, which use MAXPHYS to size embeded structures, get private MAXPHYS-like constant; their convertion is out of scope for this work. Changes to cam/, dev/ahci, dev/ata, dev/mpr, dev/mpt, dev/mvs, dev/siis, where either submitted by, or based on changes by mav. Suggested by: mav (*) Reviewed by: imp, mav, imp, mckusick, scottl (intermediate versions) Tested by: pho Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D27225 Notes: svn path=/head/; revision=368124
* Implement superpages for PowerPC64 (HPT)Leandro Lupori2020-11-061-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | This change adds support for transparent superpages for PowerPC64 systems using Hashed Page Tables (HPT). All pmap operations are supported. The changes were inspired by RISC-V implementation of superpages, by @markj (r344106), but heavily adapted to fit PPC64 HPT architecture and existing MMU OEA64 code. While these changes are not better tested, superpages support is disabled by default. To enable it, use vm.pmap.superpages_enabled=1. In this initial implementation, when superpages are disabled, system performance stays at the same level as without these changes. When superpages are enabled, buildworld time increases a bit (~2%). However, for workloads that put a heavy pressure on the TLB the performance boost is much bigger (see HPC Challenge and pgbench on D25237). Reviewed by: jhibbits Sponsored by: Eldorado Research Institute (eldorado.org.br) Differential Revision: https://reviews.freebsd.org/D25237 Notes: svn path=/head/; revision=367417
* Implement sparse core dumpsMark Johnston2020-10-021-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently we allocate and map zero-filled anonymous pages when dumping core. This can result in lots of needless disk I/O and page allocations. This change tries to make the core dumper more clever and represent unbacked ranges of virtual memory by holes in the core dump file. Add a new page fault type, VM_FAULT_NOFILL, which causes vm_fault() to clean up and return an error when it would otherwise map a zero-filled page. Then, in the core dumper code, prefault all user pages and handle errors by simply extending the size of the core file. This also fixes a bug related to the fact that vn_io_fault1() does not attempt partial I/O in the face of errors from vm_fault_quick_hold_pages(): if a truncated file is mapped into a user process, an attempt to dump beyond the end of the file results in an error, but this means that valid pages immediately preceding the end of the file might not have been dumped either. The change reduces the core dump size of trivial programs by a factor of ten simply by excluding unaccessed libc.so pages. PR: 249067 Reviewed by: kib Tested by: pho MFC after: 1 month Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D26590 Notes: svn path=/head/; revision=366368
* Add a vmparam.h constant indicating pmap support for large pages.Mark Johnston2020-09-231-0/+2
| | | | | | | | | | | Enable SHM_LARGEPAGE support on arm64. Reviewed by: alc, kib Sponsored by: Juniper Networks, Inc., Klara, Inc. Differential Revision: https://reviews.freebsd.org/D26467 Notes: svn path=/head/; revision=366090
* Support for userspace non-transparent superpages (largepages).Konstantin Belousov2020-09-091-4/+70
| | | | | | | | | | | | | | | | | | | Created with shm_open2(SHM_LARGEPAGE) and then configured with FIOSSHMLPGCNF ioctl, largepages posix shared memory objects guarantee that all userspace mappings of it are served by superpage non-managed mappings. Only amd64 for now, both 2M and 1G superpages can be requested, the later requires CPU feature. Reviewed by: markj Tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D24652 Notes: svn path=/head/; revision=365522
* vm: clean up empty lines in .c and .h filesMateusz Guzik2020-09-011-2/+0
| | | | Notes: svn path=/head/; revision=365074
* vfs: remove the thread argument from vgetMateusz Guzik2020-08-161-2/+2
| | | | | | | | | | | | | | | | | | It was already asserted to be curthread. Semantic patch: @@ expression arg1, arg2, arg3; @@ - vget(arg1, arg2, arg3) + vget(arg1, arg2) Notes: svn path=/head/; revision=364271
* Add a helper function for validating VA ranges.Mark Johnston2020-06-191-4/+1
| | | | | | | | | | | | | | Functions which take untrusted user ranges must validate against the bounds of the map, and also check for wraparound. Instead of having the same logic duplicated in a number of places, add a function to check. Reviewed by: dougm, kib Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D25328 Notes: svn path=/head/; revision=362361
* Simplify the condition to enable superpage mappings in vm_fault_soft_fast().Konstantin Belousov2020-05-271-6/+2
| | | | | | | | | | | | | The list of arches list there matches the list of arches where default VM_NRESERVLEVEL > 0. Before sparc64 removal, that was the only arch that defined VM_NRESERVLEVEL > 0 to help with cache coloring, but did not implemented superpages. Now it can be simplified. Submitted by: alc Reviewed by: markj Notes: svn path=/head/; revision=361563
* Properly sort ifdef archs in vm_fault_soft_fast superpage guards.Justin Hibbits2020-05-271-6/+6
| | | | | | | Sort broken in r360887. Notes: svn path=/head/; revision=361545
* powerpc64: Implement Radix MMU for POWER9 CPUsJustin Hibbits2020-05-111-2/+2
| | | | | | | | | | | | | | | | Summary: POWER9 supports two MMU formats: traditional hashed page tables, and Radix page tables, similar to what's presesnt on most other architectures. The PowerISA also specifies a process table -- a table of page table pointers-- which on the POWER9 is only available with the Radix MMU, so we can take advantage of it with the Radix MMU driver. Written by Matt Macy. Differential Revision: https://reviews.freebsd.org/D19516 Notes: svn path=/head/; revision=360887
* Add a blocking counter KPI.Mark Johnston2020-02-281-2/+2
| | | | | | | | | | | | | | | | | | | | refcount(9) was recently extended to support waiting on a refcount to drop to zero, as this was needed for a lockless VM object paging-in-progress counter. However, this adds overhead to all uses of refcount(9) and doesn't really match traditional refcounting semantics: once a counter has dropped to zero, the protected object may be freed at any point and it is not safe to dereference the counter. This change removes that extension and instead adds a new set of KPIs, blockcount_*, for use by VM object PIP and busy. Reviewed by: jeff, kib, mjg Tested by: pho Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D23723 Notes: svn path=/head/; revision=358432
* Restore OOM logic on page fault after r357026.Konstantin Belousov2020-01-291-5/+7
| | | | | | | | | | | | | Right now OOM is initiated unconditionally on the page allocation failure, after the wait. Reported by: Mark Millard <marklmi@yahoo.com> Reviewed by: cy, markj Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D23409 Notes: svn path=/head/; revision=357253
* (fault 9/9) Move zero fill into a dedicated function to make the object lockJeff Roberson2020-01-231-33/+38
| | | | | | | | | | state more clear. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D23326 Notes: svn path=/head/; revision=357028
* (fault 8/9) Restructure some code to reduce duplication and simplify flowJeff Roberson2020-01-231-41/+39
| | | | | | | | | | control. Reviewed by: dougm, kib, markj Differential Revision: https://reviews.freebsd.org/D23321 Notes: svn path=/head/; revision=357027
* (fault 7/9) Move fault population and allocation into a dedicated functionJeff Roberson2020-01-231-91/+108
| | | | | | | | Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D23320 Notes: svn path=/head/; revision=357026
* (fault 6/9) Move getpages and associated logic into a dedicated function.Jeff Roberson2020-01-231-96/+107
| | | | | | | | Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D23311 Notes: svn path=/head/; revision=357025
* (fault 5/9) Move the backing_object traversal into a dedicated function.Jeff Roberson2020-01-231-62/+79
| | | | | | | | Reviewed by: dougm, kib, markj Differential Revision: https://reviews.freebsd.org/D23310 Notes: svn path=/head/; revision=357024
* (fault 4/9) Move copy-on-write into a dedicated function.Jeff Roberson2020-01-231-92/+88
| | | | | | | | Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D23304 Notes: svn path=/head/; revision=357023
* (fault 3/9) Move map relookup into a dedicated function.Jeff Roberson2020-01-231-51/+56
| | | | | | | | | | | Add a new VM return code KERN_RESTART which means, deallocate and restart in fault. Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D23303 Notes: svn path=/head/; revision=357022
* (fault 2/9) Move map lookup into a dedicated function.Jeff Roberson2020-01-231-36/+53
| | | | | | | | Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D23302 Notes: svn path=/head/; revision=357021
* (fault 1/9) Move a handful of stack variables into the faultstate.Jeff Roberson2020-01-231-77/+92
| | | | | | | | | | | This additionally fixes a potential bug/pessimization where we could fail to reload the original fault_type on restart. Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D23301 Notes: svn path=/head/; revision=357020
* Move readahead and dropbehind fault functionality into a helper routine forJeff Roberson2020-01-211-40/+58
| | | | | | | | | | clarity. Reviewed by: dougm, kib, markj Differential Revision: https://reviews.freebsd.org/D23282 Notes: svn path=/head/; revision=356936
* Reduce object locking in vm_fault. Once we have an exclusively busied page weJeff Roberson2020-01-201-61/+69
| | | | | | | | | | | no longer need an object lock. This reduces the longest hold times and eliminates some trylock code blocks. Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D23034 Notes: svn path=/head/; revision=356933
* Don't hold the object lock while calling getpages.Jeff Roberson2020-01-191-0/+2
| | | | | | | | | | | | | The vnode pager does not want the object lock held. Moving this out allows further object lock scope reduction in callers. While here add some missing paging in progress calls and an assert. The object handle is now protected explicitly with pip. Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D23033 Notes: svn path=/head/; revision=356902
* Fix a long standing bug that was made worse in r355765. When we are cowing aJeff Roberson2020-01-171-3/+7
| | | | | | | | | | | | | | | | page that was previously mapped read-only it exists in pmap until pmap_enter() returns. However, we held no reference to the original page after the copy was complete. This allowed vm_object_scan_all_shadowed() to collapse an object that still had pages mapped. To resolve this, add another page pointer to the faultstate so we can keep the page xbusy until we're done with pmap_enter(). Handle busy pages in scan_all_shadowed. This is already done in vm_object_collapse_scan(). Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D23155 Notes: svn path=/head/; revision=356822
* Remove page locking for queue operations.Mark Johnston2019-12-281-22/+8
| | | | | | | | | | | | | | | | With the previous reviews, the page lock is no longer required in order to perform queue operations on a page. It is also no longer needed in the page queue scans. This change effectively eliminates remaining uses of the page lock and also the false sharing caused by multiple pages sharing a page lock. Reviewed by: jeff Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D22885 Notes: svn path=/head/; revision=356157
* Don't unnecessarily relock the vm object after sleeps. This results in aJeff Roberson2019-12-241-2/+3
| | | | | | | | | | surprising amount of object contention on loop restarts in fault. Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D22821 Notes: svn path=/head/; revision=356059
* Fix a bug introduced in r356002. Prior versions of this patchset hadJeff Roberson2019-12-221-0/+2
| | | | | | | | | | | vm_page_remove() rather than !vm_page_wired() as the condition for free. When this changed back to wired the busy lock was leaked. Reported by: pho Reviewed by: markj Notes: svn path=/head/; revision=356026
* Make page busy state deterministic on free. Pages must be xbusy whenJeff Roberson2019-12-221-5/+7
| | | | | | | | | | | | | | | | | | | removed from objects including calls to free. Pages must not be xbusy when freed and not on an object. Strengthen assertions to match these expectations. In practice very little code had to change busy handling to meet these rules but we can now make stronger guarantees to busy holders and avoid conditionally dropping busy in free. Refine vm_page_remove() and vm_page_replace() semantics now that we have stronger guarantees about busy state. This removes redundant and potentially problematic code that has proliferated. Discussed with: markj Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D22822 Notes: svn path=/head/; revision=356002
* Move vm_fault busy logic into its own function for clarity and re-use byJeff Roberson2019-12-221-35/+36
| | | | | | | | | | later changes. Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D22820 Notes: svn path=/head/; revision=355997
* Previously we did not support invalid pages in default objects. This meansJeff Roberson2019-12-151-55/+55
| | | | | | | | | | | | | | | | | | that if fault fails to progress and needs to restart the loop it must free the page it is working on and allocate again on restart. Resolve the few places that need to be modified to support this condition and simply deactivate the page. Presently, we only permit this when fault restarts for busy contention. This has an added benefit of removing some object trylocking in this case. While here consolidate some page cleanup logic into fault_page_free() and fault_page_release() to reduce redundant code and automate some teardown. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D22653 Notes: svn path=/head/; revision=355766
* Add a deferred free mechanism for freeing swap space that does not requireJeff Roberson2019-12-151-41/+20
| | | | | | | | | | | | | | | | | | | | | | an exclusive object lock. Previously swap space was freed on a best effort basis when a page that had valid swap was dirtied, thus invalidating the swap copy. This may be done inconsistently and requires the object lock which is not always convenient. Instead, track when swap space is present. The first dirty is responsible for deleting space or setting PGA_SWAP_FREE which will trigger background scans to free the swap space. Simplify the locking in vm_fault_dirty() now that we can reliably identify the first dirty. Discussed with: alc, kib, markj Differential Revision: https://reviews.freebsd.org/D22654 Notes: svn path=/head/; revision=355765
* Store the bottom of the shadow chain in OBJ_ANON object->handle member.Konstantin Belousov2019-12-011-4/+5
| | | | | | | | | | | | | | | | | | | | The handle value is stable for all shadow objects in the inheritance chain. This allows to avoid descending the shadow chain to get to the bottom of it in vm_map_entry_set_vnode_text(), and eliminate corresponding object relocking which appeared to be contending. Change vm_object_allocate_anon() and vm_object_shadow() to handle more of the cred/charge initialization for the new shadow object, in addition to set up the handle. Reported by: jeff Reviewed by: alc (previous version), jeff (previous version) Tested by: pho Sponsored by: The FreeBSD Foundation Differrential revision: https://reviews.freebsd.org/D22541 Notes: svn path=/head/; revision=355270
* Simplify anonymous memory handling with an OBJ_ANON flag. This eliminatesJeff Roberson2019-11-191-3/+2
| | | | | | | | | | | | | | | reudundant complicated checks and additional locking required only for anonymous memory. Introduce vm_object_allocate_anon() to create these objects. DEFAULT and SWAP objects now have the correct settings for non-anonymous consumers and so individual consumers need not modify the default flags to create super-pages and avoid ONEMAPPING/NOSPLIT. Reviewed by: alc, dougm, kib, markj Tested by: pho Differential Revision: https://reviews.freebsd.org/D22119 Notes: svn path=/head/; revision=354869
* Fix a race in release_page().Mark Johnston2019-11-061-1/+5
| | | | | | | | | | | | | | | | | | | Since r354156 we may call release_page() without the page's object lock held, specifically following the page copy during a CoW fault. release_page() must therefore unbusy the page only after scheduling the requeue, to avoid racing with a free of the page. Previously, the object lock prevented this race from occurring. Add some assertions that were helpful in tracking this down. Reported by: pho, syzkaller Tested by: pho Reviewed by: alc, jeff, kib Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D22234 Notes: svn path=/head/; revision=354400
* Replace OBJ_MIGHTBEDIRTY with a system using atomics. Remove the TMPFS_DIRTYJeff Roberson2019-10-291-26/+14
| | | | | | | | | | | | | | flag and use the same system. This enables further fault locking improvements by allowing more faults to proceed with a shared lock. Reviewed by: kib Tested by: pho Differential Revision: https://reviews.freebsd.org/D22116 Notes: svn path=/head/; revision=354158
* Drop the object lock earlier in fault and don't relock it after pmap_enter().Jeff Roberson2019-10-291-5/+12
| | | | | | | | | | | Recent changes in object and page locking have enabled more lock pushdown. Reviewed by: kib Tested by: pho Differential Revision: https://reviews.freebsd.org/D22036 Notes: svn path=/head/; revision=354156
* Modify release_page() to handle a missing fault page.Mark Johnston2019-10-231-5/+7
| | | | | | | | | | | | | | | r353890 introduced a case where we may call release_page() with fs.m == NULL, since the fault handler may now lock the vnode prior to allocating a page for a page-in. Reported by: jhb Reviewed by: kib MFC with: r353890 Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D22120 Notes: svn path=/head/; revision=353964
* Assert that vm_fault_lock_vnode() returns locked saved vnode.Konstantin Belousov2019-10-231-1/+3
| | | | | | | | | | Reviewed by: alc, markj Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential revision: https://reviews.freebsd.org/D22113 Notes: svn path=/head/; revision=353916
* Add VV_VMSIZEVNLOCK flag.Konstantin Belousov2019-10-221-0/+7
| | | | | | | | | | | | | | | The flag specifies that vm_fault() handler should check the vnode' vm_object size under the vnode lock. It is converted into the object' OBJ_SIZEVNLOCK flag in vnode_pager_alloc(). Tested by: pho Reviewed by: markj Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential revision: https://reviews.freebsd.org/D21883 Notes: svn path=/head/; revision=353890
* vm_fault(): extract code to lock the vnode into a helper vn_fault_lock_vnode().Konstantin Belousov2019-10-221-38/+51
| | | | | | | | | | | Tested by: pho Reviewed by: markj Sponsored by: The FreeBSD Foundation MFC after: 2 weeks Differential revision: https://reviews.freebsd.org/D21883 Notes: svn path=/head/; revision=353888
* (5/6) Move the VPO_NOSYNC to PGA_NOSYNC to eliminate the dependency on theJeff Roberson2019-10-151-7/+6
| | | | | | | | | | | | object lock in vm_page_set_validclean(). Reviewed by: kib, markj Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21595 Notes: svn path=/head/; revision=353540
* (4/6) Protect page valid with the busy lock.Jeff Roberson2019-10-151-9/+10
| | | | | | | | | | | | | | Atomics are used for page busy and valid state when the shared busy is held. The details of the locking protocol and valid and dirty synchronization are in the updated vm_page.h comments. Reviewed by: kib, markj Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21594 Notes: svn path=/head/; revision=353539
* (3/6) Add a shared object busy synchronization mechanism that blocks new pageJeff Roberson2019-10-151-5/+11
| | | | | | | | | | | | | | | | | busy acquires while held. This allows code that would need to acquire and release a very large number of page busy locks to use the old mechanism where busy is only checked and not held. This comes at the cost of false positives but never false negatives which the single consumer, vm_fault_soft_fast(), handles. Reviewed by: kib Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21592 Notes: svn path=/head/; revision=353538
* (2/6) Don't release xbusy in vm_page_remove(), defer to vm_page_free_prep().Jeff Roberson2019-10-151-5/+0
| | | | | | | | | | | | This persists busy state across operations like rename and replace. Reviewed by: kib, markj Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21549 Notes: svn path=/head/; revision=353537
* (1/6) Replace busy checks with acquires where it is trival to do so.Jeff Roberson2019-10-151-6/+6
| | | | | | | | | | | | | | This is the first in a series of patches that promotes the page busy field to a first class lock that no longer requires the object lock for consistency. Reviewed by: kib, markj Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21548 Notes: svn path=/head/; revision=353535
* Restore nofaulting operations after r352807Konstantin Belousov2019-10-131-6/+6
| | | | | | | | | | | | | | | The TDP_NOFAULTING flag should be checked in vm_fault(), not in vm_fault_trap(). Otherwise kernel accesses to userspace, like vn_io_fault(), enter vm locking when it should not. Reported and tested by: pho Reviewed by: alc, markj Sponsored by: The FreeBSD Foundation MFC after: 3 days Differential revision: https://reviews.freebsd.org/D21992 Notes: svn path=/head/; revision=353463
* Improve MD page fault handlers.Konstantin Belousov2019-09-271-10/+75
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Centralize calculation of signal and ucode delivered on unhandled page fault in new function vm_fault_trap(). MD trap_pfault() now almost always uses the signal numbers and error codes calculated in consistent MI way. This introduces the protection fault compatibility sysctls to all non-x86 architectures which did not have that bug, but apparently they were already much more wrong in selecting delivered signals on protection violations. Change the delivered signal for accesses to mapped area after the backing object was truncated. According to POSIX description for mmap(2): The system shall always zero-fill any partial page at the end of an object. Further, the system shall never write out any modified portions of the last page of an object which are beyond its end. References within the address range starting at pa and continuing for len bytes to whole pages following the end of an object shall result in delivery of a SIGBUS signal. An implementation may generate SIGBUS signals when a reference would cause an error in the mapped object, such as out-of-space condition. Adjust according to the description, keeping the existing compatibility code for SIGSEGV/SIGBUS on protection failures. For situations where kernel cannot handle page fault due to resource limit enforcement, SIGBUS with a new error code BUS_OBJERR is delivered. Also, provide a new error code SEGV_PKUERR for SIGSEGV on amd64 due to protection key access violation. vm_fault_hold() is renamed to vm_fault(). Fixed some nits in trap_pfault()s like mis-interpreting Mach errors as errnos. Removed unneeded truncations of the fault addresses reported by hardware. PR: 211924 Reviewed by: alc Discussed with: jilles, markj Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D21566 Notes: svn path=/head/; revision=352807
* Revert r352406, which contained changes I didn't intend to commit.Mark Johnston2019-09-161-0/+6
| | | | Notes: svn path=/head/; revision=352407
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-03 16:51:34 +0000