vm_fault: Shoot down multiply mapped COW source page mappings

Reviewed by:	kib, rlibby
Discussed with:	alc
Approved by:	so
Security:	CVE-2021-29626
Security:	FreeBSD-SA-21:08.vm

(cherry picked from commit 71a0b26df14a18b720faaa924bd4e18fcb9638d5)

1 files changed, 27 insertions, 0 deletions

diff --git a/sys/vm/vm_fault.c b/sys/vm/vm_fault.c
index cb21edb23f05..1d4736aa92bc 100644
--- a/sys/vm/vm_fault.c
+++ b/sys/vm/vm_fault.c
@@ -1147,6 +1147,33 @@ readrest:
 					vm_page_unwire(fs.m, PQ_INACTIVE);
 					vm_page_unlock(fs.m);
 				}
+
+				/*
+				 * Typically, the shadow object is either
+				 * private to this address space
+				 * (OBJ_ONEMAPPING) or its pages are read only.
+				 * In the highly unusual case where the pages of
+				 * a shadow object are read/write shared between
+				 * this and other address spaces, we need to
+				 * ensure that any pmap-level mappings to the
+				 * original, copy-on-write page from the backing
+				 * object are removed from those other address
+				 * spaces.
+				 *
+				 * The flag check is racy, but this is
+				 * tolerable: if OBJ_ONEMAPPING is cleared after
+				 * the check, the busy state ensures that new
+				 * mappings of fs.m can't be created.
+				 * pmap_enter() will replace an existing mapping
+				 * in the current address space.  If
+				 * OBJ_ONEMAPPING is set after the check,
+				 * removing mappings will at worse trigger some
+				 * unnecessary page faults.
+				 */
+				vm_page_assert_xbusied(fs.m);
+				if ((fs.first_object->flags & OBJ_ONEMAPPING) == 0)
+					pmap_remove_all(fs.m);
+
 				/*
 				 * We no longer need the old page or object.
 				 */