summaryrefslogtreecommitdiff
path: root/sys/vm/default_pager.c
Commit message (Collapse)AuthorAgeFilesLines
* vm: clean up empty lines in .c and .h filesMateusz Guzik2020-09-011-1/+0
| | | | Notes: svn path=/head/; revision=365074
* SPDX: wrong license.Pedro F. Giffuni2017-11-301-1/+1
| | | | Notes: svn path=/head/; revision=326402
* sys: general adoption of SPDX licensing ID tags.Pedro F. Giffuni2017-11-271-0/+2
| | | | | | | | | | | | | | | | | Mainly focus on files that use BSD 2-Clause license, however the tool I was using misidentified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. No functional change intended. Notes: svn path=/head/; revision=326272
* Provide introductory description of the default pager.Konstantin Belousov2016-12-141-0/+10
| | | | | | | | | Reviewed by: alc Sponsored by: The FreeBSD Foundation MFC after: 1 week Notes: svn path=/head/; revision=310098
* Remove locking around accounting initialization of the default object.Konstantin Belousov2016-12-141-2/+0
| | | | | | | | | | | | | The object is not yet fully constructed and must not be available to other threads. This makes default_pager_alloc() almost identical to swap_pager_alloc_init(). Reviewed by: alc Sponsored by: The FreeBSD Foundation MFC after: 1 week Notes: svn path=/head/; revision=310097
* Tidy up. Mostly, remove or replace stale comments. Most of the commentsAlan Cox2016-12-141-50/+30
| | | | | | | | | | | | | | | | in this file actually described the operation of the swap pager, not the default pager. Given that this is the wrong place to discuss the implementation of the swap pager, it shouldn't come as a surprise that as the swap pager evolved these comments became increasingly stale. In addition, apply some style fixes, like modernizing a few remaining old- style function definitions. Reviewed by: kib, markj MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D8781 Notes: svn path=/head/; revision=310083
* A change to KPI of vm_pager_get_pages() and underlying VOP_GETPAGES().Gleb Smirnoff2015-12-161-7/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | o With new KPI consumers can request contiguous ranges of pages, and unlike before, all pages will be kept busied on return, like it was done before with the 'reqpage' only. Now the reqpage goes away. With new interface it is easier to implement code protected from race conditions. Such arrayed requests for now should be preceeded by a call to vm_pager_haspage() to make sure that request is possible. This could be improved later, making vm_pager_haspage() obsolete. Strenghtening the promises on the business of the array of pages allows us to remove such hacks as swp_pager_free_nrpage() and vm_pager_free_nonreq(). o New KPI accepts two integer pointers that may optionally point at values for read ahead and read behind, that a pager may do, if it can. These pages are completely owned by pager, and not controlled by the caller. This shifts the UFS-specific readahead logic from vm_fault.c, which should be file system agnostic, into vnode_pager.c. It also removes one VOP_BMAP() request per hard fault. Discussed with: kib, alc, jeff, scottl Sponsored by: Nginx, Inc. Sponsored by: Netflix Notes: svn path=/head/; revision=292373
* Place VM objects on the object list when created and never remove them.John Baldwin2015-05-081-0/+1
| | | | | | | | | | | | | | | | | | This is ok since objects come from a NOFREE zone and allows objects to be locked while traversing the object list without triggering a LOR. Ensure that objects on the list are marked DEAD while free or stillborn, and that they have a refcount of zero. This required updating most of the pagers to explicitly mark an object as dead when deallocating it. (Only the vnode pager did this previously.) Differential Revision: https://reviews.freebsd.org/D2423 Reviewed by: alc, kib (earlier version) MFC after: 2 weeks Sponsored by: Norse Corp, Inc. Notes: svn path=/head/; revision=282660
* Fix mis-spelling of bits and types names in theKonstantin Belousov2014-11-041-7/+4
| | | | | | | | | | | | | default_pager_putpages() and swap_pager_putpages(). It is the same fix as was done for vnode_pager_putpages() in r271586. Reviewed by: alc Sponsored by: The FreeBSD Foundation MFC after: 1 week Notes: svn path=/head/; revision=274100
* Switch the vm_object mutex to be a rwlock. This will enable in theAttilio Rao2013-03-091-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | future further optimizations where the vm_object lock will be held in read mode most of the time the page cache resident pool of pages are accessed for reading purposes. The change is mostly mechanical but few notes are reported: * The KPI changes as follow: - VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK() - VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK() - VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK() - VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED() (in order to avoid visibility of implementation details) - The read-mode operations are added: VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(), VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED() * The vm/vm_pager.h namespace pollution avoidance (forcing requiring sys/mutex.h in consumers directly to cater its inlining functions using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h consumers now must include also sys/rwlock.h. * zfs requires a quite convoluted fix to include FreeBSD rwlocks into the compat layer because the name clash between FreeBSD and solaris versions must be avoided. At this purpose zfs redefines the vm_object locking functions directly, isolating the FreeBSD components in specific compat stubs. The KPI results heavilly broken by this commit. Thirdy part ports must be updated accordingly (I can think off-hand of VirtualBox, for example). Sponsored by: EMC / Isilon storage division Reviewed by: jeff Reviewed by: pjd (ZFS specific review) Discussed with: alc Tested by: pho Notes: svn path=/head/; revision=248084
* Replace pointer to "struct uidinfo" with pointer to "struct ucred"Edward Tomasz Napierala2010-12-021-5/+3
| | | | | | | | | | | | in "struct vm_object". This is required to make it possible to account for per-jail swap usage. Reviewed by: kib@ Tested by: pho@ Sponsored by: FreeBSD Foundation Notes: svn path=/head/; revision=216128
* Implement global and per-uid accounting of the anonymous memory. AddKonstantin Belousov2009-06-231-4/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | rlimit RLIMIT_SWAP that limits the amount of swap that may be reserved for the uid. The accounting information (charge) is associated with either map entry, or vm object backing the entry, assuming the object is the first one in the shadow chain and entry does not require COW. Charge is moved from entry to object on allocation of the object, e.g. during the mmap, assuming the object is allocated, or on the first page fault on the entry. It moves back to the entry on forks due to COW setup. The per-entry granularity of accounting makes the charge process fair for processes that change uid during lifetime, and decrements charge for proper uid when region is unmapped. The interface of vm_pager_allocate(9) is extended by adding struct ucred *, that is used to charge appropriate uid when allocation if performed by kernel, e.g. md(4). Several syscalls, among them is fork(2), may now return ENOMEM when global or per-uid limits are enforced. In collaboration with: pho Reviewed by: alc Approved by: re (kensmith) Notes: svn path=/head/; revision=194766
* /* -> /*- for license, minor formatting changesWarner Losh2005-01-071-1/+1
| | | | Notes: svn path=/head/; revision=139825
* Access the swap_pagers' ->putpages() through swappagerops insteadPoul-Henning Kamp2003-08-061-1/+1
| | | | | | | of directly, this is a cleaner way to do it. Notes: svn path=/head/; revision=118535
* Use sparse struct initializations for struct pagerops.Poul-Henning Kamp2003-08-051-7/+5
| | | | | | | This makes grepping for which pagers implement which methods easier. Notes: svn path=/head/; revision=118466
* Use __FBSDID().David E. O'Brien2003-06-111-2/+3
| | | | Notes: svn path=/head/; revision=116226
* Remove references to vm_zone.h and switch over to the new uma API.Jeff Roberson2002-03-201-1/+0
| | | | Notes: svn path=/head/; revision=92748
* Remove __P.Alfred Perlstein2002-03-191-8/+8
| | | | Notes: svn path=/head/; revision=92727
* - Remove a number of extra newlines that do not belong here according toEivind Eklund2002-03-101-4/+0
| | | | | | | | | | | | style(9) - Minor space adjustment in cases where we have "( ", " )", if(), return(), while(), for(), etc. - Add /* SYMBOL */ after a few #endifs. Reviewed by: alc Notes: svn path=/head/; revision=92029
* With Alfred's permission, remove vm_mtx in favor of a fine-grained approachMatthew Dillon2001-07-041-0/+1
| | | | | | | | | | | | (this commit is just the first stage). Also add various GIANT_ macros to formalize the removal of Giant, making it easy to test in a more piecemeal fashion. These macros will allow us to test fine-grained locks to a degree before removing Giant, and also after, and to remove Giant in a piecemeal fashion via sysctl's on those subsystems which the authors believe can operate without Giant. Notes: svn path=/head/; revision=79224
* Introduce a global lock for the vm subsystem (vm_mtx).Alfred Perlstein2001-05-191-0/+2
| | | | | | | | | | | | | | | | | | | | | | vm_mtx does not recurse and is required for most low level vm operations. faults can not be taken without holding Giant. Memory subsystems can now call the base page allocators safely. Almost all atomic ops were removed as they are covered under the vm mutex. Alpha and ia64 now need to catch up to i386's trap handlers. FFS and NFS have been tested, other filesystems will need minor changes (grabbing the vm lock when twiddling page properties). Reviewed (partially) by: jake, jhb Notes: svn path=/head/; revision=76827
* - If swap metadata does not fit into the KVM, reduce the number ofSeigo Tanimura2000-12-131-0/+1
| | | | | | | | | | | | | | | | | | struct swblock entries by dividing the number of the entries by 2 until the swap metadata fits. - Reject swapon(2) upon failure of swap_zone allocation. This is just a temporary fix. Better solutions include: (suggested by: dillon) o reserving swap in SWAP_META_PAGES chunks, and o swapping the swblock structures themselves. Reviewed by: alfred, dillon Notes: svn path=/head/; revision=69972
* Revert spelling mistake I made in the previous commitPhilippe Charnier2000-03-271-1/+1
| | | | | | | Requested by: Alan and Bruce Notes: svn path=/head/; revision=58705
* SpellingPhilippe Charnier2000-03-261-1/+1
| | | | Notes: svn path=/head/; revision=58634
* Remove unused #include's.Alan Cox1999-11-071-2/+0
| | | | | | | Submitted by: phk Notes: svn path=/head/; revision=52973
* The functions declared by this header file no longer exist.Alan Cox1999-11-071-1/+0
| | | | | | | Submitted by: phk (in part) Notes: svn path=/head/; revision=52960
* useracc() the prequel:Poul-Henning Kamp1999-10-291-1/+0
| | | | | | | | | | | | | | Merge the contents (less some trivial bordering the silly comments) of <vm/vm_prot.h> and <vm/vm_inherit.h> into <vm/vm.h>. This puts the #defines for the vm_inherit_t and vm_prot_t types next to their typedefs. This paves the road for the commit to follow shortly: change useracc() to use VM_PROT_{READ|WRITE} rather than B_{READ|WRITE} as argument. Notes: svn path=/head/; revision=52635
* $Id$ -> $FreeBSD$Peter Wemm1999-08-281-1/+1
| | | | Notes: svn path=/head/; revision=50477
* vm_pager_put_pages() is passed an rcval array to hold per-page returnMatthew Dillon1999-01-241-4/+4
| | | | | | | | | | values. The 'int' return value for the procedure was never used and not well defined in any case when there are mixed errors on pages, so it has been removed. vm_pager_put_pages() and associated vm_pager functions now return void. Notes: svn path=/head/; revision=43129
* The default_pager's interaction with the swap_pager has been reorganized,Matthew Dillon1999-01-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | and the swap_pager has been completely replaced. The new swap pager uses the new blist radix-tree based bitmap allocator for low level swap allocation and deallocation. The new allocator is effectively O(5) while the old one was O(N), and the new allocator allocates all required memory at init time rather then at allocate memory on the fly at run time. Swap metadata is allocated in clusters and stored in a hash table, eliminating linearly allocated structures. Many, many features have been rewritten or added. Swap space is now reallocated on the fly providing a poor-mans auto defragmentation of swap space. Swap space that is no longer needed is freed on a timely basis so no garbage collection is necessary. Swap I/O is marked B_ASYNC and NFS has been fixed to do the right thing with it, so NFS-based paging now has around 10x the performance as it did before ( previously NFS enforced synchronous I/O for paging ). Notes: svn path=/head/; revision=42966
* This is a rather large commit that encompasses the new swapper,Matthew Dillon1999-01-211-41/+41
| | | | | | | | | | | | | changes to the VM system to support the new swapper, VM bug fixes, several VM optimizations, and some additional revamping of the VM code. The specific bug fixes will be documented with additional forced commits. This commit is somewhat rough in regards to code cleanup issues. Reviewed by: "John S. Dyson" <root@dyson.iquest.net>, "David Greenman" <dg@root.com> Notes: svn path=/head/; revision=42957
* Fixed two potentially serious classes of bugs:David Greenman1998-10-131-4/+4
| | | | | | | | | | | | | | | | | | | 1) The vnode pager wasn't properly tracking the file size due to "size" being page rounded in some cases and not in others. This sometimes resulted in corrupted files. First noticed by Terry Lambert. Fixed by changing the "size" pager_alloc parameter to be a 64bit byte value (as opposed to a 32bit page index) and changing the pagers and their callers to deal with this properly. 2) Fixed a bogus type cast in round_page() and trunc_page() that caused some 64bit offsets and sizes to be scrambled. Removing the cast required adding casts at a few dozen callers. There may be problems with other bogus casts in close-by macros. A quick check seemed to indicate that those were okay, however. Notes: svn path=/head/; revision=40286
* Back out DIAGNOSTIC changes.Eivind Eklund1998-02-061-3/+1
| | | | Notes: svn path=/head/; revision=33134
* Turn DIAGNOSTIC into a new-style option.Eivind Eklund1998-02-041-1/+3
| | | | Notes: svn path=/head/; revision=33108
* Removed unused #includes.Bruce Evans1997-09-011-4/+1
| | | | Notes: svn path=/head/; revision=28992
* Fixed type mismatches for functions with args of type vm_prot_t and/orBruce Evans1997-08-251-6/+3
| | | | | | | | | | | | vm_inherit_t. These types are smaller than ints, so the prototypes should have used the promoted type (int) to match the old-style function definitions. They use just vm_prot_t and/or vm_inherit_t. This depends on gcc features to work. I fixed the definitions since this is easiest. The correct fix may be to change the small types to u_int, to optimize for time instead of space. Notes: svn path=/head/; revision=28751
* Back out part 1 of the MCFH that changed $Id$ to $FreeBSD$. We are notPeter Wemm1997-02-221-1/+1
| | | | | | | ready for it yet. Notes: svn path=/head/; revision=22975
* Make the long-awaited change from $Id$ to $FreeBSD$Jordan K. Hubbard1997-01-141-1/+1
| | | | | | | | | | | This will make a number of things easier in the future, as well as (finally!) avoiding the Id-smashing problem which has plagued developers for so long. Boy, I'm glad we're not using sup anymore. This update would have been insane otherwise. Notes: svn path=/head/; revision=21673
* Various bugfixes/cleanups from me and others:John Dyson1996-06-161-1/+14
| | | | | | | | | | | | | | | 1) Remove potential race conditions on waking up in vm_page_free_wakeup by making sure that it is at splvm(). 2) Fix another bug in vm_map_simplify_entry. 3) Be more complete about converting from default to swap pager when an object grows to be large enough that there can be a problem with data structure allocation under low memory conditions. 4) Make some madvise code more efficient. 5) Added some comments. Notes: svn path=/head/; revision=16409
* Make sure that pageout deadlocks cannot occur. There is a problemJohn Dyson1996-05-291-7/+15
| | | | | | | | | | | | that the datastructures needed to support the swap pager can take enough space to fully deplete system memory, and cause a deadlock. This change keeps large objects from being filled with dirty pages without the appropriate swap pager datastructures. Right now, default objects greater than 1/4 the size of available system memory are converted to swap objects, thereby eliminating the risk of deadlock. Notes: svn path=/head/; revision=15978
* Make the conversion from the default pager to swap pager more robustJohn Dyson1996-05-241-5/+7
| | | | | | | in the face of low memory conditions. Notes: svn path=/head/; revision=15887
* Eliminated many redundant vm_map_lookup operations for vm_mmap.John Dyson1996-01-191-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Speed up for vfs_bio -- addition of a routine bqrelse to greatly diminish overhead for merged cache. Efficiency improvement for vfs_cluster. It used to do alot of redundant calls to cluster_rbuild. Correct the ordering for vrele of .text and release of credentials. Use the selective tlb update for 486/586/P6. Numerous fixes to the size of objects allocated for files. Additionally, fixes in the various pagers. Fixes for proper positioning of vnode_pager_setsize in msdosfs and ext2fs. Fixes in the swap pager for exhausted resources. The pageout code will not as readily thrash. Change the page queue flags (PG_ACTIVE, PG_INACTIVE, PG_FREE, PG_CACHE) into page queue indices (PQ_ACTIVE, PQ_INACTIVE, PQ_FREE, PQ_CACHE), thereby improving efficiency of several routines. Eliminate even more unnecessary vm_page_protect operations. Significantly speed up process forks. Make vm_object_page_clean more efficient, thereby eliminating the pause that happens every 30seconds. Make sequential clustered writes B_ASYNC instead of B_DELWRI even in the case of filesystems mounted async. Fix a panic with busy pages when write clustering is done for non-VMIO buffers. Notes: svn path=/head/; revision=13490
* Another mega commit to staticize things.Poul-Henning Kamp1995-12-141-6/+14
| | | | Notes: svn path=/head/; revision=12820
* Changes to support 1Tb filesizes. Pages are now named by anJohn Dyson1995-12-111-4/+4
| | | | | | | (object,index) pair instead of (object,offset) pair. Notes: svn path=/head/; revision=12767
* Untangled the vm.h include file spaghetti.David Greenman1995-12-071-2/+6
| | | | Notes: svn path=/head/; revision=12662
* Added a copyright to this file.David Greenman1995-07-131-0/+33
| | | | Notes: svn path=/head/; revision=9514
* Oops, forgot to add the "default" pager files...David Greenman1995-07-131-0/+100
NOTE: libkvm, w, ps, 'top', and any other utility which depends on struct proc or any VM system structure will have to be rebuilt!!! Much needed overhaul of the VM system. Included in this first round of changes: 1) Improved pager interfaces: init, alloc, dealloc, getpages, putpages, haspage, and sync operations are supported. The haspage interface now provides information about clusterability. All pager routines now take struct vm_object's instead of "pagers". 2) Improved data structures. In the previous paradigm, there is constant confusion caused by pagers being both a data structure ("allocate a pager") and a collection of routines. The idea of a pager structure has escentially been eliminated. Objects now have types, and this type is used to index the appropriate pager. In most cases, items in the pager structure were duplicated in the object data structure and thus were unnecessary. In the few cases that remained, a un_pager structure union was created in the object to contain these items. 3) Because of the cleanup of #1 & #2, a lot of unnecessary layering can now be removed. For instance, vm_object_enter(), vm_object_lookup(), vm_object_remove(), and the associated object hash list were some of the things that were removed. 4) simple_lock's removed. Discussion with several people reveals that the SMP locking primitives used in the VM system aren't likely the mechanism that we'll be adopting. Even if it were, the locking that was in the code was very inadequate and would have to be mostly re-done anyway. The locking in a uni-processor kernel was a no-op but went a long way toward making the code difficult to read and debug. 5) Places that attempted to kludge-up the fact that we don't have kernel thread support have been fixed to reflect the reality that we are really dealing with processes, not threads. The VM system didn't have complete thread support, so the comments and mis-named routines were just wrong. We now use tsleep and wakeup directly in the lock routines, for instance. 6) Where appropriate, the pagers have been improved, especially in the pager_alloc routines. Most of the pager_allocs have been rewritten and are now faster and easier to maintain. 7) The pagedaemon pageout clustering algorithm has been rewritten and now tries harder to output an even number of pages before and after the requested page. This is sort of the reverse of the ideal pagein algorithm and should provide better overall performance. 8) Unnecessary (incorrect) casts to caddr_t in calls to tsleep & wakeup have been removed. Some other unnecessary casts have also been removed. 9) Some almost useless debugging code removed. 10) Terminology of shadow objects vs. backing objects straightened out. The fact that the vm_object data structure escentially had this backwards really confused things. The use of "shadow" and "backing object" throughout the code is now internally consistent and correct in the Mach terminology. 11) Several minor bug fixes, including one in the vm daemon that caused 0 RSS objects to not get purged as intended. 12) A "default pager" has now been created which cleans up the transition of objects to the "swap" type. The previous checks throughout the code for swp->pg_data != NULL were really ugly. This change also provides the rudiments for future backing of "anonymous" memory by something other than the swap pager (via the vnode pager, for example), and it allows the decision about which of these pagers to use to be made dynamically (although will need some additional decision code to do this, of course). 13) (dyson) MAP_COPY has been deprecated and the corresponding "copy object" code has been removed. MAP_COPY was undocumented and non- standard. It was furthermore broken in several ways which caused its behavior to degrade to MAP_PRIVATE. Binaries that use MAP_COPY will continue to work correctly, but via the slightly different semantics of MAP_PRIVATE. 14) (dyson) Sharing maps have been removed. It's marginal usefulness in a threads design can be worked around in other ways. Both #12 and #13 were done to simplify the code and improve readability and maintain- ability. (As were most all of these changes) TODO: 1) Rewrite most of the vnode pager to use VOP_GETPAGES/PUTPAGES. Doing this will reduce the vnode pager to a mere fraction of its current size. 2) Rewrite vm_fault and the swap/vnode pagers to use the clustering information provided by the new haspage pager interface. This will substantially reduce the overhead by eliminating a large number of VOP_BMAP() calls. The VOP_BMAP() filesystem interface should be improved to provide both a "behind" and "ahead" indication of contiguousness. 3) Implement the extended features of pager_haspage in swap_pager_haspage(). It currently just says 0 pages ahead/behind. 4) Re-implement the swap device (swstrategy) in a more elegant way, perhaps via a much more general mechanism that could also be used for disk striping of regular filesystems. 5) Do something to improve the architecture of vm_object_collapse(). The fact that it makes calls into the swap pager and knows too much about how the swap pager operates really bothers me. It also doesn't allow for collapsing of non-swap pager objects ("unnamed" objects backed by other pagers). Notes: svn path=/head/; revision=9513
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-06 22:41:51 +0000