FreeBSD source tree https://cgit-dev.freebsd.org/src/atom?h=stable%2F13 2025-03-31T18:36:01Z 2025-03-31T18:36:01Z Mark Johnston markj@FreeBSD.org 2025-03-17T19:12:58Z urn:sha1:910535a82a29d71eb3951b2368aef358a207f18d After commit 389a3fa693ef, uma_reclaim_domain(UMA_RECLAIM_DRAIN_CPU) calls uma_zone_reclaim_domain(UMA_RECLAIM_DRAIN_CPU) twice on each zone in addition to globally draining per-CPU caches. This was unintended and is unnecessarily slow; in particular, draining per-CPU caches requires binding to each CPU. Stop draining per-CPU caches when visiting each zone, just do it once in pcpu_cache_drain_safe() to minimize the amount of expensive sched_bind() calls. Fixes: 389a3fa693ef ("uma: Add UMA_ZONE_UNMANAGED") MFC after: 1 week Sponsored by: Klara, Inc. Sponsored by: NetApp, Inc. Reviewed by: gallatin, kib Differential Revision: https://reviews.freebsd.org/D49349 (cherry picked from commit f506d5af50fccc37f5aa9fe090e9a0d5f05506c8) 2023-12-21T13:40:09Z Olivier Certner olce.freebsd@certner.fr 2023-10-13T14:09:51Z urn:sha1:27199e55a4d568659d3a4b9985e736e8e5ee3dfe New function check_align_mask() asserts (under INVARIANTS) that the mask fits in a (signed) integer (see the comment) and that the corresponding alignment is a power of two. Use check_align_mask() in uma_set_align_mask() and also in uma_zcreate() to replace the KASSERT() there (that was checking only for a power of 2). Reviewed by: kib, markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D42263 (cherry picked from commit 87090f5e5a7b927a2ab30878435f6dcba0705a1d) Approved by: markj (mentor) 2023-12-21T13:39:53Z Olivier Certner olce.freebsd@certner.fr 2023-10-13T12:49:11Z urn:sha1:9121202fcb27e981bc4d2a7695fbba463361df93 In uma_set_align_mask(), ensure that the passed value doesn't have its highest bit set, which would lead to problems since keg/zone alignment is internally stored as signed integers. Such big values do not make sense anyway and indicate some programming error. A future commit will introduce checks for this case and other ones. Reviewed by: kib, markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D42262 (cherry picked from commit 3d8f548b9e5772ff6890bdc01f7ba7b76203857d) Approved by: markj (mentor) 2023-12-21T13:39:34Z Olivier Certner olce.freebsd@certner.fr 2023-10-13T12:13:30Z urn:sha1:4e00940e9f1f7dc1210a094d916ef930ff447e4a Having a special value of -1 that is resolved internally to 'uma_align_cache' provides no significant advantages and prevents changing that variable to an unsigned type, which is natural for an alignment mask. So suppress it and replace its use with a call to uma_get_align_mask(). The small overhead of the added function call is irrelevant since UMA_ALIGN_CACHE is only used when creating new zones, which is not performance critical. Reviewed by: markj, kib Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D42259 (cherry picked from commit e557eafe7233f8231c1f5f5b098e4bab8e818645) Approved by: markj (mentor) 2023-12-21T13:39:26Z Olivier Certner olce.freebsd@certner.fr 2023-10-13T09:52:28Z urn:sha1:56c53cc8fb3edaed82678440da5cf8e0dc482d03 Create the uma_get_cache_align_mask() accessor and put it in a separate private header so as to minimize namespace pollution in header/source files that need only this function and not the whole 'uma.h' header. Make sure the accessors have '_mask' as a suffix, so that callers are aware that the real alignment is the power of two that is the mask plus one. Rename the stem to something more explicit. Rename uma_set_cache_align_mask()'s single parameter to 'mask'. Hide 'uma_align_cache' to ensure that it cannot be set in any other way then by a call to uma_set_cache_align_mask(), which will perform sanity checks in a further commit. While here, rename it to 'uma_cache_align_mask'. This is also in preparation for some further changes, such as improving the sanity checks, eliminating internal resolving of UMA_ALIGN_CACHE and changing the type of the 'uma_cache_align_mask' variable. Reviewed by: markj, kib Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D42258 (cherry picked from commit dc8f7692fd1de628814f4eaf4a233dccf4c92199) Approved by: markj (mentor) 2023-08-23T17:43:25Z Warner Losh imp@FreeBSD.org 2023-08-22T01:31:27Z urn:sha1:3bc80996974a61a4223eae4c1ccd47b6ee32a48a Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/ Similar commit in current: (cherry picked from commit 685dc743dc3b) 2023-07-25T15:13:49Z Warner Losh imp@FreeBSD.org 2023-05-10T15:40:58Z urn:sha1:caa41f641755c935b036e17440a3b49329c904ed The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch up to that fact and revert to their recommended match of BSD-2-Clause. Discussed with: pfg MFC After: 3 days Sponsored by: Netflix (cherry picked from commit 4d846d260e2b9a3d4d0a701462568268cbfe7a5b) 2022-10-26T00:35:31Z Mark Johnston markj@FreeBSD.org 2022-10-19T18:27:49Z urn:sha1:5019af2e87438bb51c8d345e496418841b680e88 Items allocated from cache zones cannot usefully be protected by memguard. PR: 267151 Reported and tested by: pho (cherry picked from commit 2dba2288aa364c36c756770ac77590816b15d954) 2022-07-25T13:43:44Z Mark Johnston markj@FreeBSD.org 2022-07-11T19:45:36Z urn:sha1:adc4dcada3d7a975bb7d1745c2664a85d1b06da2 uma_timeout() has several responsibilities; it visits every UMA zone and as of recently will drain underutilized caches, so is rather expensive (>1ms in some cases). Currently it is executed by softclock threads and so will preempt most other CPU activity. None of this work requires a high scheduling priority, though, so defer it to a taskqueue so as to avoid stalling higher-priority work. Reviewed by: rlibby, alc, mav, kib Sponsored by: The FreeBSD Foundation (cherry picked from commit 93cd28ea82bb7b5e9489c8febfe1ae7ed2c044fb) 2022-04-13T12:10:35Z Mark Johnston markj@FreeBSD.org 2022-03-30T19:42:18Z urn:sha1:f9677b7e742c95bdb94ee5da8a4fba14c9010a6e The limit accounting in UMA does not tolerate this. Sponsored by: The FreeBSD Foundation (cherry picked from commit d53927b0bae45bf5b4b206b2dddf37bab319a1b3)