FreeBSD source tree https://cgit-dev.freebsd.org/src/atom?h=release%2F13.2.0 2022-11-14T19:23:40Z 2022-11-14T19:23:40Z Mark Johnston markj@FreeBSD.org 2021-07-19T20:09:42Z urn:sha1:2b76d0699ed46ef956628c9509ecab56c55d2dde KASAN and KCSAN implement interceptors for various primitive operations that are not instrumented by the compiler. KMSAN requires them as well. Rather than adding new cases for each sanitizer which requires interceptors, implement the following protocol: - When interceptor definitions are required, define SAN_NEEDS_INTERCEPTORS and SANITIZER_INTERCEPTOR_PREFIX. - In headers that declare functions which need to be intercepted by a sanitizer runtime, use SANITIZER_INTERCEPTOR_PREFIX to provide declarations. - When SAN_RUNTIME is defined, do not redefine the names of intercepted functions. This is typically the case in files which implement sanitizer runtimes but is also needed in, for example, files which define ifunc selectors for intercepted operations. MFC after: 2 weeks Sponsored by: The FreeBSD Foundation (cherry picked from commit a90d053b84223a4e5cb65852a9b6193570ab1c7d) 2022-02-11T10:52:55Z Konstantin Belousov kib@FreeBSD.org 2022-02-03T17:46:22Z urn:sha1:d07bca9b93fdb046af15b54f5b689c200ffab446 (cherry picked from commit 9596b349bb57e50a2baec8497ced9f712f08f147) 2022-02-11T10:52:55Z Konstantin Belousov kib@FreeBSD.org 2022-02-03T09:51:36Z urn:sha1:9d1361ef0a3131d27c113893195642e53aea0043 (cherry picked from commit 9c0b759bf9b520537616d026f21a0a98d70acd11) 2022-02-11T10:52:55Z Konstantin Belousov kib@FreeBSD.org 2022-02-03T09:25:03Z urn:sha1:725225a72a2a859762e8237b8202cc605a1dcac7 (cherry picked from commit cbf999e75d61c6efadeed8d32212eefa4dfb3dcb) 2021-11-01T14:16:39Z Mark Johnston markj@FreeBSD.org 2021-03-23T01:44:55Z urn:sha1:bf0986b742abaa493ed572391c798791468c702f Make it easy to define interceptors for new sanitizer runtimes, rather than assuming KCSAN. Lay a bit of groundwork for KASAN and KMSAN. When a sanitizer is compiled in, atomic(9) and bus_space(9) definitions in atomic_san.h are used by default instead of the inline implementations in the platform's atomic.h. These definitions are implemented in the sanitizer runtime, which includes machine/{atomic,bus}.h with SAN_RUNTIME defined to pull in the actual implementations. No functional change intended. Sponsored by: The FreeBSD Foundation (cherry picked from commit 3ead60236fd25ce64fece7ae4a453318ca18c119) 2021-03-15T15:39:11Z Mark Johnston markj@FreeBSD.org 2021-03-08T17:39:06Z urn:sha1:980da033a8686bbe40ffb01435c0803af15b4a65 Other kernel sanitizers (KMSAN, KASAN) require interceptors as well, so put these in a more generic place as a step towards importing the other sanitizers. No functional change intended. Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D29103 (cherry picked from commit 435c7cfb2418fdac48fa53e29e38ef03646b817d) 2020-03-25T23:12:43Z Conrad Meyer cem@FreeBSD.org 2020-03-25T23:12:43Z urn:sha1:ca0ec73c11a7bb9ed409466e514fa5c34b6c84b5 The goal of this change is to make the atomic_load_acq_{8,16}, atomic_testandset{,_acq}_long, and atomic_testandclear_long primitives available in MI-namespace. The second goal is to get this draft out of my local tree, as anything that requires a full tinderbox is a big burden out of tree. MD specifics can be refined individually afterwards. The generic implementations may not be ideal for your architecture; feel free to implement better versions. If no subword_atomic definitions are needed, the include can be removed from your arch's machine/atomic.h. Generic definitions are guarded by defined macros of the same name. To avoid picking up conflicting generic definitions, some macro defines are added to various MD machine/atomic.h to register an existing implementation. Include _atomic_subword.h in arm and arm64 machine/atomic.h. For some odd reason, KCSAN only generates some versions of primitives. Generate the _acq variants of atomic_load.*_8, atomic_load.*_16, and atomic_testandset.*_long. There are other questionably disabled primitives, but I didn't run into them, so I left them alone. KCSAN is only built for amd64 in tinderbox for now. Add atomic_subword implementations of atomic_load_acq_{8,16} implemented using masking and atomic_load_acq_32. Add generic atomic_subword implementations of atomic_testandset_long(), atomic_testandclear_long(), and atomic_testandset_acq_long(), using atomic_fcmpset_long() and atomic_fcmpset_acq_long(). On x86, add atomic_testandset_acq_long as an alias for atomic_testandset_long. Reviewed by: kevans, rlibby (previous versions both) Differential Revision: https://reviews.freebsd.org/D22963 2020-02-28T18:32:36Z Ryan Libby rlibby@FreeBSD.org 2020-02-28T18:32:36Z urn:sha1:6d1a70dd0a4fb6058902b3f153ba9d7eec82e521 Previously the pattern to extract status flags from inline assembly blocks was to use setcc in the block to write the flag to a register. This was suboptimal in a few ways: - It would lead to code like: sete %cl; test %cl; jne, i.e. a flag would just be loaded into a register and then reloaded to a flag. - The setcc would force the block to use an additional register. - If the client code didn't care for the flag value then the setcc would be entirely pointless but could not be eliminated by the optimizer. A more modern inline asm construct (since gcc 6 and clang 9) allows for "flag output operands", where a C variable can be written directly from a flag. The optimizer can then use this to produce direct code where the flag does not take a trip through a register. In practice this makes each affected operation sequence shorter by five bytes of instructions. It's unlikely this has a measurable performance impact. Reviewed by: kib, markj, mjg Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D23869 2019-11-21T11:22:08Z Andrew Turner andrew@FreeBSD.org 2019-11-21T11:22:08Z urn:sha1:849aef496d2ae19961310f4e92f3a0b928732d26 Update the NetBSD Kernel Concurrency Sanitizer (KCSAN) runtime to work in the FreeBSD kernel. It is a useful tool for finding data races between threads executing on different CPUs. This can be enabled by enabling KCSAN in the kernel config, or by using the GENERIC-KCSAN amd64 kernel. It works on amd64 and arm64, however the later needs a compiler change to allow -fsanitize=thread that KCSAN uses. Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D22315 2018-07-06T10:13:42Z Hans Petter Selasky hselasky@FreeBSD.org 2018-07-06T10:13:42Z urn:sha1:a7a7f5b472a8934cbafd5c2989d74ee2514c19f7 SMP systems by extending defined(SMP) to include defined(KLD_MODULE). This is a regression issue after r335873 . Discussed with: mmacy@ Sponsored by: Mellanox Technologies