aboutsummaryrefslogtreecommitdiff
path: root/sys/cddl/dev
Commit message (Collapse)AuthorAgeFilesLines
...
* Use uintptr_t for pointers in stack frames.John Baldwin2020-12-011-4/+4
| | | | | | | | | | | | This catches up to the changes made to struct unwind_state in r364180. Reviewed by: mhorne Obtained from: CheriBSD Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D27360 Notes: svn path=/head/; revision=368242
* Add a kstack_contains() helper function.John Baldwin2020-12-012-22/+10
| | | | | | | | | | | | | This is useful for stack unwinders which need to avoid out-of-bounds reads of a kernel stack which can trigger kernel faults. Reviewed by: kib, markj Obtained from: CheriBSD Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D27356 Notes: svn path=/head/; revision=368240
* malloc: move malloc_type_internal into malloc_typeMateusz Guzik2020-11-061-1/+1
| | | | | | | | | | | | | | | | According to code comments the original motivation was to allow for malloc_type_internal changes without ABI breakage. This can be trivially accomplished by providing spare fields and versioning the struct, as implemented in the patch below. The upshots are one less memory indirection on each alloc and disappearance of mt_zone. Reviewed by: markj Differential Revision: https://reviews.freebsd.org/D27104 Notes: svn path=/head/; revision=367432
* Fix build after r367020Mitchell Horne2020-10-241-3/+3
| | | | | | | | | DTrace also relies on these definitions. Reported by: jenkins Notes: svn path=/head/; revision=367033
* dtrace: fix fbt return probes on RISC-VKristof Provost2020-09-112-7/+13
| | | | | | | | | | | | | | | | Return values are passed in a0, so read it from there. We also pass a1 through to userspace, as the ABI allows small structs to be returned in registers a0/a1. While here read the register values directly from the trapframe rather than rtval, and remove the now unneeded argument from dtrace_invop(). Set fbtp_roffset so that we get the correct return location in arg0. Reviewed by: markj Sponsored by: Axiado Differential Revision: https://reviews.freebsd.org/D26389 Notes: svn path=/head/; revision=365626
* Tighten frame pointer checking in DTrace's amd64 stack unwinder.Mark Johnston2020-09-011-5/+9
| | | | | | | | | | | | Avoid assuming that the kernel was compiled with -fno-omit-frame-pointer. MFC after: 1 week Sponsored by: NetApp, Inc. Sponsored by: Klara, Inc. Notes: svn path=/head/; revision=365048
* Merge OpenZFS support in to HEAD.Matt Macy2020-08-255-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The primary benefit is maintaining a completely shared code base with the community allowing FreeBSD to receive new features sooner and with less effort. I would advise against doing 'zpool upgrade' or creating indispensable pools using new features until this change has had a month+ to soak. Work on merging FreeBSD support in to what was at the time "ZFS on Linux" began in August 2018. I first publicly proposed transitioning FreeBSD to (new) OpenZFS on December 18th, 2018. FreeBSD support in OpenZFS was finally completed in December 2019. A CFT for downstreaming OpenZFS support in to FreeBSD was first issued on July 8th. All issues that were reported have been addressed or, for a couple of less critical matters there are pull requests in progress with OpenZFS. iXsystems has tested and dogfooded extensively internally. The TrueNAS 12 release is based on OpenZFS with some additional features that have not yet made it upstream. Improvements include: project quotas, encrypted datasets, allocation classes, vectorized raidz, vectorized checksums, various command line improvements, zstd compression. Thanks to those who have helped along the way: Ryan Moeller, Allan Jude, Zack Welch, and many others. Sponsored by: iXsystems, Inc. Differential Revision: https://reviews.freebsd.org/D25872 Notes: svn path=/head/; revision=364746
* amd64 pmap: LA57 AKA 5-level pagingKonstantin Belousov2020-08-231-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since LA57 was moved to the main SDM document with revision 072, it seems that we should have a support for it, and silicons are coming. This patch makes pmap support both LA48 and LA57 hardware. The selection of page table level is done at startup, kernel always receives control from loader with 4-level paging. It is not clear how UEFI spec would adapt LA57, for instance it could hand out control in LA57 mode sometimes. To switch from LA48 to LA57 requires turning off long mode, requesting LA57 in CR4, then re-entering long mode. This is somewhat delicate and done in pmap_bootstrap_la57(). AP startup in LA57 mode is much easier, we only need to toggle a bit in CR4 and load right value in CR3. I decided to not change kernel map for now. Single PML5 entry is created that points to the existing kernel_pml4 (KML4Phys) page, and a pml5 entry to create our recursive mapping for vtopte()/vtopde(). This decision is motivated by the fact that we cannot overcommit for KVA, so large space there is unusable until machines start providing wider physical memory addressing. Another reason is that I do not want to break our fragile autotuning, so the KVA expansion is not included into this first step. Nice side effect is that minidumps are compatible. On the other hand, (very) large address space is definitely immediately useful for some userspace applications. For userspace, numbering of pte entries (or page table pages) is always done for 5-level structures even if we operate in 4-level mode. The pmap_is_la57() function is added to report the mode of the specified pmap, this is done not to allow simultaneous 4-/5-levels (which is not allowed by hw), but to accomodate for EPT which has separate level control and in principle might not allow 5-leve EPT despite x86 paging supports it. Anyway, it does not seems critical to have 5-level EPT support now. Tested by: pho (LA48 hardware) Reviewed by: alc Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D25273 Notes: svn path=/head/; revision=364527
* dtrace/fbt: fix return probe arguments on armAndriy Gapon2020-07-212-7/+13
| | | | | | | | | | | | | | | | | | | | | arg0 should be an offset of the return point within the function, arg1 should be the return value. Previously the return probe had arguments as if for the entry probe. Tested on armv7. andrew noted that the same problem seems to be present on arm64, mips, and riscv. I am not sure if I will get around to fixing those. So, platform users or anyone looking to make a contribution please be aware of this opportunity. Reviewed by: markj MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D25685 Notes: svn path=/head/; revision=363383
* Don't overflow the trap frame when accessing lr or xzr.Andrew Turner2020-07-171-4/+29
| | | | | | | | | | | | | | | | When emulating a load pair or store pair in dtrace on arm64 we need to copy the data between the stack and trap frame. When the registers are either the link register or the zero register we will access memory past the end of the trap frame as these are encoded as registers 30 and 31 respectively while the array they access only has 30 entries. Fix this by creating 2 helper functions to perform the operation with special cases for these registers. Sponsored by: Innovate UK Notes: svn path=/head/; revision=363272
* Fix entering KDB with dtrace-enabled kernel.Ruslan Bukin2020-05-261-0/+6
| | | | | | | | Reviewed by: markj, jhb Differential Revision: https://reviews.freebsd.org/D24018 Notes: svn path=/head/; revision=361533
* Handle non-dtrace-triggered kernel breakpoint traps in mips.John Baldwin2020-04-211-0/+3
| | | | | | | | | | | | | | | | | If DTRACE is enabled at compile time, all kernel breakpoint traps are first given to dtrace to see if they are triggered by a FBT probe. Previously if dtrace didn't recognize the trap, it was silently ignored breaking the handling of other kernel breakpoint traps such as the debug.kdb.enter sysctl. This only returns early from the trap handler if dtrace recognizes the trap and handles it. Submitted by: Nicolò Mazzucato <nicomazz97@gmail.com> Reviewed by: markj Obtained from: CheriBSD Differential Revision: https://reviews.freebsd.org/D24478 Notes: svn path=/head/; revision=360168
* Clear systrace_args_func when systrace probes are disabled.Mark Johnston2020-02-281-6/+6
| | | | | | | | | | | This function pointer is invalidated when systrace.ko is unloaded. Reported by: pho MFC after: 1 week Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=358433
* Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)Pawel Biernacki2020-02-263-4/+9
| | | | | | | | | | | | | | | | | | | r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are still not MPSAFE (or already are but aren’t properly marked). Use it in preparation for a general review of all nodes. This is non-functional change that adds annotations to SYSCTL_NODE and SYSCTL_PROC nodes using one of the soon-to-be-required flags. Mark all obvious cases as MPSAFE. All entries that haven't been marked as MPSAFE before are by default marked as NEEDGIANT Approved by: kib (mentor, blanket) Commented by: kib, gallatin, melifaro Differential Revision: https://reviews.freebsd.org/D23718 Notes: svn path=/head/; revision=358333
* dtrace: add missing CLTFLAG_MPSAFE annotationsMateusz Guzik2020-01-122-4/+6
| | | | Notes: svn path=/head/; revision=356652
* Remove scary-looking printf output that happens when you kldload dtrace onIan Lepore2020-01-091-1/+12
| | | | | | | | arm. Replace it with a comment block explaining why the function is empty on 32-bit arm. Notes: svn path=/head/; revision=356574
* dtrace: 64-bits registers supportMariusz Zaborski2019-10-041-0/+4
| | | | | | | | | | | | | The registers in ilumos and FreeBSD have a different number. In the illumos, last 32-bits register defined is SS an in FreeBSD is GS. While translating register we should comper it to the highest one. PR: 240358 Reported by: lwhsu@ MFC after: 2 weeks Notes: svn path=/head/; revision=353107
* Implement x86 dtrace_invop_(un)init() in C.Mark Johnston2019-09-234-34/+38
| | | | | | | | | | | | There is no reason for these routines to be written in assembly. In the ports of DTrace to other platforms, they are already written in C. No functional change intended. MFC after: 1 week Sponsored by: Netflix Notes: svn path=/head/; revision=352627
* dtrace: 64-bits registers supportMariusz Zaborski2019-06-051-0/+4
| | | | | | | | | | | | | | The registers in ilumos and FreeBSD have a different number. In the illumos, last 32-bits register defined is SS an in FreeBSD is GS. This off-by-one caused the uregs array to returns the wrong 64-bits register on amd64. Reviewed by: markj MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D20363 Notes: svn path=/head/; revision=348705
* powerpc/dtrace: Fix fbt function probing for ELFv2Justin Hibbits2019-05-271-0/+2
| | | | | | | | | | | | | '.' function names exist only in ELFv1. ELFv2 does away with function descriptors, and look more like they do on powerpc(32) and most other platforms, as direct function pointers. Stop blacklisting regular function names in ELFv2. Submitted by: Brandon Bergren Differential Revision: https://reviews.freebsd.org/D20346 Notes: svn path=/head/; revision=348298
* powerpc/dtrace: Actually fix stack tracesJustin Hibbits2019-05-172-22/+25
| | | | | | | | | | | | | Fix stack unwinding such that requesting N stack frames in lockstat will actually give you N frames, not anywhere from 0-3 as had been before. lockstat prints the mutex function instead of the caller as the reported locker, but the stack frame is detailed enough to find the real caller. MFC after: 2 weeks Notes: svn path=/head/; revision=347943
* Remove resolver_qual from DEFINE_IFUNC/DEFINE_UIFUNC macros.Konstantin Belousov2019-05-161-7/+7
| | | | | | | | | | | | | In all practical situations, the resolver visibility is static. Requested by: markj Sponsored by: The FreeBSD Foundation MFC after: 1 week Approved by: so (emaste) Differential revision: https://reviews.freebsd.org/D20281 Notes: svn path=/head/; revision=347895
* powerpc/dtrace: Fix dtrace powerpc asm, and simplify stack walkingJustin Hibbits2019-04-132-42/+22
| | | | | | | | | | | | | | | | | Fix some execution bugs in the dtrace powerpc asm. addme pulls in the carry flag which we don't want, and the result wasn't recorded anyways, so the following beq to check for exit condition wasn't checking the right condition. Simplify the stack walking in dtrace_isa.c, so there's only a single walker that handles both pc and sp. This should make it easier to follow, and any bugfix that may be needed for walking only needs to be made in one place instead of two now. MFC after: 2 weeks Notes: svn path=/head/; revision=346173
* Use an explicit comparison with VM_GUEST_NO.Mark Johnston2019-03-212-2/+2
| | | | | | | | | Reported by: jhb MFC with: r345359 Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=345384
* Don't attempt to measure TSC skew when running as a VM guest.Mark Johnston2019-03-212-0/+6
| | | | | | | | | | | | | It simply doesn't work in general since VCPUs may migrate between physical cores. The approach used to measure skew also doesn't make much sense in a VM. PR: 218452 MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=345359
* Fix a tracepoint lookup race in fasttrap_pid_probe().Mark Johnston2019-02-211-1/+4
| | | | | | | | | | | | | | | | | | | | fasttrap hooks the userspace breakpoint handler; the hook looks up the breakpoint address in a hash table of tracepoints. It is possible for the tracepoint to be removed by a different thread in between the breakpoint trap and the hash table lookup, in which case SIGTRAP gets delivered to the target process. Fix the problem by adding a per-process generation counter that gets incremented when a tracepoint belonging to that process is removed. Then, when a lookup fails, the trapping instruction is restarted if the thread's counter doesn't match that of the process. Reviewed by: cem MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D19273 Notes: svn path=/head/; revision=344452
* dtrace: fix userspace access on boxes with SMAPMateusz Guzik2018-12-132-14/+161
| | | | | | | | | | | | | | dtrace has its own routines which were not updated after SMAP support got implemented. Use ifunc just like for other routines. This in particular fixes ustack(). Reviewed by: markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D18542 Notes: svn path=/head/; revision=342053
* DTrace/powerpc: Fix FBT return probesJustin Hibbits2018-11-211-1/+1
| | | | | | | | | The FBT fuction boundary prober was setting one return probe marker value, but the dtrace handler was expecting another. This causes a hang when tracing return probes. Notes: svn path=/head/; revision=340726
* powerpc/dtrace: Use explicit bit numbers to mask out PSL_EEJustin Hibbits2018-10-211-1/+1
| | | | | | | | | | There seems to be a race in CI, such that dtrace_asm.S might be assembled before the genassym is completed. This causes a build failure when PSL_EE doesn't exist, and is read as 0. Get around this by explicitly specifying the bits in the mask instead. Notes: svn path=/head/; revision=339516
* Various fixes for TLB management on RISC-V.John Baldwin2018-10-151-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Remove the arm64-specific cpu_*cache* and cpu_tlb_flush* functions. Instead, add RISC-V specific inline functions in cpufunc.h for the fence.i and sfence.vma instructions. - Catch up to changes in the arm64 pmap and remove all the cpu_dcache_* calls, pmap_is_current, pmap_l3_valid_cacheable, and PTE_NEXT bits from pmap. - Remove references to the unimplemented riscv_setttb(). - Remove unused cpu_nullop. - Add a link to the SBI doc to sbi.h. - Add support for a 4th argument in SBI calls. It's not documented but it seems implied for the asid argument to SBI_REMOVE_SFENCE_VMA_ASID. - Pass the arguments from sbi_remote_sfence*() to the SEE. BBL ignores them so this is just cosmetic. - Flush icaches on other CPUs when they resume from kdb in case the debugger wrote any breakpoints while the CPUs were paused in the IPI_STOP handler. - Add SMP vs UP versions of pmap_invalidate_* similar to amd64. The UP versions just use simple fences. The SMP versions use the sbi_remove_sfence*() functions to perform TLB shootdowns. Since we don't have a valid pm_active field in the riscv pmap, just IPI all CPUs for all invalidations for now. - Remove an extraneous TLB flush from the end of pmap_bootstrap(). - Don't do a TLB flush when writing new mappings in pmap_enter(), only if modifying an existing mapping. Note that for COW faults a TLB flush is only performed after explicitly clearing the old mapping as is done in other pmaps. - Sync the i-cache on all harts before updating the PTE for executable mappings in pmap_enter and pmap_enter_quick. Previously the i-cache was only sync'd after updating the PTE in pmap_enter. - Use sbi_remote_fence() instead of smp_rendezvous in pmap_sync_icache(). Reviewed by: markj Approved by: re (gjb, kib) Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D17414 Notes: svn path=/head/; revision=339367
* Add support for 'C'-compressed ISA extension to DTrace FBT provider.Ruslan Bukin2018-09-032-24/+134
| | | | | | | | Approved by: re (kib) Sponsored by: DARPA, AFRL Notes: svn path=/head/; revision=338444
* Fix the hash table lookup in fbt_destroy().Mark Johnston2018-09-021-1/+1
| | | | | | | | | Reported and tested by: pho Approved by: re (kib) X-MFC with: r338359 Notes: svn path=/head/; revision=338430
* Allow multiple FBT probes to share a tracepoint.Mark Johnston2018-08-288-47/+97
| | | | | | | | | | | | | | | | | With GNU ifuncs, multiple FBT probes may correspond to the same instruction. fbt_invop() assumed that this could not happen and would return after the first probe found in the global FBT hash table, which might not be the one that's enabled. Fix the problem on x86 by linking probes that share a tracepoint and having each linked probe fire when the tracepoint is hit. PR: 230846 Approved by: re (gjb) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D16921 Notes: svn path=/head/; revision=338359
* dtrace/powerpc: Correct register indices for non-indexed registers in the ↵Justin Hibbits2018-07-161-7/+7
| | | | | | | | | | trapframe Fix an off-by-one error, LR starts at index 32, not index 33, and the others follow suit. Notes: svn path=/head/; revision=336355
* Revert r326083, it doesn't behave as expected.Justin Hibbits2018-06-031-1/+1
| | | | | | | | | Even though there do appear to be more artificial frames, with 12, stack traces no longer list at all. Revert until a better, more stable value can be determined. Notes: svn path=/head/; revision=334550
* Protect dtrace_getpcstack() from a NULL stack pointer in a trap frameJustin Hibbits2018-05-301-1/+5
| | | | | | | | Found when trying to use lockstat on a POWER9, the stack pointer (r1) could be NULL, and result in a NULL pointer dereference, crashing the kernel. Notes: svn path=/head/; revision=334370
* DTrace aarch64: Avoid calling unwind_frame() in the probe context.Mark Johnston2018-05-121-4/+8
| | | | | | | | | | | | | | | unwind_frame() may be instrumented by FBT, leading to recursion into dtrace_probe(). Manually inline unwind_frame() as we do with stack unwinding code for other architectures. Submitted by: Domagoj Stolfa Reviewed by: manu MFC after: 1 week Sponsored by: DARPA / AFRL Differential Revision: https://reviews.freebsd.org/D15359 Notes: svn path=/head/; revision=333570
* systrace: track it like sdt probesMateusz Guzik2018-04-271-0/+8
| | | | | | | | | | | While here predict false. Note the code is wrong (regardless of this change). Dereference of the pointer can race with module unload. A fix would set the probe to a nop stub instead of NULL. Notes: svn path=/head/; revision=333064
* dtrace: depessimize dtmalloc when dtrace is activeMateusz Guzik2018-04-241-0/+9
| | | | | | | | | | | Each malloc/free was testing dtrace_malloc_enabled and forcing extra reads from the malloc type struct to see if perhaps a dtmalloc probe was on. Treat it like lockstat and sdt: have a global bolean. Notes: svn path=/head/; revision=332901
* lockstat: track lockstat just like sdt probesMateusz Guzik2018-04-241-4/+11
| | | | | | | In particular flip the frequently tested var to bool. Notes: svn path=/head/; revision=332900
* Assert that dtrace_probe() doesn't re-enter itself.Mark Johnston2018-04-101-0/+2
| | | | | | | | | | | | | This helps catch cases where an instrumented function is called while in probe context. Submitted by: Domagoj Stolfa <domagoj.stolfa@gmail.com> MFC after: 2 weeks Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D14863 Notes: svn path=/head/; revision=332364
* Remove very old and unused signal information codes.John Baldwin2018-03-272-0/+2
| | | | | | | | | | | | | These have been supplanted by the MI signal information codes in <sys/signal.h> since 7.0. The FPE_*_TRAP ones were deprecated even earlier in 1999. PR: 226579 (exp-run) Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D14637 Notes: svn path=/head/; revision=331650
* Revert part of r331264: disable interrupts before disabling WP.Mark Johnston2018-03-201-0/+3
| | | | | | | | | | | We might otherwise be preempted, leaving WP disabled while another thread runs on the CPU. Reported by: kib X-MFC with: r331264 Notes: svn path=/head/; revision=331268
* Make use of the KPI added in r331252.Mark Johnston2018-03-201-7/+4
| | | | | | | MFC after: 2 weeks Notes: svn path=/head/; revision=331264
* Rename assym.s to assym.incEd Maste2018-03-206-6/+6
| | | | | | | | | | | | assym is only to be included by other .s files, and should never actually be assembled by itself. Reviewed by: imp, bdrewery (earlier) Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D14180 Notes: svn path=/head/; revision=331254
* Unbreak amd64 FBT after r330539.Mark Johnston2018-03-061-0/+9
| | | | | | | X-MFC with: r330539 Notes: svn path=/head/; revision=330541
* Avoid CPU migration in dtrace_gethrtime() on x86.Mark Johnston2017-12-182-6/+12
| | | | | | | | | | | | | | dtrace_gethrtime() may be called outside of probe context, and in particular, from the DTRACEIOC_BUFSNAP handler. Disable interrupts rather than using sched_pin() to help ensure that we don't call any external functions when in probe context. PR: 218452 MFC after: 1 week Notes: svn path=/head/; revision=326935
* Don't use pcpu_find() to determine if a CPU ID is valid.Mark Johnston2017-11-271-8/+2
| | | | | | | | | This addresses assertion failures after r326218. MFC after: 1 week Notes: svn path=/head/; revision=326286
* PowerPC has 12 artificial frames for the profilerJustin Hibbits2017-11-221-1/+1
| | | | | | | | | | It may need to be different between AIM and Book-E, this was tested only on Book-E (64- and 32-bit) MFC after: 3 weeks Notes: svn path=/head/; revision=326083
* sdt: make all sdt probe sites test one variableMateusz Guzik2017-10-221-0/+8
| | | | | | | | | | | This saves on cache misses at the expense of a slight grow of .text. Note this is a bandaid for lack of hotpatching. Discussed with: markj Notes: svn path=/head/; revision=324869
generated by cgit v1.3 (git 2.53.0) at 2026-04-10 08:28:13 +0000