summaryrefslogtreecommitdiff
path: root/sys/dev/hwpmc
Commit message (Collapse)AuthorAgeFilesLines
* [PowerPC] hwpmc: add support for POWER8/9 PMCsLeandro Lupori2020-11-057-1027/+1077
| | | | | | | | | | | | | | | This change adds support for POWER8 and POWER9 PMCs (bare metal and pseries). All PowerISA 2.07B non-random events are supported. Implementation was based on that of PPC970. Reviewed by: jhibbits Sponsored by: Eldorado Research Institute (eldorado.org.br) Differential Revision: https://reviews.freebsd.org/D26110 Notes: svn path=/head/; revision=367390
* [PowerPC] Make PPC 970 PMC SPRs the standard onesLeandro Lupori2020-11-052-74/+77
| | | | | | | | | | | | | | And add a _74XX suffix to 74XX SPRs. This is a preparation for adding support to POWER8/9 PMCs, which have most SPRs equal to 970 ones. Reviewed by: jhibbits Sponsored by: Eldorado Research Institute (eldorado.org.br) Differential Revision: https://reviews.freebsd.org/D26532 Notes: svn path=/head/; revision=367383
* cache: drop the always curthread argument from reverse lookup routinesMateusz Guzik2020-08-241-1/+1
| | | | | | | | | Note VOP_VPTOCNP keeps getting it as temporary compatibility for zfs. Tested by: pho Notes: svn path=/head/; revision=364633
* Add support for Cortex-A76/Neoverse-N1 to hwpmcAndrew Turner2020-08-122-11/+56
| | | | | | | | | | | | | | This adds support for the Cortex-A76 and Neoverse-N1 PMU counters to pmc. While here add more PMCR_IDCODE values and check the implementers code is correct before setting the PMU type. Reviewed by: bz, emaste (looks reasonable to me) Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D25959 Notes: svn path=/head/; revision=364153
* Remove free_domain() and uma_zfree_domain().Mark Johnston2020-08-041-7/+7
| | | | | | | | | | | | | | These functions were introduced before UMA started ensuring that freed memory gets placed in domain-local caches. They no longer serve any purpose since UMA now provides their functionality by default. Remove them to simplyify the kernel memory allocator interfaces a bit. Reviewed by: cem, kib Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D25937 Notes: svn path=/head/; revision=363834
* libpmc: Use known pmc_cpuid buffer sizeRyan Moeller2020-07-281-1/+1
| | | | | | | | | | | | | | | | | Use the existing PMC_CPUID_LEN to size pmc_cpuid in the kernel and various buffers for reading it in libpmc. This avoids some extra syscalls and malloc/frees. While in here, use strlcpy to copy a user-provided cpuid string instead of memcpy, to make sure we terminate the buffer. Reviewed by: mav MFC after: 1 week Sponsored by: iXsystems, Inc. Differential Revision: https://reviews.freebsd.org/D25679 Notes: svn path=/head/; revision=363630
* hwpmc: Always set pmc_cpuid to somethingRyan Moeller2020-07-142-12/+16
| | | | | | | | | | | | | | | pmc_cpuid was uninitialized for most AMD processor families. We can still populate this string for unimplemented families. Also added a CPUID_TO_STEPPING macro and converted existing code to use it. Reviewed by: mav MFC after: 2 weeks Sponsored by: iXsystems, Inc. Differential Revision: https://reviews.freebsd.org/D25673 Notes: svn path=/head/; revision=363206
* Add stepping to the kern.hwpmc.cpuid string on x86.Alexander Motin2020-07-142-7/+8
| | | | | | | | | | | | It follows the equivalent Linux change to be able to differentiate skylakex and cascadelakex, sharing the same model but not stepping. This fixes skylakex handling broken by r363144. MFC after: 6 days Notes: svn path=/head/; revision=363188
* Add family 0x5F (Denverton) to PMC_CPU_INTEL_ATOM_GOLDMONT.Alexander Motin2020-04-241-0/+1
| | | | | | | | | According to the 325462-071US document, they should be the same. MFC after: 1 week Notes: svn path=/head/; revision=360266
* Add hwpmc support for Intel Atom Goldmont microarchitectureMarcin Wojtas2020-04-061-0/+6
| | | | | | | | | | | | | | | Recognize new micro-architecture in hwpmc_intel driver. Based on Intel document 325462-071US. Tested with tools/test/hwpmc/pmctest.py on Atom E3930 SoC. Submitted by: Dawid Gorecki <dgr@semihalf.com> Reviewed by: kib Obtained from: Semihalf Sponsored by: Stormshield Differential Revision: https://reviews.freebsd.org/D24310 Notes: svn path=/head/; revision=359667
* Remove the "config" taskqgroup and its KPIs.Mark Johnston2020-03-301-12/+9
| | | | | | | | | | | Equivalent functionality is already provided by taskqueue(9), just use that instead. MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=359438
* Fix accounting of hwpmc's thread descriptor freelist.Mark Johnston2020-03-301-2/+3
| | | | | | | | MFC after: 1 week Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=359437
* Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)Pawel Biernacki2020-02-261-3/+5
| | | | | | | | | | | | | | | | | | | r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are still not MPSAFE (or already are but aren’t properly marked). Use it in preparation for a general review of all nodes. This is non-functional change that adds annotations to SYSCTL_NODE and SYSCTL_PROC nodes using one of the soon-to-be-required flags. Mark all obvious cases as MPSAFE. All entries that haven't been marked as MPSAFE before are by default marked as NEEDGIANT Approved by: kib (mentor, blanket) Commented by: kib, gallatin, melifaro Differential Revision: https://reviews.freebsd.org/D23718 Notes: svn path=/head/; revision=358333
* pmc: Add Hygon Dhyana support.Konstantin Belousov2020-02-072-2/+7
| | | | | | | | | | | | | To make the PMC tool pmcstat working properly on Hygon platform, add support for Hygon Dhyana family 18h by using the PMC initialization code path of AMD family 17h. Submitted by: Pu Wen <puwen@hygon.cn> MFC after: 1 week Differential revision: https://reviews.freebsd.org/D23562 Notes: svn path=/head/; revision=357670
* Remove sparc64 kernel supportWarner Losh2020-02-031-68/+0
| | | | | | | | | Remove all sparc64 specific files Remove all sparc64 ifdefs Removee indireeect sparc64 ifdefs Notes: svn path=/head/; revision=357455
* Add x86 msr tweak KPI.Konstantin Belousov2019-11-181-11/+4
| | | | | | | | | | | | Use the KPI to tweak MSRs in mitigation code. Reviewed by: markj, scottl Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D22431 Notes: svn path=/head/; revision=354828
* hwpmc : fix AMD perf counter MSR accessAndrew Gallatin2019-11-072-7/+52
| | | | | | | | | | | | | | | | | | | | | | | | - amd_intr() does not account for the offset (0x200) in the counter MSR address and ends up accessing invalid regions while reading counter value after the 4th counter (0xC001000[8,9,..]) and erroneously updates the counter values for counters [1-4]. - amd_intr() should only check core pmcs for interrupts since other types of pmcs (L3,DF) cannot generate interrupts. - fix pmc NMI's being ignored due to NMI latency on newer AMD processors Note that this fixes a kernel panic due to GPFs accessing MSRs on higher core count AMD cpus (seen on both Rome 7502P, and Threadripper 2990WX 32-core CPUs) Discussed with: markj Submitted by: Shreyank Amartya Differential Revision: https://reviews.freebsd.org/D21553 Notes: svn path=/head/; revision=354470
* Use a lowercase name for arm64 special registers so they don't conflictAndrew Turner2019-10-301-14/+14
| | | | | | | | | with macros of the same name. Sponsored by: DARPA, AFRL Notes: svn path=/head/; revision=354177
* Fix sample check in hwpmcMatt Macy2019-10-131-1/+1
| | | | | | | | | | | | | Don't drop samples with callchain pending Tested by: mjg@ Submitted by: Rajeeb Barman at amd.com MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D17011 Notes: svn path=/head/; revision=353483
* Define macro VM_MAP_ENTRY_FOREACH for enumerating the entries in a vm_map.Doug Moore2019-10-081-2/+2
| | | | | | | | | | | | | | | | | In case the implementation ever changes from using a chain of next pointers, then changing the macro definition will be necessary, but changing all the files that iterate over vm_map entries will not. Drop a counter in vm_object.c that would have an effect only if the vm_map entry count was wrong. Discussed with: alc Reviewed by: markj Tested by: pho (earlier version) Differential Revision: https://reviews.freebsd.org/D21882 Notes: svn path=/head/; revision=353298
* Add support for BERI statcounters.Ruslan Bukin2019-09-183-0/+711
| | | | | | | | | | | | | | | | | | | | BERI stands for Bluespec Extensible RISC Implementation, based on MIPS. BERI has not implemented standard MIPS perfomance monitoring counters, instead it provides statistical counters. BERI statcounters have a several limitations: - They can't be written - They don't support start/stop operation - None of hardware interrupt is provided on a counter overflow. So make it separate to hwpmc_mips module and support process/system counting mode only. Sponsored by: DARPA, AFRL Notes: svn path=/head/; revision=352487
* Don't refer to the cpu variable in a KASSERT before initializing it.Ian Lepore2019-06-061-1/+1
| | | | Notes: svn path=/head/; revision=348740
* hwpmc_intel: List all Silvermont ids.Konstantin Belousov2019-06-031-0/+4
| | | | | | | | | PR: 238310 Based on submission by: Masse Nicolas <nicolas.masse@stormshield.eu> MFC after: 1 week Notes: svn path=/head/; revision=348544
* hwpmc/core: Adopt to upcoming Skylake TSX errata.Konstantin Belousov2019-03-121-0/+25
| | | | | | | | | | | | | | | | | | | The forthcoming microcode update will fix a TSX bug by clobbering PMC3 when TSX instructions are executed (even speculatively). There is an alternate mode where CPU executes all TSX instructions by aborting them, in which case PMC3 is still available to OS. Any code that correctly uses TSX must be ready to handle abort anyway. Since it is believed that FreeBSD population of hwpmc(4) users is significantly larger than the population of TSX users, switch the microcode into TSX abort mode whenever a pmc is allocated, and back to bug avoidance mode when the last pmc is deallocated. Sponsored by: The FreeBSD Foundation MFC after: 1 week Notes: svn path=/head/; revision=345078
* Remove useless version check.Konstantin Belousov2019-03-121-4/+0
| | | | | | | | Sponsored by: The FreeBSD Foundation MFC after: 3 days Notes: svn path=/head/; revision=345074
* hwpmc: Plug memory disclosures from PMC_OP_{GETPMCINFO,GETCPUINFO}.Mark Johnston2019-01-211-1/+2
| | | | | | | | | | | admbugs: 765 Reported by: Vlad Tsyrklevich <vlad@tsyrklevich.net> MFC after: 1 day Security: Kernel memory disclosure Sponsored by: The FreeBSD Foundation Notes: svn path=/head/; revision=343265
* hwpmc: limit wait for user callchain collection to 1 tickMatt Macy2018-11-051-2/+5
| | | | | | | | | | | | | | | | | | | The hwpmc pcpu sample buffer is prone to head of line blocking when waiting for user process to return to user space and collect a pending callchain. If more than one tick has elapsed between the time the sample entry was marked for collection and the time that the hardclock pmc handler runs to copy the records to a larger temporary buffer, mark the sample entry as not in use. This changes reduces the number of samples marked as not valid when collecting under load from ~99.5% to 5-20%. Reported by: mjg@ MFC after: 3 days Notes: svn path=/head/; revision=340149
* Add aditional counter descriptions to AMD 0x17Matt Macy2018-11-042-10/+191
| | | | | | | | | Submitted by: Somalapuram Amaranath MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D17401 Notes: svn path=/head/; revision=340108
* Add malloc_domainset(9) and _domainset variants to other allocator KPIs.Mark Johnston2018-10-302-41/+29
| | | | | | | | | | | | | | | | | | | | | | | | Remove malloc_domain(9) and most other _domain KPIs added in r327900. The new functions allow the caller to specify a general NUMA domain selection policy, rather than specifically requesting an allocation from a specific domain. The latter policy tends to interact poorly with M_WAITOK, resulting in situations where a caller is blocked indefinitely because the specified domain is depleted. Most existing consumers of the _domain KPIs are converted to instead use a DOMAINSET_PREF() policy, in which we fall back to other domains to satisfy the allocation request. This change also defines a set of DOMAINSET_FIXED() policies, which only permit allocations from the specified domain. Discussed with: gallatin, jeff Reported and tested by: pho (previous version) MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D17418 Notes: svn path=/head/; revision=339927
* hwpmc: Refactor sample ring buffer handling to fix racesMatt Macy2018-10-052-127/+108
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactor sample ring buffer ring handling to make it more robust to long running callchain collection handling r338112 introduced a (now fixed) regression that exposed a number of race conditions within the management of the sample buffers. This simplifies the handling and moves the decision to overwrite a callchain sample that has taken too long out of the NMI in to the hardlock handler. With this change the problem no longer shows up as a ring corruption but as the code spending all of its time in callchain collection. - Makes the producer / consumer index incrementing monotonic, making it easier (for me at least) to reason about. - Moves the decision to overwrite a sample from NMI context to interrupt context where we can enforce serialization. - Puts a time limit on waiting to collect a user callchain - putting a bound on head-of-line blocking causing samples to be dropped - Removes the flush routine which was previously needed to purge dangling references to the pmc from the sample buffers but now is only a source of a race condition on unload. Previously one could lock up or crash HEAD by running: pmcstat -S inst_retired.any_p -T and then hitting ^C After this change it is no longer possible. PR: 231793 Reviewed by: markj@ Approved by: re (gjb@) Differential Revision: https://reviews.freebsd.org/D17011 Notes: svn path=/head/; revision=339188
* hwpmc: set default rate if event description lacks one / filter rate against ↵Matt Macy2018-09-141-6/+21
| | | | | | | | | | | | | | | | | | misuse Not all event descriptions have a sample rate (such as inst_retired.any) this will restore the legacy behavior of using 65536 in that case. It also prevents accidental API misuse that could lead to panic. PR: 230985 Reported by: markj Reviewed by: markj Approved by: re (gjb) Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D16958 Notes: svn path=/head/; revision=338677
* Add library and kernel support for AMD Family 17h countersMatt Macy2018-08-141-1/+13
| | | | | | | NB: lacks default sample rate for most counters Notes: svn path=/head/; revision=337735
* Revert r336773: it removed too much.Warner Losh2018-07-272-0/+121
| | | | | | | | | | r336773 removed all things xscale. However, some things xscale are really armv5. Revert that entirely. A more modest removal will follow. Noticed by: andrew@ Notes: svn path=/head/; revision=336783
* Remove xscale supportWarner Losh2018-07-272-121/+0
| | | | | | | | | | | | | The OLD XSCALE stuff hasn't been useful in a while. The original committer (cognet@) was the only one that had boards for it. He's blessed this removal. Newer XSCALE (GUMSTIX) is for hardware that's quite old. After discussion on arm@, it was clear there was no support for keeping it. Differential Review: https://reviews.freebsd.org/D16313 Notes: svn path=/head/; revision=336773
* Use https over http for FreeBSD pagesEitan Adler2018-07-271-1/+1
| | | | Notes: svn path=/head/; revision=336757
* Remove kernel support for armebWarner Losh2018-07-171-3/+0
| | | | | | | | | | Remove all the big-endian arm architectures (ixp425 and ixp435) support in the kernel and associated drivers. Differential Revision: https://reviews.freebsd.org/D16257 Notes: svn path=/head/; revision=336436
* hwpmc: remove hacks to work around incorrect pc_domainMatt Macy2018-07-061-10/+2
| | | | Notes: svn path=/head/; revision=336022
* epoch(9): allow preemptible epochs to composeMatt Macy2018-07-041-24/+27
| | | | | | | | | | | | | | | | | | | | | | | - Add tracker argument to preemptible epochs - Inline epoch read path in kernel and tied modules - Change in_epoch to take an epoch as argument - Simplify tfb_tcp_do_segment to not take a ti_locked argument, there's no longer any benefit to dropping the pcbinfo lock and trying to do so just adds an error prone branchfest to these functions - Remove cases of same function recursion on the epoch as recursing is no longer free. - Remove the the TAILQ_ENTRY and epoch_section from struct thread as the tracker field is now stack or heap allocated as appropriate. Tested by: pho and Limelight Networks Reviewed by: kbowling at llnw dot com Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D16066 Notes: svn path=/head/; revision=335924
* Use the cached curthread reference in pmc_process_interrupt().Mark Johnston2018-06-111-3/+3
| | | | | | | Fix indentation while here. Notes: svn path=/head/; revision=334959
* hwpmc: remove dangling references to hwpmc_xscaleMatt Macy2018-06-081-11/+0
| | | | | | | Reported by: mjg Notes: svn path=/head/; revision=334857
* hwpmc: retire never completed xscale supportMatt Macy2018-06-081-675/+0
| | | | | | | | hwpmc xscale support is not actually functional and the architecture is well past its shelf life. Notes: svn path=/head/; revision=334847
* hwpmc: update files missed by r334827Matt Macy2018-06-084-10/+13
| | | | Notes: svn path=/head/; revision=334845
* hwpmc: fix arm64 INVARIANTS buildMatt Macy2018-06-081-1/+1
| | | | Notes: svn path=/head/; revision=334831
* hwpmc: avoid undefined variable on LINTMatt Macy2018-06-081-2/+3
| | | | Notes: svn path=/head/; revision=334828
* hwpmc: simplify calling convention for hwpmc interrupt handlingMatt Macy2018-06-089-44/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | pmc_process_interrupt takes 5 arguments when only 3 are needed. cpu is always available in curcpu and inuserspace can always be derived from the passed trapframe. While facially a reasonable cleanup this change was motivated by the need to workaround a compiler bug. core2_intr(cpu, tf) -> pmc_process_interrupt(cpu, ring, pmc, tf, inuserspace) -> pmc_add_sample(cpu, ring, pm, tf, inuserspace) In the process of optimizing the tail call the tf pointer was getting clobbered: (kgdb) up at /storage/mmacy/devel/freebsd/sys/dev/hwpmc/hwpmc_mod.c:4709 4709 pmc_save_kernel_callchain(ps->ps_pc, (kgdb) up 1205 error = pmc_process_interrupt(cpu, PMC_HR, pm, tf, resulting in a crash in pmc_save_kernel_callchain. Notes: svn path=/head/; revision=334827
* hwpmc: don't do EMIT64 on constantMatt Macy2018-06-071-1/+2
| | | | Notes: svn path=/head/; revision=334750
* pmc: convert native to jsonl and track TSC value of samplesMatt Macy2018-06-072-47/+79
| | | | | | | | | | | | | | | | | | - add '-j' options to filter to enable converting native pmc log format to json lines format to enable the use of scripts and external tooling % pmc filter -j pmc.log pmc.jsonl - Record the tsc value in sampling interrupts as opposed to recording nanotime when the sample is copied to a global log in hardclock - potentially many milliseconds later. - At initialize record the tsc_freq and the time of day to give us an offset for translating the tsc values in callchain records Notes: svn path=/head/; revision=334749
* hwpmc: don't log pid->name more than onceMatt Macy2018-06-071-3/+3
| | | | Notes: svn path=/head/; revision=334747
* hwpmc: add summary command and further metadata extensionsMatt Macy2018-06-062-3/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | metadata changes: - log pmc sample rate with pmcallocate - log proc flags with thread / process logging to identify user vs kernel threads fixes: - use log cpuid to translate event id to event name Implement rudimentary summary command to track sample counts by thread and process name within a pmc log. % make -j4 buildkernel >& /dev/null & % sudo pmcstat -S unhalted_core_cycles -S llc-misses -O foo sleep 15 % pmc summary foo cpu_clk_unhalted.thread_p_any: idle: 138108207162 clang-6.0: 105336158004 sh: 72340108510 make: 8642012963 kernel: 7754011631 longest_lat_cache.miss: clang-6.0: 87502625 sh: 40901227 make: 5500165 kernel: 3300099 awk: 2000060 % pmc summary -f ~/foo idx: 278 name: cpu_clk_unhalted.thread_p_any rate: 2000003 idle: 69054 clang-6.0: 52668 sh: 36170 make: 4321 kernel: 3877 hwpmc: proc(7445): 3319 awk: 1289 xargs: 357 rand_harvestq: 181 mtree: 102 intr: 53 zfskern: 31 usb: 7 pagedaemon: 4 ntpd: 3 syslogd: 1 acpi_thermal: 1 logger: 1 syncer: 1 snmptrapd: 1 sleep: 1 idx: 17 name: longest_lat_cache.miss rate: 100003 clang-6.0: 875 sh: 409 make: 55 kernel: 33 awk: 20 hwpmc: proc(7445): 14 xargs: 9 idle: 8 intr: 3 zfskern: 2 Notes: svn path=/head/; revision=334701
* hwpmc: log name->pid, name->tid mappingsMatt Macy2018-06-052-16/+145
| | | | | | | | | | | | | | By logging all threads and processes 'pmc filter' can now filter on process or thread name, relieving the user of the burden of determining which tid or pid was which when the sample was taken. % pmc filter -T if_io_tqg -P nginx pmc.log pmc-iflib.log % pmc filter -x -T idle pmc.log pmc-noidle.log Notes: svn path=/head/; revision=334647
generated by cgit v1.2.3 (git 2.25.1) at 2025-08-08 03:59:58 +0000