aboutsummaryrefslogtreecommitdiff
path: root/sys/powerpc/include
Commit message (Collapse)AuthorAgeFilesLines
* [PowerPC] Floating-point exception trap followupBrandon Bergren2020-12-131-4/+4
| | | | | | | | | | | | | | * Fix incorrect operation on 32-bit caused by incorrectly-sized storage for a temporary FPSCR. * Fix several whitespace problems. * Don't try to enable VSX during cleanup_fpscr(). Reviewed by: alfredo, jhibbits (earlier version) Sponsored by: Tag1 Consulting, Inc. Differential Revision: https://reviews.freebsd.org/D27453 Notes: svn path=/head/; revision=368604
* Add CFI start/end proc directives to arm64, i386, and ppcConrad Meyer2020-12-051-4/+12
| | | | | | | | | | | | | | Follow-up to r353959 and r368070: do the same for other architectures. arm32 already seems to use its own .fnstart/.fnend directives, which appear to be ARM-specific variants of the same thing. Likewise, MIPS uses .frame directives. Reviewed by: arichardson Differential Revision: https://reviews.freebsd.org/D27387 Notes: svn path=/head/; revision=368354
* [PowerPC64LE] Fix LE VSX/fpr interopBrandon Bergren2020-12-031-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | In the PCB struct, we need to match the VSX register file layout correctly, as the VSRs shadow the FPRs. In LE, we need to have a dword of padding before the fprs so they end up on the correct side, as the struct may be manipulated by either the FP routines or the VSX routines. Additionally, when saving and restoring fprs, we need to explicitly target the fpr union member so it gets offset correctly on LE. Fixes weirdness with FP registers in VSX-using programs (A FPR that was saved by the FP routines but restored by the VSX routines was becoming 0 due to being loaded to the wrong side of the VSR.) Original patch by jhibbits. Reviewed by: jhibbits Differential Revision: https://reviews.freebsd.org/D27431 Notes: svn path=/head/; revision=368290
* [PowerPC] Don't overwrite vm.pmap sysctl nodeLeandro Lupori2020-11-171-0/+1
| | | | | | | | | | | | | | | | | | After r367417, both mmu_oea64 and mmu_radix were defining the vm.pmap sysctl node, resulting in the later definition hiding the properties of the previous one. Avoid this issue by defining vm.pmap in a common source file and declaring it where needed. This change also standardizes the tunable name used to enable superpages and change its default to disabled on radix MMU, because it still has some issues with superpages. Reviewed by: bdragon, jhibbits Sponsored by: Eldorado Research Institute (eldorado.org.br) Differential Revision: https://reviews.freebsd.org/D27156 Notes: svn path=/head/; revision=367758
* Make powerpc use MAXARGS (defined as 8) instead of hardcoding '10'.Edward Tomasz Napierala2020-11-061-1/+2
| | | | | | | | | | | | This brings its 'struct syscall_args' in sync with other architectures. Reviewed by: bdragon, jhibbits MFC after: 2 weeks Sponsored by: EPSRC Differential Revision: https://reviews.freebsd.org/D26605 Notes: svn path=/head/; revision=367430
* Implement superpages for PowerPC64 (HPT)Leandro Lupori2020-11-064-12/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | This change adds support for transparent superpages for PowerPC64 systems using Hashed Page Tables (HPT). All pmap operations are supported. The changes were inspired by RISC-V implementation of superpages, by @markj (r344106), but heavily adapted to fit PPC64 HPT architecture and existing MMU OEA64 code. While these changes are not better tested, superpages support is disabled by default. To enable it, use vm.pmap.superpages_enabled=1. In this initial implementation, when superpages are disabled, system performance stays at the same level as without these changes. When superpages are enabled, buildworld time increases a bit (~2%). However, for workloads that put a heavy pressure on the TLB the performance boost is much bigger (see HPC Challenge and pgbench on D25237). Reviewed by: jhibbits Sponsored by: Eldorado Research Institute (eldorado.org.br) Differential Revision: https://reviews.freebsd.org/D25237 Notes: svn path=/head/; revision=367417
* [POWERPC] Floating-Point Exception trap supportAlfredo Dal'Ava Junior2020-11-063-1/+22
| | | | | | | | | | | | | | Add support for Floating-Point Exception traps on 32 and 64 bit platforms. Also make sure to clean FPSCR on EXEC and thread exit Author of initial version: Renato Riolino <renato.riolino@eldorad.org.br> Reviewed by: jhibbits Sponsored by: Eldorado Research Institute (eldorado.org.br) Differential Revision: https://reviews.freebsd.org/D23623 Notes: svn path=/head/; revision=367416
* Fix powerpc and powerpcspe buildsLeandro Lupori2020-11-051-7/+7
| | | | | | | | This change fixes 32-bit PowerPC builds, that r367390 broke (shift count >= width of type). Notes: svn path=/head/; revision=367397
* [PowerPC] hwpmc: add support for POWER8/9 PMCsLeandro Lupori2020-11-051-0/+1
| | | | | | | | | | | | | | | This change adds support for POWER8 and POWER9 PMCs (bare metal and pseries). All PowerISA 2.07B non-random events are supported. Implementation was based on that of PPC970. Reviewed by: jhibbits Sponsored by: Eldorado Research Institute (eldorado.org.br) Differential Revision: https://reviews.freebsd.org/D26110 Notes: svn path=/head/; revision=367390
* [PowerPC] Make PPC 970 PMC SPRs the standard onesLeandro Lupori2020-11-051-52/+73
| | | | | | | | | | | | | | And add a _74XX suffix to 74XX SPRs. This is a preparation for adding support to POWER8/9 PMCs, which have most SPRs equal to 970 ones. Reviewed by: jhibbits Sponsored by: Eldorado Research Institute (eldorado.org.br) Differential Revision: https://reviews.freebsd.org/D26532 Notes: svn path=/head/; revision=367383
* Get rid of sa->narg. It serves no purpose; use sa->callp->sy_narg instead.Edward Tomasz Napierala2020-09-271-1/+0
| | | | | | | | | Reviewed by: kib Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D26458 Notes: svn path=/head/; revision=366205
* Add a vmparam.h constant indicating pmap support for large pages.Mark Johnston2020-09-231-0/+5
| | | | | | | | | | | Enable SHM_LARGEPAGE support on arm64. Reviewed by: alc, kib Sponsored by: Juniper Networks, Inc., Klara, Inc. Differential Revision: https://reviews.freebsd.org/D26467 Notes: svn path=/head/; revision=366090
* [PowerPC64LE] Fix AP spinup on powernv.Brandon Bergren2020-09-231-0/+3
| | | | | | | | | | | | | | | | | | | | OPAL unconditionally enters secondary CPUs with only HV and SF set. I tried writing a secondary entry point instead, but OPAL rejected it and I am unsure why, so I resorted to making the system reset interrupt endian-flexible. This means we take a slight performance hit on wakeup on LE, but it is a good stopgap until we can figure out a reliable way to make OPAL enter where we want it to. It probably makes sense to have it around anyway, because I can imagine scenarios where the cpu resets itself to BE and does a software reset. Sponsored by: Tag1 Consulting, Inc. Notes: svn path=/head/; revision=366053
* [PowerPC64LE] powernv ILE setup code.Brandon Bergren2020-09-231-0/+1
| | | | | | | | | | | | | | | | | When running without a hypervisor, we need to set the ILE bit in the LPCR ourselves. For the boot processor, handle it in powernv_attach() like we do for other LPCR bits. No change for the APs, as they will use the lpcr global to set up their own LPCR when they do their own cpudep_ap_early_bootstrap() and pick up this automatically. Sponsored by: Tag1 Consulting, Inc. Notes: svn path=/head/; revision=366041
* [PowerPC64LE] Set up powerpc.powerpc64le architectureBrandon Bergren2020-09-221-0/+4
| | | | | | | | | | | | | | | | | | This is the initial set up for PowerPC64LE. The current plan is for this arch to remain experimental for FreeBSD 13. This started as a weekend learning project for me and kinda snowballed from there. (More to follow momentarily.) Reviewed by: imp (earlier version), emaste Sponsored by: Tag1 Consulting, Inc. Differential Revision: https://reviews.freebsd.org/D26399 Notes: svn path=/head/; revision=366032
* Sparsify the vm_page_dump bitmapD Scott Phillips2020-09-211-1/+2
| | | | | | | | | | | | | | | | | | | On Ampere Altra systems, the sparse population of RAM within the physical address space causes the vm_page_dump bitmap to be much larger than necessary, increasing the size from ~8 Mib to > 2 Gib (and overflowing `int` for the size). Changing the page dump bitmap also changes the minidump file format, so changes are also necessary in libkvm. Reviewed by: jhb Approved by: scottl (implicit) MFC after: 1 week Sponsored by: Ampere Computing, Inc. Differential Revision: https://reviews.freebsd.org/D26131 Notes: svn path=/head/; revision=365978
* Move vm_page_dump bitset array definition to MI codeD Scott Phillips2020-09-212-5/+12
| | | | | | | | | | | | | | | | | | | These definitions were repeated by all architectures, with small variations. Consolidate the common definitons in machine independent code and use bitset(9) macros for manipulation. Many opportunities for deduplication remain in the machine dependent minidump logic. The only intended functional change is increasing the bit index type to vm_pindex_t, allowing the indexing of pages with address of 8 TiB and greater. Reviewed by: kib, markj Approved by: scottl (implicit) MFC after: 1 week Sponsored by: Ampere Computing, Inc. Differential Revision: https://reviews.freebsd.org/D26129 Notes: svn path=/head/; revision=365977
* Add missing assignment forgotten in r365899Michal Meloun2020-09-201-0/+1
| | | | | | | | | Noticed by: mav MFC after: 1 month MFC with: r365899 Notes: svn path=/head/; revision=365926
* Add NetBSD compatible bus_space_peek_N() and bus_space_poke_N() functions.Michal Meloun2020-09-191-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | | One problem with the bus_space_read_N() and bus_space_write_N() family of functions is that they provide no protection against exceptions which can occur when no physical hardware or device responds to the read or write cycles. In such a situation, the system typically would panic due to a kernel-mode bus error. The bus_space_peek_N() and bus_space_poke_N() family of functions provide a mechanism to handle these exceptions gracefully without the risk of crashing the system. Typical example is access to PCI(e) configuration space in bus enumeration function on badly implemented PCI(e) root complexes (RK3399 or Neoverse N1 N1SDP and/or access to PCI(e) register when device is in deep sleep state. This commit adds a real implementation for arm64 only. The remaining architectures have bus_space_peek()/bus_space_poke() emulated by using bus_space_read()/bus_space_write() (without exception handling). MFC after: 1 month Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D25371 Notes: svn path=/head/; revision=365899
* [PowerPC64LE] Adjust ELF definitions for LE.Brandon Bergren2020-09-131-2/+5
| | | | | | | | | Set ELF_TARG_DATA correctly on PowerPC64LE. Sponsored by: Tag1 Consulting, Inc. Notes: svn path=/head/; revision=365693
* [PowerPC64LE] Add LOAD_LR_NIA and RETURN_TO_NATIVE_ENDIAN defines.Brandon Bergren2020-09-101-0/+37
| | | | | | | | | | | | | * Add LOAD_LR_NIA define. This is preferred to "bl 1f; 1:" because it doesn't pollute the branch predictor. * Add magic sequence to return the CPU to the correct endianness after jumping to cross-endian code, similar to the sequence from Linux. Sponsored by: Tag1 Consulting, Inc. Notes: svn path=/head/; revision=365607
* [PowerPC] Implement VDSO timebase access on powerpc*Brandon Bergren2020-09-081-0/+2
| | | | | | | | | | | | | | | | | Implement the remaining pieces needed to allow userland timestamp reading. Rewritten based on an intial essay into the problem by Justin Hibbits. (Copyright changed to my own on his request.) Tested on ppc64 (POWER9 Talos II), powerpcspe (e500v2 RB800), and powerpc (g4 PowerBook). Reviewed by: jhibbits (in irc) Sponsored by: Tag1 Consulting, Inc. Differential Revision: https://reviews.freebsd.org/D26347 Notes: svn path=/head/; revision=365442
* Add the MEM_EXTRACT_PADDR ioctl to /dev/mem.Mark Johnston2020-09-021-1/+1
| | | | | | | | | | | | | | | This allows privileged userspace processes to find information about the physical page backing a given mapping. It is useful in applications such as DPDK which perform some of their own memory management. Reviewed by: kib, jhb (previous version) MFC after: 2 weeks Sponsored by: Juniper Networks, Inc. Sponsored by: Klara Inc. Differential Revision: https://reviews.freebsd.org/D26237 Notes: svn path=/head/; revision=365265
* powerpc: clean up empty lines in .c and .h filesMateusz Guzik2020-09-0113-15/+3
| | | | Notes: svn path=/head/; revision=365073
* [PowerPC] Fix PMCLOG address widthLeandro Lupori2020-09-011-0/+5
| | | | | | | | | | | | PMCLOG macros were always using 32-bit addresses, even on PPC64. This resulted in truncated addresses in logs, when running on 64-bit PPC machines. Sponsored by: Eldorado Research Institute (eldorado.org.br) Differential Revision: https://reviews.freebsd.org/D26112 Notes: svn path=/head/; revision=365033
* [PowerPC] Fix booke64 qemu infinite loop in L2 cache enableBrandon Bergren2020-06-191-0/+1
| | | | | | | | | | | | | | | | | | Since qemu does not implement the L2 cache, we get stuck forever waiting for a bit to be set when trying to invalidate it. To prevent that, we should bail out if the L2 cache is missing. One easy way to check this is L2CFG0 == 0 (since L2CSIZE always has at least one bit set in a valid implementation) (tested on qemu, rb800, and x5000) Reviewed by: jhibbits Sponsored by: Tag1 Consulting, Inc. Differential Revision: https://reviews.freebsd.org/D25225 Notes: svn path=/head/; revision=362407
* powerpc: Use IFUNCs for copyin/copyout/etcJustin Hibbits2020-06-063-0/+34
| | | | | | | | | | | | | | | | Summary: Radix on AIM, and all of Book-E (currently), can do direct addressing of user space, instead of needing to map user addresses into kernel space. Take advantage of this to optimize the copy(9) functions for this behavior, and avoid effectively NOP translations. Test Plan: Tested on powerpcspe, powerpc64/booke, powerpc64/AIM Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D25129 Notes: svn path=/head/; revision=361861
* powerpc: Add a (CPU/runtime features) flags set to pcpu structJustin Hibbits2020-06-061-0/+4
| | | | | | | | | | | | | | | | | | Summary: The point of this addition is to cache CPU behavior 'features', to avoid having to recompute based on CPU, etc. The first such use case is to avoid the unnecessary manipulation of the SLBs (Segment Lookaside Buffers) when using the Radix pmap on POWER9. Since we already get the PCPU pointer wherever we swap the SLB entries, we can use a cached flag to check if it's necessary to perform the operation anyway, and skip it when not. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24908 Notes: svn path=/head/; revision=361859
* powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobjJustin Hibbits2020-05-272-64/+151
| | | | | | | | | | | | | With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon Notes: svn path=/head/; revision=361544
* [PowerPC] Fix atomic_cmpset_masked().Brandon Bergren2020-05-261-1/+1
| | | | | | | | | | | | | | | | | A recent kernel change caused the previously unused atomic_cmpset_masked() to be used. It had a typo in it. Instead of reading the old value from an uninitialized variable, read it from the passed-in pointer as intended. This fixes crashes on 64 bit Book-E. Obtained from: jhibbits Notes: svn path=/head/; revision=361535
* powerpc: Handle machine checks caused by D-ERAT multihitJustin Hibbits2020-05-211-0/+1
| | | | | | | | | | | | Instead of crashing the user process when a D-ERAT multihit is detected, try to flush the ERAT, and continue. This machine check indicates a likely PMAP invalidation shortcoming that will need to be addressed, but it's recoverable, so just recover. The recovery is pmap-specific to flush the ERAT, so add a pmap function to do so, currently only implemented by the POWER9 radix pmap. Notes: svn path=/head/; revision=361317
* powerpc/radix mmu: No need for delayed TLB invalidationJustin Hibbits2020-05-201-6/+0
| | | | | | | | | x86 needs delayed TLB invalidation because invalidation requires an expensive IPI. PowerPC has had a TLB invalidation instruction since the POWER1 in 1990, so there's no need to delay anything. Notes: svn path=/head/; revision=361271
* powerpc64: Implement Radix MMU for POWER9 CPUsJustin Hibbits2020-05-119-23/+199
| | | | | | | | | | | | | | | | Summary: POWER9 supports two MMU formats: traditional hashed page tables, and Radix page tables, similar to what's presesnt on most other architectures. The PowerISA also specifies a process table -- a table of page table pointers-- which on the POWER9 is only available with the Radix MMU, so we can take advantage of it with the Radix MMU driver. Written by Matt Macy. Differential Revision: https://reviews.freebsd.org/D19516 Notes: svn path=/head/; revision=360887
* powerpc: Add a CPU-custom machine check handlerJustin Hibbits2020-05-102-3/+28
| | | | | | | | | | | | | | | | | Summary: Some machine checks are process-recoverable, others are not. Let a CPU-specific handler decide what to do. This works around a machine check error hit while building www/firefox and mail/thunderbird, which would otherwise cause the build to fail. More work is needed to handle all possible machine check conditions, but this is sufficient to unblock some ports building. Differential Revision: https://reviews.freebsd.org/D23731 Notes: svn path=/head/; revision=360881
* [PowerPC] kernel ifunc support for powerpc*, fix ppc64 relocation oddities.Brandon Bergren2020-05-072-4/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a general cleanup of the relocatable kernel support on powerpc, needed to enable kernel ifuncs. * Fix some relocatable issues in the kernel linker, and change to using a RELOCATABLE_KERNEL #define instead of #ifdef __powerpc__ for parts that other platforms can use in the future if they wish to have ET_DYN kernels. * Get rid of the DB_STOFFS hack now that the kernel is relocated to the DMAP properly across the board on powerpc64. * Add powerpc64 and powerpc32 ifunc functionality. * Allow AIM64 virtual mode OF kernels to run from the DMAP like other AIM64 by implementing a virtual mode restart. This fixes the runtime address on PowerMac G5. * Fix symbol relocation problems on post-relocation kernels by relocating the symbol table. * Add an undocumented method for supplying kernel symbols on powernv and other powerpc machines using linux-style kernel/initrd loading -- If you pass the kernel in as the initrd as well, the copy resident in initrd will be used as a source for symbols when initializing the debugger. This method is subject to removal once we have a better way of doing this. Approved by: jhibbits Relnotes: yes Sponsored by: Tag1 Consulting, Inc. Differential Revision: https://reviews.freebsd.org/D23156 Notes: svn path=/head/; revision=360794
* powerpc/booke: Change Book-E 64-bit pmap to 4-level tableJustin Hibbits2020-04-113-25/+40
| | | | | | | | | | | | | Summary: The existing page table is fraught with errors, since it creates a hole in the address space bits. Fix this by taking a cue from the POWER9 radix pmap, and make the page table 4 levels, 52 bits. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24220 Notes: svn path=/head/; revision=359791
* Expand generic subword atomic primitivesConrad Meyer2020-03-251-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The goal of this change is to make the atomic_load_acq_{8,16}, atomic_testandset{,_acq}_long, and atomic_testandclear_long primitives available in MI-namespace. The second goal is to get this draft out of my local tree, as anything that requires a full tinderbox is a big burden out of tree. MD specifics can be refined individually afterwards. The generic implementations may not be ideal for your architecture; feel free to implement better versions. If no subword_atomic definitions are needed, the include can be removed from your arch's machine/atomic.h. Generic definitions are guarded by defined macros of the same name. To avoid picking up conflicting generic definitions, some macro defines are added to various MD machine/atomic.h to register an existing implementation. Include _atomic_subword.h in arm and arm64 machine/atomic.h. For some odd reason, KCSAN only generates some versions of primitives. Generate the _acq variants of atomic_load.*_8, atomic_load.*_16, and atomic_testandset.*_long. There are other questionably disabled primitives, but I didn't run into them, so I left them alone. KCSAN is only built for amd64 in tinderbox for now. Add atomic_subword implementations of atomic_load_acq_{8,16} implemented using masking and atomic_load_acq_32. Add generic atomic_subword implementations of atomic_testandset_long(), atomic_testandclear_long(), and atomic_testandset_acq_long(), using atomic_fcmpset_long() and atomic_fcmpset_acq_long(). On x86, add atomic_testandset_acq_long as an alias for atomic_testandset_long. Reviewed by: kevans, rlibby (previous versions both) Differential Revision: https://reviews.freebsd.org/D22963 Notes: svn path=/head/; revision=359311
* powerpc: Axe PPC4xx support.Justin Hibbits2020-03-185-153/+36
| | | | | | | | | | | | Summary: The support was added almost a decade ago, and never completed. Just axe it. It was also inadvertently broken 5 years ago, and nobody noticed. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23753 Notes: svn path=/head/; revision=359058
* Define MAXCPU consistently between the kernel and KLDs.Mark Johnston2020-02-051-2/+2
| | | | | | | | | | | | | | | | This reverts r177661. The change is no longer very useful since out-of-tree KLDs will be built to target SMP kernels anyway. Moveover it breaks the KBI in !SMP builds since cpuset_t's layout depends on the value of MAXCPU, and several kernel interfaces, notably smp_rendezvous_cpus(), take a cpuset_t as a parameter. PR: 243711 Reviewed by: jhb, kib Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D23512 Notes: svn path=/head/; revision=357585
* D23057: [PowerPC] Fix offset calculations in bridge modeBrandon Bergren2020-01-181-2/+3
| | | | | | | | | | | | | | | | | | | | | | In rS354701, I replaced text relocations with offsets from &generictrap. Unfortunately, the magic variable I was using doesn't actually mean the address of &generictrap, in bridge mode it actually means &generictrap64. So, for bridge mode to work, it is necessary to differentiate between "where do we need to branch to to handle a trap" and "where is &generictrap for purposes of doing relative math". Introduce a new TRAP_ENTRY and use it instead of TRAP_GENTRAP for doing actual calls to the generic trap handler. Reported by: Mark Millard <marklmi@yahoo.com> Reviewed by: jhibbits Sponsored by: Tag1 Consulting, Inc. Differential Revision: https://reviews.freebsd.org/D23057 Notes: svn path=/head/; revision=356862
* powerpc: Fix the NUMA domain list on powernvJustin Hibbits2020-01-182-0/+5
| | | | | | | | | | | | | | | | | | Summary: Consolidate the NUMA associativity handling into a platform function. Non-NUMA platforms will just fall back to the default (0). Currently only implemented for powernv, which uses a lookup table to map the device tree associativity into a system NUMA domain. Fixes hangs on powernv after r356534, and corrects a fairly longstanding bug in powernv's NUMA handling, which ended up using domains 1 and 2 for devices and memory on power9, while CPUs were bound to domains 0 and 1. Reviewed by: bdragon, luporl Differential Revision: https://reviews.freebsd.org/D23220 Notes: svn path=/head/; revision=356858
* [PPC] Fix wrong commentLeandro Lupori2020-01-171-1/+1
| | | | | | | pcb_context[20] holds r12-r31 and not r14-r31, as the comment said. Notes: svn path=/head/; revision=356831
* [PowerPC] [MIPS] Implement 32-bit kernel emulation of atomic64 operationsBrandon Bergren2020-01-021-0/+4
| | | | | | | | | | | | | | | | | | | This is a lock-based emulation of 64-bit atomics for kernel use, split off from an earlier patch by jhibbits. This is needed to unblock future improvements that reduce the need for locking on 64-bit platforms by using atomic updates. The implementation allows for future integration with userland atomic64, but as that implies going through sysarch for every use, the current status quo of userland doing its own locking may be for the best. Submitted by: jhibbits (original patch), kevans (mips bits) Reviewed by: jhibbits, jeff, kevans Differential Revision: https://reviews.freebsd.org/D22976 Notes: svn path=/head/; revision=356308
* [PowerPC] Fix panic when attempting to handle an HMI from an idle threadBrandon Bergren2019-12-301-1/+2
| | | | | | | | | | | | | | | | | | | | | In IRC, sfs_ finally managed to get a good trace of a kernel panic that was happening when attempting to use webengine. As it turns out, we were using vtophys() from interrupt context on an idle thread in opal_hmi_handler2(). Since this involves locking the kernel pmap on PPC64 at the moment, this ended up tripping a KASSERT in mtx_lock(), which then caused a parallel panic stampede. So, avoid this by preallocating the flags variable and storing it in PCPU. Fixes "panic: mtx_lock() by idle thread 0x... on sleep mutex kernelpmap". Differential Revision: https://reviews.freebsd.org/D22962 Notes: svn path=/head/; revision=356199
* Fix the powerpc copyout fixup from r356113Justin Hibbits2019-12-271-0/+28
| | | | | | | | | | | | | | Summary: r356113 used an older patch, which predated the freebsd_copyout_auxargs() addition. Fix this by using a private powerpc_copyout_auxargs() instead, and keep it private to powerpc, not in MI files. Reviewed by: kib, bdragon Differential Revision: https://reviews.freebsd.org/D22935 Notes: svn path=/head/; revision=356129
* [PowerPC] Define powerpc IFUNC kernel/userspace ABI.Brandon Bergren2019-12-241-0/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a prerequisite for anything IFUNC in the ELFv2 / clang switch. Since probing cpu info on powerpc is a privileged operation, define that we pass AT_HWCAP / AT_HWCAP2 through as cpu_features and cpu_features2 to ifunc resolvers. This is particularly important when dealing with non-PLT GNU IFUNC, which is not allowed to PLT call from resolvers and therefore can't access global variables. The naming convention "cpu_features"/"cpu_features2" is an existing FreeBSD PowerPC convention and matches the way we treat these variables in machine/cpu.h. The underlying variables are u_long, however, as per the commit message for r332868, only the low 32 bits are ever used, so the underlying flags are compatible across all of PowerPC. The resolver prototype is defined to reserve the maximum number of register-passed parameters the various PowerPC ABIs allow. This leaves plenty of room for growth without needing to resort to passing via the stack in the future. Reviewed by: jhibbits Differential Revision: https://reviews.freebsd.org/D22787 Notes: svn path=/head/; revision=356061
* [PPC] Handle qOffsets packetLeandro Lupori2019-12-161-0/+1
| | | | | | | | | | | | | | On PowerPC, this is needed in order for the debugger to find out the memory offset where the kernel image was loaded on the remote target. This fixes symbol resolution when remote debugging a PowerPC kernel. Reviewed by: cem Differential Revision: https://reviews.freebsd.org/D22767 Notes: svn path=/head/; revision=355801
* Introduce vm_page_astate.Mark Johnston2019-12-101-1/+1
| | | | | | | | | | | | | | | | | This is a 32-bit structure embedded in each vm_page, consisting mostly of page queue state. The use of a structure makes it easy to store a snapshot of a page's queue state in a stack variable and use cmpset loops to update that state without requiring the page lock. This change merely adds the structure and updates references to atomic state fields. No functional change intended. Reviewed by: alc, jeff, kib Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D22650 Notes: svn path=/head/; revision=355586
* powerpc: Use builtins for fls/flslJustin Hibbits2019-12-081-0/+14
| | | | | | | | | | | | | | Summary: There's no need to use the fallback fls() and flsl() libkern functions when the PowerISA includes instructions that already do the bulk of the work. Take advantage of this through the GCC builtins __builtin_clz() and __builtin_clzl(). Reviewed by: luporl Differential Revision: https://reviews.freebsd.org/D22340 Notes: svn path=/head/; revision=355521
* powerpc/pmap: NUMA-ize vm_page_array on powerpcJustin Hibbits2019-12-072-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Summary: This matches r351198 from amd64. This only applies to AIM64 and Book-E. On AIM64 it short-circuits with one domain, to behave similar to existing. Otherwise it will allocate 16MB huge pages to hold the page array, across all NUMA domains. On the first domain it will shift the page array base up, to "upper-align" the page array in that domain, so as to reduce the number of pages from the next domain appearing in this domain. After the first domain, subsequent domains will be allocated in full 16MB pages, until the final domain, which can be short. This means some inner domains may have pages accounted in earlier domains. On Book-E the page array is setup at MMU bootstrap time so that it's always mapped in TLB1, on both 32-bit and 64-bit. This reduces the TLB0 overhead for touching the vm_page_array, which reduces up to one TLB miss per array access. Since page_range (vm_page_startup()) is no longer used on Book-E but is on 32-bit AIM, mark the variable as potentially unused, rather than using a nasty #if defined() list. Reviewed by: luporl Differential Revision: https://reviews.freebsd.org/D21449 Notes: svn path=/head/; revision=355481
generated by cgit v1.3 (git 2.53.0) at 2026-04-18 08:03:30 +0000