summaryrefslogtreecommitdiff
path: root/sys/x86
Commit message (Collapse)AuthorAgeFilesLines
* dmar: reserve memory windows of PCIe root portRyan Libby2020-12-091-0/+62
| | | | | | | | | | | | | | | | | | PCI memory address space is shared between memory-mapped devices (MMIO) and host memory (which may be remapped by an IOMMU). Device accesses to an address within a memory aperture in a PCIe root port will be treated as peer-to-peer and not forwarded to an IOMMU. To avoid this, reserve the address space of the root port's memory apertures in the address space used by the IOMMU for remapping. Reviewed by: kib, tychon Discussed with: Anton Rang <rang@acm.org> Tested by: tychon Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D27503 Notes: svn path=/head/; revision=368490
* busdma: Annotate bus_dmamap_sync() with fenceWarner Losh2020-12-041-1/+3
| | | | | | | | | | | | | | | | | | | | | | Add an explicit thread fence release before returning from bus_dmamap_sync. This should be a no-op in practice, but makes explicit that all ordinary stores will be completed before subsequent reads/writes to ordinary device memory. On x86, normal memory ordering is strong enough to generally guarantee this. The fence keeps the optimizer (likely LTO) from reordering other calls around this. The other architectures already have calls, as appropriate, that are equivalent. Note: On x86, there is one exception to this rule. If you've mapped memory as write combining, then you will need to add a sfence or similar. Normally, though, busdma doesn't operate on such memory, and drivers that do already cope appropriately. Reviewed by: kib@, gallatin@, chuck@, mav@ Differential Revision: https://reviews.freebsd.org/D27448 Notes: svn path=/head/; revision=368351
* Add a kstack_contains() helper function.John Baldwin2020-12-011-3/+1
| | | | | | | | | | | | | This is useful for stack unwinders which need to avoid out-of-bounds reads of a kernel stack which can trigger kernel faults. Reviewed by: kib, markj Obtained from: CheriBSD Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D27356 Notes: svn path=/head/; revision=368240
* Add VT driver for VBE framebuffer deviceToomas Soome2020-11-301-0/+14
| | | | | | | | | | | | | | | Implement vt_vbefb to support Vesa Bios Extensions (VBE) framebuffer with VT. vt_vbefb is built based on vt_efifb and is assuming similar data for initialization, use MODINFOMD_VBE_FB to identify the structure vbe_fb in kernel metadata. struct vbe_fb, is populated by boot loader, and is passed to kernel via metadata payload. Differential Revision: https://reviews.freebsd.org/D27373 Notes: svn path=/head/; revision=368168
* hwpstate_intel: don't unconditionally print the error messageYuri Pankov2020-11-291-2/+4
| | | | | | | | | | | Actually check the wrmsr_safe() return value when setting autonomous HWP for package. PR: 245582 Differential Revision: https://reviews.freebsd.org/D24744 Notes: svn path=/head/; revision=368140
* Add device_t member to struct iommu.Ruslan Bukin2020-11-161-0/+1
| | | | | | | | | | | | This is needed on arm64 for the interface between iommu framework and iommu controller drivers. Reviewed by: kib Sponsored by: Innovate DSbD Differential Revision: https://reviews.freebsd.org/D27229 Notes: svn path=/head/; revision=367725
* linux(4): Deduplicate unimpl/dummy syscall handlersConrad Meyer2020-11-051-0/+69
| | | | | | | | | | No functional change. Reviewed by: emaste, trasz Differential Revision: https://reviews.freebsd.org/D27099 Notes: svn path=/head/; revision=367395
* Move the iommu stubs to a generic place, so they are available on all theRuslan Bukin2020-10-231-20/+0
| | | | | | | | | | | | | | | platforms. This allows to not depend on the IOMMU macro in AHCI driver. Requested by: kib Suggested by: andrew Reviewed by: kib Sponsored by: Innovate DSbD Differential Revision: https://reviews.freebsd.org/D26887 Notes: svn path=/head/; revision=366980
* Assign the reserved apic region (GAS entry) to the iommu domain msi_entry.Ruslan Bukin2020-10-191-1/+1
| | | | | | | | | | Requested by: kib Reviewed by: kib Sponsored by: Innovate DSbD Differential Revision: https://reviews.freebsd.org/D26859 Notes: svn path=/head/; revision=366835
* Limit workaround for errata E400 to appropriate AMD cpus.Konstantin Belousov2020-10-143-33/+12
| | | | | | | | | | | | | | | | | | | | | From Linux sources and several datasheets I looked at, it seems that the workaround is only needed on families 0xf and 0x10. For instance, Ryzens do not implement the accessed MSR at all, it is documented as reserved. Also, hypervisors should not allow guest to put CPU into idle state, so activate workaround only when on bare hardware. While there, style the code: move MSR defines to specialreg.h move identification to initcpu.c Reported by: whu Reviewed by: avg Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D26470 Notes: svn path=/head/; revision=366712
* Avoid dump_avail[] redefinition.Konstantin Belousov2020-10-141-0/+1
| | | | | | | | | | | | | Move dump_avail[] extern declaration and inlines into a new header vm/vm_dumpset.h. This fixes default gcc build for mips. Reviewed by: alc, scottph Tested by: kevans (previous version) Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D26741 Notes: svn path=/head/; revision=366711
* timer_restore is now unused, remove itWarner Losh2020-10-081-20/+0
| | | | | | | | apm was the only consumer of timer_restore. Now that it's gone, this can be removed. Notes: svn path=/head/; revision=366547
* Remove unused function cpu_boot()Mitchell Horne2020-10-061-11/+0
| | | | | | | | | | | The prototype was added with the creation of kern_shutdown.c in r17658, but it appears to have never been implemented. Remove it now. Reviewed by: cem, kib Differential Revision: https://reviews.freebsd.org/D26702 Notes: svn path=/head/; revision=366503
* Explicit CTLFLAG_DYN not neededRyan Moeller2020-10-041-5/+5
| | | | | | | | | | | | Dynamically created OIDs automatically get this flag set. Reviewed by: jhb MFC after: 1 week Sponsored by: iXsystems, Inc. Differential Revision: https://reviews.freebsd.org/D26561 Notes: svn path=/head/; revision=366433
* Move ctx_switch_xsave declaration to amd64 md_var.h.Konstantin Belousov2020-10-031-1/+0
| | | | | | | | Sponsored by: The FreeBSD Foundation MFC after: 3 days Notes: svn path=/head/; revision=366415
* Rename kernel option ACPI_DMAR to IOMMU.Ruslan Bukin2020-09-293-14/+17
| | | | | | | | | | | This is mostly needed for a common arm64/amd64 iommu code. Reviewed by: kib Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D26587 Notes: svn path=/head/; revision=366267
* Add missing declarations of 64-bit variants of bus_peek/bus_poke on amd64.Michal Meloun2020-09-241-0/+6
| | | | | | | | | | | It fixes GENERIC-KCSAN build. Reported by: rpokala MFC after: 1 month MFC with: r365899 Notes: svn path=/head/; revision=366107
* Use envvar rather than nonstandard hint. linesWarner Losh2020-09-231-9/+9
| | | | | | | | | | | | | The NOTES files have a bunch of hint lines that are removed when generating LINT. However, we can achieve the same effect by prepending each of the lines with 'envvar' so the NOTES files become standard config(8) files. No functional changes as the sed script to generate the LINT files filters these either way. Suggested by: kevans Notes: svn path=/head/; revision=366088
* Move vm_page_dump bitset array definition to MI codeD Scott Phillips2020-09-211-3/+0
| | | | | | | | | | | | | | | | | | | These definitions were repeated by all architectures, with small variations. Consolidate the common definitons in machine independent code and use bitset(9) macros for manipulation. Many opportunities for deduplication remain in the machine dependent minidump logic. The only intended functional change is increasing the bit index type to vm_pindex_t, allowing the indexing of pages with address of 8 TiB and greater. Reviewed by: kib, markj Approved by: scottl (implicit) MFC after: 1 week Sponsored by: Ampere Computing, Inc. Differential Revision: https://reviews.freebsd.org/D26129 Notes: svn path=/head/; revision=365977
* Add missing assignment forgotten in r365899Michal Meloun2020-09-201-0/+1
| | | | | | | | | Noticed by: mav MFC after: 1 month MFC with: r365899 Notes: svn path=/head/; revision=365926
* Add NetBSD compatible bus_space_peek_N() and bus_space_poke_N() functions.Michal Meloun2020-09-191-0/+25
| | | | | | | | | | | | | | | | | | | | | | | | | One problem with the bus_space_read_N() and bus_space_write_N() family of functions is that they provide no protection against exceptions which can occur when no physical hardware or device responds to the read or write cycles. In such a situation, the system typically would panic due to a kernel-mode bus error. The bus_space_peek_N() and bus_space_poke_N() family of functions provide a mechanism to handle these exceptions gracefully without the risk of crashing the system. Typical example is access to PCI(e) configuration space in bus enumeration function on badly implemented PCI(e) root complexes (RK3399 or Neoverse N1 N1SDP and/or access to PCI(e) register when device is in deep sleep state. This commit adds a real implementation for arm64 only. The remaining architectures have bus_space_peek()/bus_space_poke() emulated by using bus_space_read()/bus_space_write() (without exception handling). MFC after: 1 month Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D25371 Notes: svn path=/head/; revision=365899
* Refine the busdma template interface. Provide tools for filling in fieldsScott Long2020-09-141-32/+1
| | | | | | | | | | that can be extended, but also ensure compile-time type checking. Refactor common code out of arch-specific implementations. Move the mpr and mps drivers to this new API. The template type remains visible to the consumer so that it can be allocated on the stack, but should be considered opaque. Notes: svn path=/head/; revision=365706
* amd64: prevent KCSan false positives on LAPIC mappingJason A. Harmening2020-09-121-3/+13
| | | | | | | | | | | | | | | | | | | | | | For configurations without x2APIC support (guests, older hardware), the global LAPIC MMIO mapping will trigger false-positive KCSan reports as it will appear that multiple CPUs are concurrently reading and writing the same address. This isn't actually true, as the underlying physical access will be performed on the local CPU's APIC. Additionally, because LAPIC access can happen during event timer configuration, the resulting KCSan printf can produce a panic due to attempted recursion on event timer resources. Add a __nosanitizethread preprocessor define to prevent the compiler from inserting TSan hooks, and apply it to the x86 LAPIC accessors. PR: 249149 Reported by: gbe Reviewed by: andrew, kib Tested by: gbe Differential Revision: https://reviews.freebsd.org/D26354 Notes: svn path=/head/; revision=365652
* Add constant for the DE_CFG MSR on AMD CPUs.John Baldwin2020-09-111-0/+1
| | | | | | | | | Reported by: Patrick Mooney <pmooney@pfmooney.com> MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D25885 Notes: svn path=/head/; revision=365642
* Move the rid variable to the generic iommu context.Ruslan Bukin2020-09-102-8/+8
| | | | | | | | | | | It could be used in various IOMMU platforms, not only DMAR. Reviewed by: kib Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D26373 Notes: svn path=/head/; revision=365577
* x86: clean up empty lines in .c and .h filesMateusz Guzik2020-09-0140-63/+15
| | | | Notes: svn path=/head/; revision=365079
* amd64: Handle 5-level paging on wakeup.Konstantin Belousov2020-08-231-6/+16
| | | | | | | | | | We can switch into long mode directly with LA57 enabled. Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D25273 Notes: svn path=/head/; revision=364534
* Add amd64 procctl(2) ops to manage forced LA48/LA57 VA after exec.Konstantin Belousov2020-08-231-1/+10
| | | | | | | | | Tested by: pho (LA48 hardware) Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D25273 Notes: svn path=/head/; revision=364530
* Add definition for CR4.LA57 bit.Konstantin Belousov2020-08-231-0/+1
| | | | | | | | | Tested by: pho Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D25273 Notes: svn path=/head/; revision=364522
* Export a routine to provide the TSC_AUX MSR value and use this in vmm.Peter Grehan2020-08-181-0/+1
| | | | | | | | | | | Also, drop an unnecessary set of braces. Requested by: kib Reviewed by: kib MFC after: 3 weeks Notes: svn path=/head/; revision=364343
* Move dmar_domain_unload_task to busdma_iommu.c.Ruslan Bukin2020-08-061-23/+0
| | | | | | | | | Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25972 Notes: svn path=/head/; revision=363949
* Add iommu_domain constructor and destructor.Ruslan Bukin2020-08-063-18/+11
| | | | | | | | | Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25956 Notes: svn path=/head/; revision=363946
* o Add machine/iommu.h and include MD iommu headers from it,Ruslan Bukin2020-08-051-0/+13
| | | | | | | | | | | | | so we don't ifdef for every arch in busdma_iommu.c; o No need to include specialreg.h for x86, remove it. Requested by: andrew Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25957 Notes: svn path=/head/; revision=363929
* Add a few macroses for conversion between DMAR unit, domain, ctxRuslan Bukin2020-08-045-45/+56
| | | | | | | | | | | and IOMMU unit, domain, ctx. Reviewed by: kib Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D25926 Notes: svn path=/head/; revision=363864
* Remove free_domain() and uma_zfree_domain().Mark Johnston2020-08-041-4/+4
| | | | | | | | | | | | | | These functions were introduced before UMA started ensuring that freed memory gets placed in domain-local caches. They no longer serve any purpose since UMA now provides their functionality by default. Remove them to simplyify the kernel memory allocator interfaces a bit. Reviewed by: cem, kib Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D25937 Notes: svn path=/head/; revision=363834
* Add iommu_domain_map_ops virtual table with map/unmap methodsRuslan Bukin2020-07-313-8/+25
| | | | | | | | | | | so x86 can support Intel DMAR and AMD IOMMU simultaneously. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25894 Notes: svn path=/head/; revision=363747
* o Don't include headers from iommu.h, include them from the headerRuslan Bukin2020-07-297-36/+37
| | | | | | | | | | | | consumers instead; o Order includes properly. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25878 Notes: svn path=/head/; revision=363675
* Fix !ACPI_DMAR build.Ruslan Bukin2020-07-291-0/+4
| | | | | | | | | Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25882 Notes: svn path=/head/; revision=363669
* o Move iommu_set_buswide_ctx, iommu_is_buswide_ctx toRuslan Bukin2020-07-293-29/+0
| | | | | | | | | | | | | the generic iommu busdma backend; o Move bus_dma_iommu_set_buswide, bus_dma_iommu_load_ident prototypes to iommu.h. Reviewed by: kib Sponsored by: DARPA, AFRL Differential Revision: https://reviews.freebsd.org/D25866 Notes: svn path=/head/; revision=363663
* o Move the buswide_ctxs bitmap to iommu_unit and rename related functions.Ruslan Bukin2020-07-285-26/+16
| | | | | | | | | | | o Rename bus_dma_dmar_load_ident() as well. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25852 Notes: svn path=/head/; revision=363650
* Add initial driver for ACPI Platform Error Interfaces.Alexander Motin2020-07-272-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | APEI allows platform to report different kinds of errors to OS in several ways. We've found that Supermicro X10/X11 motherboards report PCIe errors appearing on hot-unplug via this interface using NMI. Without respective driver it ended up in kernel panic without any additional information. This driver introduces support for the APEI Generic Hardware Error Source reporting via NMI, SCI or polling. It decodes the reported errors and either pass them to pci(4) for processing or just logs otherwise. Errors marked as fatal still end up in kernel panic, but some more informative. When somebody get to native PCIe AER support implementation both of the reporting mechanisms should get common error recovery code. Since in our case errors happen when the device is already gone, there is nothing to recover, so the code just clears the error statuses, practically ignoring the otherwise destructive NMIs in nicer way. MFC after: 2 weeks Relnotes: yes Sponsored by: iXsystems, Inc. Notes: svn path=/head/; revision=363624
* Rename DMAR flags:Ruslan Bukin2020-07-264-55/+56
| | | | | | | | | | | | o DMAR_DOMAIN_* -> IOMMU_DOMAIN_* o DMAR_PGF_* -> IOMMU_PGF_* Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25812 Notes: svn path=/head/; revision=363560
* o Make the _hw_iommu sysctl node non-static;Ruslan Bukin2020-07-251-5/+5
| | | | | | | | | | | o Move the dmar sysctl knobs to _hw_iommu_dmar. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25807 Notes: svn path=/head/; revision=363534
* o Move iommu gas prototypes, DMAR flags to iommu.h;Ruslan Bukin2020-07-253-47/+0
| | | | | | | | | | | o Move hw.dmar sysctl node to iommu_gas.c. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25802 Notes: svn path=/head/; revision=363533
* Allow swi_sched() to be called from NMI context.Alexander Motin2020-07-254-1/+26
| | | | | | | | | | | | | | | | | | For purposes of handling hardware error reported via NMIs I need a way to escape NMI context, being too restrictive to do something significant. To do it this change introduces new swi_sched() flag SWI_FROMNMI, making it careful about used KPIs. On platforms allowing IPI sending from NMI context (x86 for now) it immediately wakes clk_intr_event via new IPI_SWI, otherwise it works just like SWI_DELAY. To handle the delayed SWIs this patch calls clk_intr_event on every hardclock() tick. MFC after: 2 weeks Sponsored by: iXsystems, Inc. Differential Revision: https://reviews.freebsd.org/D25754 Notes: svn path=/head/; revision=363527
* Move Intel GAS to dev/iommu/ as now a part of generic iommu framework.Ruslan Bukin2020-07-251-729/+0
| | | | | | | | | Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25799 Notes: svn path=/head/; revision=363524
* Split-out the Intel GAS (Guest Address Space) management componentRuslan Bukin2020-07-256-216/+210
| | | | | | | | | | | from Intel DMAR support, so it can be used on other IOMMU systems. Reviewed by: kib Sponsored by: DARPA/AFRL Differential Revision: https://reviews.freebsd.org/D25743 Notes: svn path=/head/; revision=363515
* Introduce ipi_self_from_nmi().Alexander Motin2020-07-242-0/+16
| | | | | | | | | | | | | It allows safe IPI sending to current CPU from NMI context. Unlike other ipi_*() functions this waits for delivery to leave LAPIC in a state safe for interrupted code. MFC after: 2 weeks Sponsored by: iXsystems, Inc. Notes: svn path=/head/; revision=363495
* Use APIC_IPI_DEST_OTHERS for bitmapped IPIs too.Alexander Motin2020-07-242-24/+35
| | | | | | | | | It should save bunch of LAPIC register accesses. MFC after: 2 weeks Notes: svn path=/head/; revision=363493
* Make lapic_ipi_vectored(APIC_IPI_DEST_SELF) NMI safe.Alexander Motin2020-07-241-58/+48
| | | | | | | | | | | | | | | | | | | Sending IPI to self or all CPUs does not require write into upper part of the ICR, prone to races. Previously the code disabled interrupts, but it was not enough for NMIs. Instead of that when possible write only lower part of the register, or use special SELF IPI register in x2APIC mode. This also removes ICR reads used to preserve reserved bits on write. It was there from the beginning, but I failed to find explanation why, neither I see Linux doing it. Specification even tells that ICR content may be lost in deep C-states, so if hardware does not bother to preserve it, why should we? MFC after: 2 weeks Sponsored by: iXsystems, Inc. Notes: svn path=/head/; revision=363490
generated by cgit v1.2.3 (git 2.25.1) at 2025-11-02 11:26:13 +0000