| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
|
| |
Support for these processors was removed a few years ago, but a few
references remain that should be removed.
Sponsored by: Netflix
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/2039
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This patch adds support for AMD IBS. It adds a new class of performance
counter that cotains two events: ibs-fetch and ibs-op events. Unlike
most existing sampled events, IBS events provide a number of values
containing extra information regarding the sample. To support this we
use the existing callchain event, and introduce a new flag for multipart
payloads. The first 8 bytes of the pc_sample contains a header that
defines up to four payloads.
Sponsored by: Netflix
Reviewed by: imp,mhorne
Pull Request: https://github.com/freebsd/freebsd-src/pull/2022
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In principle, do_cpuid() should only be used for CPUID leaves without
sub-leaves. Even accessing sub-leaf zero (ECX=0), one must use
cpuid_count(ax, 0) rather than cpuid(ax).
However, one might assume do_cpuid(ax) is equivalent to
cpuid_count(ax, 0), but the old do_cpuid() did not initialize ECX before
executing the CPUID instruction. If ECX contained a non-zero value, the
instruction could return unexpected results, potentially leading to
subtle and hard-to-debug issues, especially in ported code.
To be future-proof and to help port code, adjust do_cpuid(ax) to be
cpuid_count(ax, 0) to explicitly set ECX=0.
It's believed that this change does not fix any real bugs in FreeBSD.
See also the DragonFly commit:
https://github.com/DragonFlyBSD/DragonFlyBSD/commit/0087a1d163488a57787a9a6431dd94070b1988d4
Signed-off-by: Aaron LI <aly@aaronly.me>
Reviewed by: kib
Pull Request: https://github.com/freebsd/freebsd-src/pull/2027
|
| |
|
|
|
|
|
| |
kgdb only uses the marked fields from dumppcb for initial frame
reconstruction.
This reverts commit 8f23665fed2fbaf4481359b4d2fcdd7b9feb40e3.
|
| |
|
|
|
|
|
|
|
|
| |
There are programs that depend on this structure (e.g. kernel debuggers)
that breaks when the ABI changes.
Signed-off-by: Minsoo Choo <minsoochoo0122@proton.me>
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D55149
|
| |
|
|
|
| |
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Define _ALIGNBYTES using sizeof(void *) (no functional change on any
existing architecture) which will allow it to work with CHERI were we
must align things up to capability alignment.
In _ALIGN, replace integer manipulation which does not preserve pointer
provenance with a type and provenance preserving builtin. This requires
modest changes in code which assumes _ALIGN returns an integer, but
those are relatively rare.
Reviewed by: kib, markj
Effort: CHERI upstreaming
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D53947
|
| |
|
|
|
|
|
|
| |
kexec hasn't been ported to these architectures, yet, so appease the
build with dummy headers.
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D51625
|
| |
|
|
|
|
|
|
|
| |
We already use clflushopt in support.S, there is no reason to manually
construct the encoding. Initially it was done because toolchains did
not supported the (then) new instruction.
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
|
| |
|
|
|
|
|
| |
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D47935
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The 'runq' machinery now depends on only two settable parameters,
RQ_MAX_PRIO, the maximum priority number that can be accepted, the
minimum being 0, and RQ_PPQ, the number of priorities per queue (to
reduce the number of queues).
All other parameters are deduced from these ones. Also, all
architectures automatically get a runq word that is their natural word.
RQB_FFS() always was 'ffsl() - 1' except for amd64 where it was
'bsfq()'. Now that all these finally call compiler builtins, the
resulting assembly code is the same, so there is no cost to removing
this special case.
After all these changes, <machine/runq.h> headers have no more purpose,
so remove them.
While here, fix potentially confusing parameter name for RQB_WORD() and
RQB_BIT().
While here, include all necessary headers so that <sys/runq.h> can be
included standalone.
No functional change (intended).
Reviewed by: kib
MFC after: 1 month
Event: Kitchener-Waterloo Hackathon 202506
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D45387
|
| |
|
|
|
|
|
|
| |
It is no longer used anywhere (since 62508c531e0f), and not defined for
arm64 which also supports ACPI.
Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D48486
|
| |
|
|
|
|
|
|
| |
This flag was used as a transition for differing pcib implementations.
Today it is defined for all supported architectures, and can be removed.
Reviewed by: imp, jhb
Differential Revision: https://reviews.freebsd.org/D48485
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
These macros have substantially identical implementations on each
platform. Use roundup2/rounddown2 for round_page/trunc_page.
This version standardizes on not using explicit casts and instead
preserving the original type. A couple of tweaks were required to
make this work.
Reviewed by: brooks, kib, markj
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D48450
|
| |
|
|
|
|
|
|
|
|
| |
This macro has not been in use since commit "inline atomics and allow tied
modules to inline locks" (r335873, f4b3640475cec929).
Reviewed by: markj, kib, emaste, imp
MFC after: 5 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D48061
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In order to match reality, allow using these functions with pointers on
const objects, and bring us closer to C11.
Remove the '+' modifier in the atomic_load_acq_64_i586()'s inline asm
statement's constraint for '*p' (the value to load). CMPXCHG8B always
writes back some value, even when the value exchange does not happen in
which case what was read is written back. atomic_load_acq_64_i586()
further takes care of the operation atomically writing back the same
value that was read in any case. All in all, this makes the inline
asm's write back undetectable by any other code, whether executing on
other CPUs or code on the same CPU before and after the call to
atomic_load_acq_64_i586(), except for the fact that CMPXCHG8B will
trigger a #GP(0) if the memory address is part of a read-only mapping.
This unfortunate property is however out of scope of the C abstract
machine, and in particular independent of whether the 'uint64_t' pointed
to is declared 'const' or not.
Approved by: markj (mentor)
MFC after: 5 days
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D46887
|
| |
|
|
|
|
|
|
|
|
| |
For current architectures, these are just aliases for the existing
operation on the relevant scalar integer.
Reviewed by: imp, kib
Obtained from: CheriBSD
Sponsored by: AFRL, DARPA
Differential Revision: https://reviews.freebsd.org/D47631
|
| |
|
|
|
|
|
|
|
|
|
| |
All architectures enable NEW_PCIB in DEFAULTS (arm being the most recent
to do so in 121be555997b (arm: Set NEW_PCIB in DEFAULTS rather than a
subset of kernel configs")), so it's time we removed the legacy code
that no longer sees much testing and has a significant maintenance
burden.
Reviewed by: jhb, andrew, emaste
Differential Revision: https://reviews.freebsd.org/D32954
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
- Add missing cc clobber to __PCPU_ADD (which is currently unused).
- Allow the compiler the opportunity to marginally improve code
generation from __PCPU_PTR by letting it figure out how to do the add
(also removing the addition fixes a missing cc clobber).
- Quiet gcc -Warray-bounds by using constant operands instead of bogus
memory references.
- Remove the struct __s __s temporaries, just cast through the type.
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D45826
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Use a constant input operand instead of an output operand to tell the
compiler about __OFFSETOF_MONITORBUF. If we tell it we are writing to
*(u_int *)__OFFSETOF_MONITORBUF, it rightly complains, but we aren't.
The memory clobber already covers the necessary semantics for the
compiler.
Reviewed by: kib
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D45695
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The idea here is to avoid a memory access and conditional branch per
probe site. Instead, the probe is represented by an "unreachable"
unconditional function call. asm goto is used to store the address of
the probe site (represented by a no-op sled) and the address of the
function call into a tracepoint record. Each SDT probe carries a list
of tracepoints.
When the probe is enabled, the no-op sled corresponding to each
tracepoint is overwritten with a jmp to the corresponding label. The
implementation uses smp_rendezvous() to park all other CPUs while the
instruction is being overwritten, as this can't be done atomically in
general. The compiler moves argument marshalling code and the
sdt_probe() function call out-of-line, i.e., to the end of the function.
Per gallatin@ in D43504, this approach has less overhead when probes are
disabled. To make the implementation a bit simpler, I removed support
for probes with 7 arguments; nothing makes use of this except a
regression test case. It could be re-added later if need be.
The approach taken in this patch enables some more improvements:
1. We can now automatically fill out the "function" field of SDT probe
names. The SDT macros let the programmer specify the function and
module names, but this is really a bug and shouldn't have been
allowed. The intent was to be able to have the same probe in
multiple functions and to let the user restrict which probes actually
get enabled by specifying a function name or glob.
2. We can avoid branching on SDT_PROBES_ENABLED() by adding the ability
to include blocks of code in the out-of-line path. For example:
if (SDT_PROBES_ENABLED()) {
int reason = CLD_EXITED;
if (WCOREDUMP(signo))
reason = CLD_DUMPED;
else if (WIFSIGNALED(signo))
reason = CLD_KILLED;
SDT_PROBE1(proc, , , exit, reason);
}
could be written
SDT_PROBE1_EXT(proc, , , exit, reason,
int reason;
reason = CLD_EXITED;
if (WCOREDUMP(signo))
reason = CLD_DUMPED;
else if (WIFSIGNALED(signo))
reason = CLD_KILLED;
);
In the future I would like to use this mechanism more generally, e.g.,
to remove branches and marshalling code used by hwpmc, and generally to
make it easier to add new tracepoint consumers without having to add
more conditional branches to hot code paths.
Reviewed by: Domagoj Stolfa, avg
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D44483
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The kernel source contains several definitions of an ilog2 function;
some are slower than necessary, and one of them is incorrect.
Elimininate them all and define an ilog2 macro in libkern to replace
them, in a way that is fast, correct for all argument types, and, in a
GENERIC kernel, includes a check for an invalid zero parameter.
Folks at Microsoft have verified that having a correct ilog2
definition for their MANA driver doesn't break it.
Reviewed by: alc, markj, mhorne (older version), jhibbits (older version)
Differential Revision: https://reviews.freebsd.org/D45170
Differential Revision: https://reviews.freebsd.org/D45235
|
| |
|
|
|
|
|
|
|
|
| |
This commit introduces the MINIDUMP_STARTUP_PAGE_TRACKING symbol and
uses it to simplify several instances of a complex preprocessor conditional
for adding pages allocated when bootstraping the kernel to minidumps.
Reviewed by: markj, mhorne
Approved by: markj (mentor)
Differential Revision: https://reviews.freebsd.org/D45085
|
| |
|
|
|
|
| |
- s/slighly/slightly/
MFC after: 1 week
|
| |
|
|
|
| |
Reviewed by: kib (older version)
Differential Revision: https://reviews.freebsd.org/D39921
|
| |
|
|
|
|
|
|
|
| |
These are in fact GPLv2 when distributed with the Linux kernel, but the
license also allows MIT if distributed separately. Add the markers to
avoid interference by automated tools.
Differential Revision: https://reviews.freebsd.org/D32796
Reviewed by: royger
|
| |
|
|
|
|
|
|
| |
A precursor to merging them. The spacing differs quite a bit between
the i386 and amd64 hypercall headers, despite very similar content.
Consistently use tabs instead of spaces.
Reviewed by: royger
|
| |
|
|
|
|
|
|
| |
Remove ancient SCCS tags from the tree, automated scripting, with two
minor fixup to keep things compiling. All the common forms in the tree
were removed with a perl script.
Sponsored by: Netflix
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
All of these used the 'immediately at beginning' variation of the
BSD-2-Clause license. This wasn't intentional, just what I copied from
from a random file in the tree back in 2005. It was not an intentional
decision.
The different arch bus.h files are a mix of BSD-2-Clause and
BSD-4-Clause that have various copyright holders (Charles M. Hannum,
Christopher G. Demetriou, The NetBSD Foundation and KATO Takenori), and
some of the content of these files were likely copied from there.
However, apart from the uncopyrightable interface lines, there are very
few comments. It's unclear if these comments are 'original material'
here to copyright, but to the extent that there is, license it under the
standard BSD-2-Clause copyright that's the norm for the project today.
In any event, the standard BSD-2-Clause is also closer to those
originals.
In addition, FreeBSD uses different type definitions than the original
NetBSD code in part. The comments that were copied have been copied a
lot, but appear in NetBSD's bus.h files in NetBSD 1.3.
While I'm here, assign the copyright, to the extent any exists from me,
to the FreeBSD Foundation. I just cut and pasted these into _bus.h from
the different machine files and those files have a rich history of
modification from the original imports from NetBSD over more than 25
years so it's tricky to say who, exactly, wrote each bit. Given the size
of the files, this seems like the best compromise. Also add an
acknowledgement to the NetBSD 1.3 bus.h files and their authors (there
were no additional FreeBSD authors listed in the various
sys/*/include/bus.h files). Finally, use the SPDX identifier instead of
multiple copies of the text.
Differential Revision: https://reviews.freebsd.org/D42532
Sponsored by: Netflix
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Don't recompute mpte during promotion.
Optimize MADV_WILLNEED on existing superpages.
Standardize promotion conditions across amd64, arm64, and i386.
Stop requiring the accessed bit for superpage promotion.
Tidy up pmap_promote_pde() calls.
Retire PMAP_INLINE. It's no longer used.
Note: Some of these changes are a prerequisite to fixing a panic that
arises when attempting to create a wired superpage mapping by
pmap_enter(psind=1) (as opposed to promotion).
Reviewed by: kib
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D41944
|
| |
|
|
| |
Remove /^/[*/]\s*\$FreeBSD\$.*\n/
|
| |
|
|
| |
Remove /^\s*\*+\s*\$FreeBSD\$.*$\n/
|
| |
|
|
| |
Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Include systm.h in i386's counter.h, so it can use critical_*().
In file included from /usr/src/sys/netinet6/ip6_var.h:250:
In file included from /usr/src/sys/sys/counter.h:37:
./machine/counter.h:191:3: error: call to undeclared function 'critical_enter'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
critical_enter();
^
./machine/counter.h:193:3: error: call to undeclared function 'critical_exit'; ISO C99 and later do not support implicit function declarations [-Werror,-Wimplicit-function-declaration]
critical_exit();
^
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use of compiler builtin ffs/ctz functions will result in optimized
instruction sequences when possible, and fall back to calling a function
provided by the compiler run-time library. We have slowly shifted our
platforms to take advantage of these builtins in 60645781d613 (arm64),
1c76d3a9fbef (arm), 9e319462a03a (powerpc, partial).
Some platforms still rely on the libkern implementations of these
functions provided by libkern, namely riscv, powerpc (ffs*, flsll), and
i386 (ffsll and flsll). These routines are slow, as they perform a
linear search for the bit in question. Even on platforms lacking
dedicated bit-search instructions, such as riscv, the compiler library
will provide better-optimized routines, e.g. by using binary search.
Consolidate all definitions of these functions (whether currently using
builtins or not) to libkern.h. This should result in equivalent or
better performing routines in all cases.
One wart in all of this is the existing HAVE_INLINE_F*** macros, which
we use in a few places to conditionally avoid the slow libkern routines.
These aren't easily removed in one commit. For now, provide these
defines unconditionally, but marked for removal after subsequent
cleanup.
Removal of the now unused libkern routines will follow in the next
commit.
Reviewed by: dougm, imp (previous version)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D40698
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This avoids bloating the kernel image when MAXCPU is large.
A follow-up patch for kgdb and other kernel debuggers is needed since
the stoppcbs symbol is now a pointer. Bump __FreeBSD_version so that
debuggers can use osreldate to figure out how to handle stoppcbs.
PR: 269572
MFC after: never
Reviewed by: mjg, emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39806
|
| |
|
|
|
|
|
|
|
| |
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.
Discussed with: pfg
MFC After: 3 days
Sponsored by: Netflix
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
This existing helper function is preferable to the hand-rolled
calculation of the kstack bounds.
Make some small style improvements while here. Notably, rename every
instance of "r", the return address, to "ra". Tidy the includes in the
affected files.
Reviewed by: jkoshy
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39909
|
| |
|
|
|
|
|
|
|
| |
No functional change.
Reviewed by: jkoshy
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D39904
|
| |
|
|
|
|
| |
Reviewed by: emaste, jhb
Differential Revision: https://reviews.freebsd.org/D39656
MFC after: 1 month
|
| |
|
|
|
| |
Differential Revision: https://reviews.freebsd.org/D39655
MFC after: 1 month
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When vm_map_remove() is called from vm_swapout_map_deactivate_pages()
due to swapout, PKRU attributes for the removed range must be kept
intact. Provide a variant of pmap_remove(), pmap_map_delete(), to
allow pmap to distinguish between real removes of the UVA mappings
and any other internal removes, e.g. swapout.
For non-amd64, pmap_map_delete() is stubbed by define to pmap_remove().
Reported by: andrew
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D39556
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move the xenisrc structure which needs to be shared between the core Xen
interrupt code and architecture-dependent code into a separate header. A
similar situation exists for the NR_EVENT_CHANNELS constant.
Turn xi_intsrc into a type definition named xi_arch to reflect the new
purpose of being an architectural variable for the interrupt source.
This was originally implemented by Julien Grall, but has been heavily
modified. The core side was renamed "intr-internal.h" and is #include'd
by "arch-intr.h" instead of the other way around. This allows the
architecture to add function definitions which use struct xenisrc.
The original version only moved xi_intsrc into xen_arch_isrc_t. Moving
xi_vector was done by the submitter.
The submitter had also moved xi_activehi and xi_edgetrigger into
xen_arch_isrc_t. Those disappeared with the removal of PVHv1 support.
Copyright note. The current xenisrc structure was introduced at
76acc41fb7c7 by Justin T. Gibbs. Traces remain, but the strength of
Copyright claims from before 2013 seem pretty weak.
Reviewed by: royger
Submitted by: Elliott Mitchell <ehem+freebsd@m5p.com>, 2021-03-17 19:09:01
Original implementation: Julien Grall <julien@xen.org>, 2015-10-20 09:14:56
Differential Revision: https://reviews.freebsd.org/D30648
[royger]
- Adjust some line lengths
- Fix comment about NR_EVENT_CHANNELS after movement.
- Use #include instead of symlinks.
|
| |
|
|
|
|
|
|
|
| |
This overlaps the purpose of __XEN_INTERFACE_VERSION__. Remove Xen 3.0.2
compatibility. __XEN_INTERFACE_VERSION__ has compatibility to Xen 3.2.8
enabled. As Xen 3.3 was released almost 15 years ago, it seems unlikely
anyone hasn't updated.
Reviewed by: royger
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Now that the atomic macros are always genuinely atomic on x86, they can
be used for synchronization with Xen. A single core VM isn't too
unusual, but actual single core hardware is uncommon.
Replace an open-coding of evtchn_clear_port() with the inline.
Substantially inspired by work done by Julien Grall <julien@xen.org>,
2014-01-13 17:40:58.
Reviewed by: royger
MFC after: 1 week
|
| |
|
|
|
|
|
| |
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D38320
|
| |
|
|
|
|
|
| |
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D38320
|
| |
|
|
|
|
| |
Reviewed by: kib
Differential Revision: https://reviews.freebsd.org/D38289
MFC after: 1 week
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a <sys/_pv_entry.h> intended for use in <machine/pmap.h> to
define struct pv_entry, pv_chunk, and related macros and inline
functions.
Note that powerpc does not yet use this as while the mmu_radix pmap
in powerpc uses the new scheme (albeit with fewer PV entries in a
chunk than normal due to an used pv_pmap field in struct pv_entry),
the Book-E pmaps for powerpc use the older style PV entries without
chunks (and thus require the pv_pmap field).
Suggested by: kib
Reviewed by: kib
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D36685
|
| |
|
|
|
|
|
|
| |
This matches the return type of pmap_mapdev/bios.
Reviewed by: kib, markj
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D36548
|