aboutsummaryrefslogtreecommitdiff
path: root/sys/arm64
Commit message (Collapse)AuthorAgeFilesLines
* Move the literal pool to the end of asm functionsAndrew Turner2020-12-231-1/+1
| | | | | This keeps the data at the end of a function, near to where it's used. Sponsored by: Innovate UK
* Mark all arm64 locore functions with ENTRY/LENTRYAndrew Turner2020-12-231-13/+20
| | | | | | | It is useful to know where these are within the code, and will be needed by later changes. Sponsored by: Innovate UK
* Add LENTRY and LEND to arm64Andrew Turner2020-12-231-3/+6
| | | | | | These allow us to mark local asm functions as a function Sponsored by: Innovate UK
* Use the new PAGE_SIZE_4K in the GICv3 driverAndrew Turner2020-12-231-2/+2
| | | | | | | Stop assuming PAGE_SIZE is 4k in the GICv3 ITS driver. We could use a 16k or 64k page in the future. Sponsored by: Innovate UK
* Rename the arm64 4k PAGE_* macrosAndrew Turner2020-12-231-3/+7
| | | | | | | These now have a _4K suffix to allow us to be explicit when we mean to use a 4k page rather than assuming PAGE_SIZE is 4k. Sponsored by: Innovate UK
* More the arm64 early page tables and stack to .bssAndrew Turner2020-12-231-5/+7
| | | | | | | This removes 806k from the kernel ELF file that is only needed while the kernel is running, not in the static file. Sponsored by: Innovate UK
* Improve address generation in the early arm64 bootAndrew Turner2020-12-231-11/+22
| | | | | | | | | | | | | The adr instruction allows for an address of +-1M from the instruction. If we replace these with an adrp and an add instruction we can generate an address +-4G. The adrp will get an address of the 4k page the label is within, and the add uses the :lo12: prefix to add just the low bits to this address. This will allow us to move things around with fewer issues than if we needed to keep them within the +-1MB range. Sponsored by: Innovate UK
* Skip the vm.pmap.kernel_maps sysctl by default.John Baldwin2020-12-181-1/+1
| | | | | | | | | | | This sysctl node can generate very verbose output, so don't trigger it for sysctl -a or sysctl vm.pmap. Reviewed by: markj, kib Differential Revision: https://reviews.freebsd.org/D27504 Notes: svn path=/head/; revision=368768
* arm64: rk3399: Export the watchdog clockEmmanuel Vadot2020-12-181-0/+5
| | | | | | | | This clock is used by the watchdog IP and can be controlled only in the secure world. Notes: svn path=/head/; revision=368766
* Fix whitespace in comment modified by r368697Jessica Clarke2020-12-161-1/+1
| | | | Notes: svn path=/head/; revision=368699
* Allocate right number of pages for the bounced buffers crossing the page.Michal Meloun2020-12-161-5/+13
| | | | | | | | | | | | | | | | | | | | One of the disadvantages of our current busdma code is the fact that we process the bounced buffer in a page-by-page manner. This means that the short (subpage) buffer allocated across page boundaries is bounced to 2 separate pages. This suboptimal behavior is consistent across all platforms and can be related to (probably unimplementable or incompatible with bouncing) BUS_DMA_KEEP_PG_OFFSET flag. Therefore, allocate one additional page to be fully comply with this requirement. Discused with: markj PR: 251018 Notes: svn path=/head/; revision=368697
* Enable ROUTE_MPATH support in GENERIC kernels.Alexander V. Chernikov2020-12-141-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | Ability to load-balance traffic over multiple path is a must-have thing for routers. It may be used by the servers to balance outgoing traffic over multiple default gateways. The previous implementation, RADIX_MPATH stayed in the shadow for too long. It was not well maintained, which lead us to a vicious circle - people were using non-contiguous mask or firewalls to achieve similar goals. As a result, some routing daemons implementation still don't have multipath support enabled for FreeBSD. Turning on ROUTE_MPATH by default would fix it. It will allow to reduce networking feature gap to other operating systems. Linux and OpenBSD enabled similar support at least 5 years ago. ROUTE_MPATH does not consume memory unless actually used. It enables around ~1k LOC. It does not bring any behaviour changes for userland. Additionally, feature is (temporarily) turned off by the net.route.multipath sysctl defaulting to 0. Differential Revision: https://reviews.freebsd.org/D27428 Notes: svn path=/head/; revision=368648
* Verify (and fix) the context_id argument passed to the mpentry () by PSCI.Michal Meloun2020-12-141-0/+15
| | | | | | | | | | | | Some older PSCI implementations corrupt (or do not pass) the context_id argument to newly started secondary cores. Although the ideal solution to this problem is u-boot update, we can find the correct value for the argument (cpuid) by comparing of real core mpidr register with the value stored in pcu->mpidr. MFC after: 2 weeks Notes: svn path=/head/; revision=368633
* arm64: fix struct l_sigaction_t layoutMitchell Horne2020-12-081-1/+1
| | | | | | | | | | | | | | | The definition was copied from amd64, but the layout of the struct differs slightly between these platforms. This fixes spurious `unsupported sigaction flag 0xXXXXXXXX` messages when executing some Linux binaries on arm64. Reviewed by: emaste MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D27460 Notes: svn path=/head/; revision=368458
* Check that the frame pointer is within the current stack.John Baldwin2020-12-085-21/+21
| | | | | | | | | | | | | | This same check is used on other architectures. Previously this would permit a stack frame to unwind into any arbitrary kernel address (including unmapped addresses). Reviewed by: andrew, markj Obtained from: CheriBSD Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D27362 Notes: svn path=/head/; revision=368455
* Use a macro to find the offset of kern_ttbr0Andrew Turner2020-12-082-1/+2
| | | | | | | | | | Rather than hard coding the offset of kern_ttbr0 within arm64_bootparams use a macro like the other fields. Sponsored by: Innovate UK Notes: svn path=/head/; revision=368445
* Free the arm64 bootparams memory after initarmAndrew Turner2020-12-081-0/+2
| | | | | | | | | | This is only needed in initarm, we can return this memory to the stack used by mi_startup. Sponsored by: Innivate UK Notes: svn path=/head/; revision=368444
* Ensure the boot CPU is CPU 0 on arm64Andrew Turner2020-12-071-39/+66
| | | | | | | | | | | | | | | | | We assume the boot CPU is always CPU 0 on arm64. To allow for this reserve cpuid 0 for the boot CPU in the ACPI and FDT cases but otherwise start the CPU as normal. We then check for the boot CPU in start_cpu and return as if it was started. While here extract the FDT CPU init code into a new function to simplify cpu_mp_start and return FALSE from start_cpu when the CPU fails to start. Reviewed by: mmel Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D27497 Notes: svn path=/head/; revision=368416
* Simplify startup of secondary cores and store MPIDR register to pcpu.Michal Meloun2020-12-053-68/+35
| | | | | | | | | | | | | | | | | | | | | | | | | - record MPIDR for all started cores in pcpu, they will be used as link between physical locality of given core, ID in external description (FDT or ACPI) and cupid. - because of above, cpuid can (and should) be freely assigned, only boot CPU must have cpuid 0. Simplify startup code according this. Please note that pure cpuid is not sufficient instrument to hold any information about core or cluster topology, nor to determistically iterate over subpart of cores in CPU (iterate over all cores in single cluster for example). Situation is more complicated by fact that PSCI can reject start of core without reporting error (because power budget for example), or by fact that is possible that we booted on non-first core in cluster (thus with cpuid 0 assigned to random core). Given cores topology should be exhibited to other parts of system (for example to scheduler for big.little or multicluster systems) by using smp_topo interface. Differential Revision: https://reviews.freebsd.org/D13863 Notes: svn path=/head/; revision=368370
* Add basic support for Freescale LX2160A SoC.Michal Meloun2020-12-054-0/+917
| | | | | | | All peripherals but the network processor are supported. Notes: svn path=/head/; revision=368369
* Connect DTB files based on LX2160A SoC to build.Michal Meloun2020-12-051-1/+1
| | | | | | | Mainly LX2K Honeycomb and ClearFog-CX boards. Notes: svn path=/head/; revision=368367
* Add CFI start/end proc directives to arm64, i386, and ppcConrad Meyer2020-12-053-2/+12
| | | | | | | | | | | | | | Follow-up to r353959 and r368070: do the same for other architectures. arm32 already seems to use its own .fnstart/.fnend directives, which appear to be ARM-specific variants of the same thing. Likewise, MIPS uses .frame directives. Reviewed by: arichardson Differential Revision: https://reviews.freebsd.org/D27387 Notes: svn path=/head/; revision=368354
* Add support for RK3288 SoC.Michal Meloun2020-12-049-1/+1663
| | | | Notes: svn path=/head/; revision=368340
* [arm64] Bump MAXMEMDOM value to 8 to match amd64Oleksandr Tymoshenko2020-12-011-1/+1
| | | | | | | | | | | | | | | On some of the server-grade ARM64 machines the number of NUMA domains is higher than 2. When booting GENERIC kernel on such machines the SRAT parser fails leaving the system with a single domain. To make GENERIC kernel usable on those server, match the parameter value with the one for amd64 arch. Reviewed by: allanjude Differential Revision: https://reviews.freebsd.org/D27368 Sponsored by: Ampere Computing Submitted by: Klara, Inc. Notes: svn path=/head/; revision=368258
* Use uintptr_t instead of uint64_t for pointers in stack frames.John Baldwin2020-12-015-20/+20
| | | | | | | | | | Reviewed by: andrew Obtained from: CheriBSD Sponsored by: DARPA Differential Revision: https://reviews.freebsd.org/D27361 Notes: svn path=/head/; revision=368245
* arm64: Do not rely on SPCR table to detect acpiEmmanuel Vadot2020-11-261-1/+1
| | | | | | | | | | | | | Since EDK2 commit d8e36289cef7bde628b023219cd65fa8e8d4562a, the Graphical console may completely hide SPCR, causing panics later when locating timers. As such simply rely on the ACPI Root pointer presence. Submitted by: dan.kotowski@a9development.com Reviewed by: andrew, mw Differential Revision: https://reviews.freebsd.org/D27306 Notes: svn path=/head/; revision=368081
* arm64: Check if we have a map before checking the flagsEmmanuel Vadot2020-11-241-1/+1
| | | | | | | | This fixes amdgpu on arm64 where linuxkpi is calling id_mapped and we call might_bounce without a map. Notes: svn path=/head/; revision=367984
* Linuxolator: Replace use of eventhandlers by sysent hooks.Konstantin Belousov2020-11-231-0/+3
| | | | | | | | | Reviewed by: markj Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D27309 Notes: svn path=/head/; revision=367960
* Fall back to use the GICR address from the generic interrupt structAndrew Turner2020-11-191-0/+47
| | | | | | | | | | | | | | | | | When there is no ACPI redistributor sub-table in the MADT we need to fall back to use the GICR base address from the GIC CPU interface structure. Handle this fallback when adding memory to the device and when counting the number of redistributors. PR: 251171 Reported by: Andrey Fesenko <f0andrey_gmail.com> Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D27247 Notes: svn path=/head/; revision=367841
* 'make sysent' for r367773Conrad Meyer2020-11-173-6/+60
| | | | | | | X-MFC-With: r367773 Notes: svn path=/head/; revision=367774
* linux(4): Implement name_to_handle_at(), open_by_handle_at()Conrad Meyer2020-11-171-2/+12
| | | | | | | | | They are similar to our getfhat(2) and fhopen(2) syscalls. Differential Revision: https://reviews.freebsd.org/D27111 Notes: svn path=/head/; revision=367773
* Stop calling gic_v3_detach when we haven't called gic_v3_attachAndrew Turner2020-11-171-3/+4
| | | | | | | | | | | The former tries to dereference memory allocated by the latter. If counting the redistributor fails it may try to dereference memory that was never allocated. Sponsored by: Innovate UK Notes: svn path=/head/; revision=367755
* Allow the GICv3 ACPI driver to attach to a GICv4Andrew Turner2020-11-171-2/+9
| | | | | | | | | | | The same driver works on both, allow the driver to attach to a GICv4 controller with the ACPI attachment. Reported by: Andrey Fesenko <f0andrey_gmail.com> Sponsored by: Innovate UK Notes: svn path=/head/; revision=367754
* Introduce IOMMU support for arm64 platform.Ruslan Bukin2020-11-169-0/+3523
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds an arm64 iommu interface and a driver for Arm System Memory Management Unit version 3.2 (ARM SMMU v3.2) specified in ARM IHI 0070C document. Hardware overview is provided in the header of smmu.c file. The support is disabled by default. To enable add 'options IOMMU' to your kernel configuration file. The support was developed on Arm Neoverse N1 System Development Platform (ARM N1SDP), kindly provided by ARM Ltd. Currently, PCI-based devices and ACPI platforms are supported only. The support was tested on IOMMU-enabled Marvell SATA controller, Realtek Ethernet controller and a TI xHCI USB controller with a low to medium load only. Many thanks to Konstantin Belousov for help forming the generic IOMMU framework that is vital for this project; to Andrew Turner for adding IOMMU support to MSI interrupt code; to Mark Johnston for help with SMMU page management; to John Baldwin for explaining various IOMMU bits. Reviewed by: mmel Relnotes: yes Sponsored by: DARPA / AFRL Sponsored by: Innovate UK (Digital Security by Design programme) Differential Revision: https://reviews.freebsd.org/D24618 Notes: svn path=/head/; revision=367736
* imx7gpc: Remove unused functionsEmmanuel Vadot2020-11-161-14/+0
| | | | Notes: svn path=/head/; revision=367722
* arm64: bs_sr_<N> take IIBjoern A. Zeeb2020-11-091-12/+8
| | | | | | | | | | | | In r367327 generic_bs_sr_<n> were derived from mips. Given we are calling generic_bs_w_<n> and no write directly, we do not have to do the address calculations ourselves as eneric_bs_w_<n> will do a str val [bsh, offset]. All we actually have to do is increment offset. MFC after: 3 days Notes: svn path=/head/; revision=367538
* linux(4): Deduplicate unimpl/dummy syscall handlersConrad Meyer2020-11-052-167/+63
| | | | | | | | | | No functional change. Reviewed by: emaste, trasz Differential Revision: https://reviews.freebsd.org/D27099 Notes: svn path=/head/; revision=367395
* Stop trying to bounce in memory allocated by bus dmaAndrew Turner2020-11-051-0/+4
| | | | | | | | | | | | | Memory allocated by bus_dmamem_alloc will take into account any alignment requirements of the CPU it's running on. Stop trying to bounce in this case as there is no bounce zone allocated. Reported by: manu, tuexen Tested by: manu Sponsored by: Innovate UK Notes: svn path=/head/; revision=367365
* arm64: implement bs_sr_<N>Bjoern A. Zeeb2020-11-041-4/+48
| | | | | | | | | | | | | | | | Implement the bs_sr_<N> generic functions based on the generic mips implementation calling the generic bs_w_<N> functions in a loop. ral(4) (rt2860.c) panics in RAL_SET_REGION_4() because bs_sr_4() is NULL. It seems ral(4) and ti(4) might be the only consumers of these functions I could find quickly so keeping them in C rather than asm. Reported by: Steve Wheeler (https://redmine.pfsense.org/issues/11021) Reviewed by: mmel MFC after: 3 days Notes: svn path=/head/; revision=367327
* Add the pmap.h changes missed in r367320Andrew Turner2020-11-041-1/+3
| | | | | | | | Reported by: bz Sponsored by: Innovate UK Notes: svn path=/head/; revision=367325
* Allow the creation of 3 level page tables on arm64Andrew Turner2020-11-041-7/+44
| | | | | | | | | | | | | | | | | | The stage 2 arm64 page tables may need to start at a lower level. This is because we may only be able to map a limited IPA range and trying to use a full 4 levels will cause the CPU to fault in an unrecoverable way. To simplify the code we still allocate the full 4 levels, however level 0 will only ever be used to find the level 1 table used as the base. Handle this by creating a dummy entry in the level 0 table to point to the level 1 table. Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D26066 Notes: svn path=/head/; revision=367320
* Add routines for ARM System MMU (SMMU) pmap management.Ruslan Bukin2020-11-022-0/+184
| | | | | | | | | | Reviewed by: markj Discussed with: kib Sponsored by: DARPA, Innovate UK Differential Revision: https://reviews.freebsd.org/D26877 Notes: svn path=/head/; revision=367282
* Tidy up the #includes. Recent changes, such as the introduction ofAlan Cox2020-11-021-4/+0
| | | | | | | | | | | VM_ALLOC_WAITOK and vm_page_unwire_noq(), have eliminated the need for many of the #includes. Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D27052 Notes: svn path=/head/; revision=367281
* Improve loading of multipage aligned buffers.Michal Meloun2020-11-021-15/+37
| | | | | | | | | | | | | | | | | | | | The multipage alignment requirements is incompatible with many aspects of actual busdma code. Multi-page alignment requests are incompatible with many aspects of current busdma code. Mainly with partially bounced buffer segments and per-page loop in bus_dmamap_load_buffer(). Because proper implementation would be a major restructuring of the code, add the fix only for already known uses and do KASSERT for all other cases. For this reason, bus_dmamap_load_buffer () should take the memory allocated by bus_dmam_alloc () as one segment bypassing per page segmentation. We can do this because it is guaranteed that the memory is physically continuous. Reviewed by: bz Tested by: imp, mv, daniel.engberg.lists_pyret.net, kjopek_gmail.com Differential Revision: https://reviews.freebsd.org/D26735 Notes: svn path=/head/; revision=367268
* o Add the domain member to the struct bus_dma_tag_common as required byRuslan Bukin2020-10-271-0/+2
| | | | | | | | | | busdma_iommu.c. o Add tag_set_domain() pointer to the struct bus_dma_impl as well. Sponsored by: Innovate DSbD Notes: svn path=/head/; revision=367085
* Take the ITS device lock around gicv3_its_release_irqsrc() since thatRuslan Bukin2020-10-271-0/+2
| | | | | | | | | | | | function checks that the mutex lock is owned. This fixes 'devctl disable re0' operation. Sponsored by: Innovate DSbD Differential Revision: https://reviews.freebsd.org/D26904 Notes: svn path=/head/; revision=367084
* arm64: Remove a racy KASSERT from pmap_remove_pages()Mark Johnston2020-10-271-2/+0
| | | | | | | | | | | | | | PCPU_GET(curpmap) expands to multiple instructions on arm64, and if the current thread is migrated in between execution of those instructions, a stale value may be used in the assertion condition. Diagnosed by: mmel Reported by: mmel, Bob Prohaska <fbsd@www.zefox.net> Submitted by: alc MFC after: 1 week Notes: svn path=/head/; revision=367081
* Fix misnomer - linux_to_bsd_errno() does the exact opposite.Edward Tomasz Napierala2020-10-271-4/+2
| | | | | | | | | | Reported by: arichardson MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D26965 Notes: svn path=/head/; revision=367079
* audit: also correctly audit linux_execve()Kyle Evans2020-10-261-0/+3
| | | | | | | | | | Linux execve() gets audited as AUE_EXECVE as well, we should also interpret the return from this correctly for the same reasoning as in r367002. MFC with: r367002 Notes: svn path=/head/; revision=367060
* Add IOMMU support to GICv3 Interrupt Translation Service (ITS) driver.Ruslan Bukin2020-10-251-1/+52
| | | | | | | | | Submitted by: andrew Sponsored by: Innovate DSbD Differential Revision: https://reviews.freebsd.org/D26878 Notes: svn path=/head/; revision=367037
generated by cgit v1.3 (git 2.53.0) at 2026-04-09 17:01:33 +0000