src/sys/boot/forth, branch release/10.1.0

MFC UEFI loader

2014-09-04T21:01:10Z

This MFC consists of the following SVN revisions: 258741 261568 261603 261668 263115 263117 263968 264078 264087 264088 264092 264095 264115 264132 264208 264261 264262 264263 264319 265028 265057 268974 Detailed commit messages: r258741: Note that libstand is 32-bit on amd64 and powerpc64 r261568: Build libstand as a 64-bit library on amd64 The 32-bit bootloaders now link against libstand.a in sys/boot/libstand32, so there is no need to force /usr/lib/libstand.a to be 32-bit. r261603: Don't force efi to a 32-bit build on amd64 r261668: Build libstand as a 64-bit library on ppc64 The 32-bit bootloaders now link against libstand.a in sys/boot/libstand32, so there is no need to force /usr/lib/libstand.a to be 32-bit. This is equivalent to r261568 for amd64. r263115: Add amd64 EFI headers r263117: Connect 64-bit boot ficl to the build It is not yet used, but this will ensure it doesn't get broken. r263968: Use EFI types for EFI values (silences warnings). EFI UINTN is actually a 64-bit type on 64-bit processors. r264078: Put each source file on a separate line This will simplify rebasing the amd64 UEFI patch set. r264087: Build boot/ficl as 64-bit library on amd64 The 32-bit bootloaders on amd64 now use the 32-bit version in ficl32, as is done with libstand32. The native 64-bit ficl will be used by the upcoming UEFI loader. r264088: Merge efilib changes from projects/uefi r247216: Add the ability for a device to have an "alias" handle. r247379: Fix network device registration. r247380: Adjust our load device when we boot from CD under UEFI. The process for booting from a CD under UEFI involves adding a FAT filesystem containing your loader code as an El Torito boot image. When UEFI detects this, it provides a block IO instance that points at the FAT filesystem as a child of the device that represents the CD itself. The problem being that the CD device is flagged as a "raw device" while the boot image is flagged as a "logical partition". The existing EFI partition code only looks for logical partitions and so the CD filesystem was rendered invisible. To fix this, check the type of each block IO device. If it's found to be a CD, and thus an El Torito boot image, look up its parent device and add that instead so that the loader will then load the kernel from the CD filesystem. This is done by using the handle for the boot filesystem as an alias. Something similar to this will be required for booting from other media as well as the loader will live in the EFI system partition, not on the partition containing the kernel. r247381: Remove a scatalogical debug printf that crept in. r264092: Add -fPIC for amd64 r264095: Support UEFI booting on amd64 via loader.efi This is largely the work from the projects/uefi branch, with some additional refinements. This is derived from (and replaces) the original i386 efi implementation; i386 support will be restored later. Specific revisions of note from projects/uefi: r247380: Adjust our load device when we boot from CD under UEFI. The process for booting from a CD under UEFI involves adding a FAT filesystem containing your loader code as an El Torito boot image. When UEFI detects this, it provides a block IO instance that points at the FAT filesystem as a child of the device that represents the CD itself. The problem being that the CD device is flagged as a "raw device" while the boot image is flagged as a "logical partition". The existing EFI partition code only looks for logical partitions and so the CD filesystem was rendered invisible. To fix this, check the type of each block IO device. If it's found to be a CD, and thus an El Torito boot image, look up its parent device and add that instead so that the loader will then load the kernel from the CD filesystem. This is done by using the handle for the boot filesystem as an alias. Something similar to this will be required for booting from other media as well as the loader will live in the EFI system partition, not on the partition containing the kernel. r246231: Add necessary code to hand off from loader to an amd64 kernel. r246335: Grab the EFI memory map and store it as module metadata on the kernel. This is the same approach used to provide the BIOS SMAP to the kernel. r246336: Pass the ACPI table metadata via hints so the kernel ACPI code can find them. r246608: Rework copy routines to ensure we always use memory allocated via EFI. The previous code assumed it could copy wherever it liked. This is not the case. The approach taken by this code is pretty ham-fisted in that it simply allocates a large (32MB) buffer area and stages into that, then copies the whole area into place when it's time to execute. A more elegant solution could be used but this works for now. r247214: Fix a number of problems preventing proper handover to the kernel. There were two issues at play here. Firstly, there was nothing preventing UEFI from placing the loader code above 1GB in RAM. This meant that when we switched in the page tables the kernel expects to be running on, we are suddenly unmapped and things no longer work. We solve this by making our trampoline code not dependent on being at any given position and simply copying it to a "safe" location before calling it. Secondly, UEFI could allocate our stack wherever it wants. As it happened on my PC, that was right where I was copying the kernel to. This did not cause happiness. The solution to this was to also switch to a temporary stack in a safe location before performing the final copy of the loaded kernel. r246231: Add necessary code to hand off from loader to an amd64 kernel. r246335: Grab the EFI memory map and store it as module metadata on the kernel. This is the same approach used to provide the BIOS SMAP to the kernel. r246336: Pass the ACPI table metadata via hints so the kernel ACPI code can find them. r246608: Rework copy routines to ensure we always use memory allocated via EFI. The previous code assumed it could copy wherever it liked. This is not the case. The approach taken by this code is pretty ham-fisted in that it simply allocates a large (32MB) buffer area and stages into that, then copies the whole area into place when it's time to execute. A more elegant solution could be used but this works for now. r247214: Fix a number of problems preventing proper handover to the kernel. There were two issues at play here. Firstly, there was nothing preventing UEFI from placing the loader code above 1GB in RAM. This meant that when we switched in the page tables the kernel expects to be running on, we are suddenly unmapped and things no longer work. We solve this by making our trampoline code not dependent on being at any given position and simply copying it to a "safe" location before calling it. Secondly, UEFI could allocate our stack wherever it wants. As it happened on my PC, that was right where I was copying the kernel to. This did not cause happiness. The solution to this was to also switch to a temporary stack in a safe location before performing the final copy of the loaded kernel. r247216: Use the UEFI Graphics Output Protocol to get the parameters of the framebuffer. r264115: Fix printf format mismatches r264132: Connect sys/boot/amd64 to the build r264208: Do not build the amd64 UEFI loader with GCC The UEFI loader causes buildworld to fail when building with (in-tree) GCC, due to a typedef redefinition. As it happens the in-tree GCC cannot successfully build the UEFI loader anyhow, as it does not support __attribute__((ms_abi)). Thus, just avoid trying to build it with GCC, rather than disconnecting it from the build until the underlying issue is fixed. r264261: Correct a variable's type for 64-bit Ficl FICL_INT is long. r264262: Fix printf args for 64-bit archs r264263: Add explicit casts to quiet warnings in libefi r264319: Fix EFI loader object tree creation on 9.x build hosts Previously ${COMPILER_TYPE} was checked in sys/boot/amd64, and the efi subdirectory was skipped altogether for gcc (since GCC does not support a required attribute). However, during the early buildworld stages ${COMPILER_TYPE} is the existing system compiler (i.e., gcc on 9.x build hosts), not the compiler that will eventually be used. This caused "make obj" to skip the efi subdirectory. In later build stages ${COMPILER_TYPE} is "clang", and then the efi loader would attempt to build in the source directory. r265028 (dteske): Disable the beastie menu for EFI console ... which doesn't support ANSI codes (so things like `at-xy', `clear', and other commands don't work making it impossible to generate a living menu). r265057 (nwhitehorn): Turn off various fancy instruction sets... as well as deduplicate some options. This makes the EFI loader build work with CPUTYPE=native in make.conf on my Core i5. r268974 (sbruno): Supress clang warning for FreeBSD printf %b and %D formats Relnotes: Yes Sponsored by: The FreeBSD Foundation

MFC r266938:

2014-06-03T14:50:51Z

Allow customization of the brand displayed in the boot menu. If the user specifies in /boot/loader.conf: loader_brand="mycustom-brand" Then "mycustom-brand" will be executed instead of "fbsd-logo". Submitted by: alfred Obtained from: FreeNAS

MFC r258270:

2014-03-03T07:31:55Z

Add a try-include word (which acts the same as "include") and use it to conditionally include (but ignore failures) /boot/loader.rc.local and /boot/menu.rc.local -- to make customizing the menu easier. Reviewed by: alfred Discussed on: -hackers

MFC r258269: Refactor draw-beastie function.

2014-03-03T07:28:56Z

Discussed on: -hackers

MFC r257821:

2014-03-03T07:24:50Z

Extend loader_delay as-documented in beastie.4th(8) and delay.4th(8), making it available to architectures that do not use or load the beastie menu. This is reported to save headaches on some PPC systems where unload followed by load does not produce the desired results wherein if-given the opportunity to abort the initial loading sequence, you can customize the first load. Reviewed by: nwhitehorn, kan Discussed on: -current

MFC r257650:

2014-03-03T07:16:39Z

Defer loading of kernel and modules if the beastie menu is enabled. Add a kernel selection menu to the beastie menu. List of kernels is taken from `kernels' in loader.conf(5) as a space (or comma) separated list of names to display (up to 9). If not set, default value is "kernel kernel.old". Does not validate that kernels exist because the next enhancement will be to allow selection of the root device. Discussed on: -current

Merge from project branch via main. Uninteresting commits are trimmed.

2013-10-12T15:31:36Z

Refactor of /dev/random device. Main points include: * Userland seeding is no longer used. This auto-seeds at boot time on PC/Desktop setups; this may need some tweeking and intelligence from those folks setting up embedded boxes, but the work is believed to be minimal. * An entropy cache is written to /entropy (even during installation) and the kernel uses this at next boot. * An entropy file written to /boot/entropy can be loaded by loader(8) * Hardware sources such as rdrand are fed into Yarrow, and are no longer available raw. ------------------------------------------------------------------------ r256240 | des | 2013-10-09 21:14:16 +0100 (Wed, 09 Oct 2013) | 4 lines Add a RANDOM_RWFILE option and hide the entropy cache code behind it. Rename YARROW_RNG and FORTUNA_RNG to RANDOM_YARROW and RANDOM_FORTUNA. Add the RANDOM_* options to LINT. ------------------------------------------------------------------------ r256239 | des | 2013-10-09 21:12:59 +0100 (Wed, 09 Oct 2013) | 2 lines Define RANDOM_PURE_RNDTEST for rndtest(4). ------------------------------------------------------------------------ r256204 | des | 2013-10-09 18:51:38 +0100 (Wed, 09 Oct 2013) | 2 lines staticize struct random_hardware_source ------------------------------------------------------------------------ r256203 | markm | 2013-10-09 18:50:36 +0100 (Wed, 09 Oct 2013) | 2 lines Wrap some policy-rich code in 'if NOTYET' until we can thresh out what it really needs to do. ------------------------------------------------------------------------ r256184 | des | 2013-10-09 10:13:12 +0100 (Wed, 09 Oct 2013) | 2 lines Re-add /dev/urandom for compatibility purposes. ------------------------------------------------------------------------ r256182 | des | 2013-10-09 10:11:14 +0100 (Wed, 09 Oct 2013) | 3 lines Add missing include guards and move the existing ones out of the implementation namespace. ------------------------------------------------------------------------ r256168 | markm | 2013-10-08 23:14:07 +0100 (Tue, 08 Oct 2013) | 10 lines Fix some just-noticed problems: o Allow this to work with "nodevice random" by fixing where the MALLOC pool is defined. o Fix the explicit reseed code. This was correct as submitted, but in the project branch doesn't need to set the "seeded" bit as this is done correctly in the "unblock" function. o Remove some debug ifdeffing. o Adjust comments. ------------------------------------------------------------------------ r256159 | markm | 2013-10-08 19:48:11 +0100 (Tue, 08 Oct 2013) | 6 lines Time to eat crow for me. I replaced the sx_* locks that Arthur used with regular mutexes; this turned out the be the wrong thing to do as the locks need to be sleepable. Revert this folly. # Submitted by: Arthur Mesh (In original diff) ------------------------------------------------------------------------ r256138 | des | 2013-10-08 12:05:26 +0100 (Tue, 08 Oct 2013) | 10 lines Add YARROW_RNG and FORTUNA_RNG to sys/conf/options. Add a SYSINIT that forces a reseed during proc0 setup, which happens fairly late in the boot process. Add a RANDOM_DEBUG option which enables some debugging printf()s. Add a new RANDOM_ATTACH entropy source which harvests entropy from the get_cyclecount() delta across each call to a device attach method. ------------------------------------------------------------------------ r256135 | markm | 2013-10-08 07:54:52 +0100 (Tue, 08 Oct 2013) | 8 lines Debugging. My attempt at EVENTHANDLER(multiuser) was a failure; use EVENTHANDLER(mountroot) instead. This means we can't count on /var being present, so something will need to be done about harvesting /var/db/entropy/... . Some policy now needs to be sorted out, and a pre-sync cache needs to be written, but apart from that we are now ready to go. Over to review. ------------------------------------------------------------------------ r256094 | markm | 2013-10-06 23:45:02 +0100 (Sun, 06 Oct 2013) | 8 lines Snapshot. Looking pretty good; this mostly works now. New code includes: * Read cached entropy at startup, both from files and from loader(8) preloaded entropy. Failures are soft, but announced. Untested. * Use EVENTHANDLER to do above just before we go multiuser. Untested. ------------------------------------------------------------------------ r256088 | markm | 2013-10-06 14:01:42 +0100 (Sun, 06 Oct 2013) | 2 lines Fix up the man page for random(4). This mainly removes no-longer-relevant details about HW RNGs, reseeding explicitly and user-supplied entropy. ------------------------------------------------------------------------ r256087 | markm | 2013-10-06 13:43:42 +0100 (Sun, 06 Oct 2013) | 6 lines As userland writing to /dev/random is no more, remove the "better than nothing" bootstrap mode. Add SWI harvesting to the mix. My box seeds Yarrow by itself in a few seconds! YMMV; more to follow. ------------------------------------------------------------------------ r256086 | markm | 2013-10-06 13:40:32 +0100 (Sun, 06 Oct 2013) | 11 lines Debug run. This now works, except that the "live" sources haven't been tested. With all sources turned on, this unlocks itself in a couple of seconds! That is no my box, and there is no guarantee that this will be the case everywhere. * Cut debug prints. * Use the same locks/mutexes all the way through. * Be a tad more conservative about entropy estimates. ------------------------------------------------------------------------ r256084 | markm | 2013-10-06 13:35:29 +0100 (Sun, 06 Oct 2013) | 5 lines Don't use the "real" assembler mnemonics; older compilers may not understand them (like when building CURRENT on 9.x). # Submitted by: Konstantin Belousov ------------------------------------------------------------------------ r256081 | markm | 2013-10-06 10:55:28 +0100 (Sun, 06 Oct 2013) | 12 lines SNAPSHOT. Simplify the malloc pools; We only need one for this device. Simplify the harvest queue. Marginally improve the entropy pool hashing, making it a bit faster in the process. Connect up the hardware "live" source harvesting. This is simplistic for now, and will need to be made rate-adaptive. All of the above passes a compile test but needs to be debugged. ------------------------------------------------------------------------ r256042 | markm | 2013-10-04 07:55:06 +0100 (Fri, 04 Oct 2013) | 25 lines Snapshot. This passes the build test, but has not yet been finished or debugged. Contains: * Refactor the hardware RNG CPU instruction sources to feed into the software mixer. This is unfinished. The actual harvesting needs to be sorted out. Modified by me (see below). * Remove 'frac' parameter from random_harvest(). This was never used and adds extra code for no good reason. * Remove device write entropy harvesting. This provided a weak attack vector, was not very good at bootstrapping the device. To follow will be a replacement explicit reseed knob. * Separate out all the RANDOM_PURE sources into separate harvest entities. This adds some secuity in the case where more than one is present. * Review all the code and fix anything obviously messy or inconsistent. Address som review concerns while I'm here, like rename the pseudo-rng to 'dummy'. # Submitted by: Arthur Mesh (the first item) ------------------------------------------------------------------------ r255319 | markm | 2013-09-06 18:51:52 +0100 (Fri, 06 Sep 2013) | 4 lines Yarrow wants entropy estimations to be conservative; the usual idea is that if you are certain you have N bits of entropy, you declare N/2. ------------------------------------------------------------------------ r255075 | markm | 2013-08-30 18:47:53 +0100 (Fri, 30 Aug 2013) | 4 lines Remove short-lived idea; thread to harvest (eg) RDRAND enropy into the usual harvest queues. It was a nifty idea, but too heavyweight. # Submitted by: Arthur Mesh ------------------------------------------------------------------------ r255071 | markm | 2013-08-30 12:42:57 +0100 (Fri, 30 Aug 2013) | 4 lines Separate out the Software RNG entropy harvesting queue and thread into its own files. # Submitted by: Arthur Mesh ------------------------------------------------------------------------ r254934 | markm | 2013-08-26 20:07:03 +0100 (Mon, 26 Aug 2013) | 2 lines Remove the short-lived namei experiment. ------------------------------------------------------------------------ r254928 | markm | 2013-08-26 19:35:21 +0100 (Mon, 26 Aug 2013) | 2 lines Snapshot; Do some running repairs on entropy harvesting. More needs to follow. ------------------------------------------------------------------------ r254927 | markm | 2013-08-26 19:29:51 +0100 (Mon, 26 Aug 2013) | 15 lines Snapshot of current work; 1) Clean up namespace; only use "Yarrow" where it is Yarrow-specific or close enough to the Yarrow algorithm. For the rest use a neutral name. 2) Tidy up headers; put private stuff in private places. More could be done here. 3) Streamline the hashing/encryption; no need for a 256-bit counter; 128 bits will last for long enough. There are bits of debug code lying around; these will be removed at a later stage. ------------------------------------------------------------------------ r254784 | markm | 2013-08-24 14:54:56 +0100 (Sat, 24 Aug 2013) | 39 lines 1) example (partially humorous random_adaptor, that I call "EXAMPLE") * It's not meant to be used in a real system, it's there to show how the basics of how to create interfaces for random_adaptors. Perhaps it should belong in a manual page 2) Move probe.c's functionality in to random_adaptors.c * rename random_ident_hardware() to random_adaptor_choose() 3) Introduce a new way to choose (or select) random_adaptors via tunable "rngs_want" It's a list of comma separated names of adaptors, ordered by preferences. I.e.: rngs_want="yarrow,rdrand" Such setting would cause yarrow to be preferred to rdrand. If neither of them are available (or registered), then system will default to something reasonable (currently yarrow). If yarrow is not present, then we fall back to the adaptor that's first on the list of registered adaptors. 4) Introduce a way where RNGs can play a role of entropy source. This is mostly useful for HW rngs. The way I envision this is that every HW RNG will use this functionality by default. Functionality to disable this is also present. I have an example of how to use this in random_adaptor_example.c (see modload event, and init function) 5) fix kern.random.adaptors from kern.random.adaptors: yarrowpanicblock to kern.random.adaptors: yarrow,panic,block 6) add kern.random.active_adaptor to indicate currently selected adaptor: root@freebsd04:~ # sysctl kern.random.active_adaptor kern.random.active_adaptor: yarrow # Submitted by: Arthur Mesh Submitted by: Dag-Erling Smørgrav , Arthur Mesh Reviewed by: des@FreeBSD.org Approved by: re (delphij) Approved by: secteam (des,delphij)

Update copyright.

2013-08-27T06:09:28Z

Building upon SVN r254237, disable automated activation of alternate layouts

2013-08-26T23:37:11Z

and add support for default underride to $loader_version, acting as a way to name a release. Release text is not displayed for the aforementioned feature of alternate display layout (introduced in r254237); however, for all other layouts (incl. default), the release name is displayed at lower-right. See version.4th(8) for additional information and/or historical details. NOTE: Also a minor edit to version.4th(8) while we're here.

Add optional support for default override of standard setup; but only if

2013-08-12T03:52:23Z

corresponding functions are provided. If override function does not exist, boot remains unmodified. This patch should not result in any changes.