1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
|
=== Bhyve Improvements
Links: +
link:https://callfortesting.org/[bhyve production users calls] URL: link:https://callfortesting.org[]
link:https://wiki.freebsd.org/EnterpriseWorkingGroup[FreeBSD Wiki - Enterprise Working Group] URL: link:https://wiki.freebsd.org/EnterpriseWorkingGroup[] +
link:https://wiki.freebsd.org/ChrisMoerz/bhyve_management[FreeBSD Wiki - EWG - bhyve and jails management tooling] URL: link:https://wiki.freebsd.org/ChrisMoerz/bhyve_management[] +
link:http://static.bultmann.eu/s6-talk/#(1)[Jan Bramkamp's work on s6rc] URL:http://static.bultmann.eu/s6-talk/#(1)[] +
link:https://github.com/christian-moerz/vmstated[vmstated on Github] URL: link:https://github.com/christian-moerz/vmstated[] +
link:https://www.youtube.com/watch?v=f60NCrunXyw[YouTube - vmstated explained] URL: link:https://www.youtube.com/watch?v=f60NCrunXyw[]
Contact: Chris Moerz <freebsd@ny-central.org>
==== Bhyve I/O Performance Measurements
Participants of the weekly bhyve production users calls recently discussed bhyve's I/O performance.
Various ways of measuring and comparing were brought up, however it was quickly clear that there is currently no formal analysis and report on this.
So, we started this effort in the hopes of better understanding the various impacts of configuration options for a guest on its I/O performance.
We created a set of shell scripts that harness a FreeBSD guest for running package:benchmarks/fio[] I/O performance measurements under various configurations.
This allows us to compare multiple criteria like bandwidth, latency, IOPS, and more.
So far, we are testing for
* different storage backends (i.e. ahci-hd, nvme, virtio-blk)
* different memory settings
* different CPU pinning options
* different block sizes for the backing storage
* different block sizes for accessing virtual disks
We are also pitting results for different CPU manufacturers against each other and contrasting guest vs host performance to better understand the performance impact of virtualization.
We plan to continue discussing our results during Michael Dexter's weekly bhyve production users call - come join us if you are interested.
We also hope to be able to present the results at EuroBSDCon in Q3.
==== Bhyve Virtual Machine Tooling
Last year, Greg Wallace at the FreeBSD Foundation founded the Enterprise Working Group with the specific goal of addressing pain points of Enterprise users of FreeBSD.
One of the work groups that emerged clustered around bhyve and jails management tooling.
After collecting a set of desired features and functionality, one overarching key point for bhyve emerged: the desire to have configuration concepts and tooling for bhyve like the ones available for jails.
While other desirable features were identified as well, i.e. TPM software emulation and snapshot/restore/host-migration, the conceptual tooling question won over those due to the lower degree of complexity and its clarity on goal and the path on how to take steps towards it.
Technically, this means working out existing gaps around process supervision and virtual machine state management.
First steps were taken by experimenting with existing frameworks (i.e. s6rc work by Jan Bramkamp) and eventually -- through discussions in the weekly bhyve production user's calls (organized by Michael Dexter) -- this led to a proof-of-concept implementation of "vmstated".
Started as an experiment to better understand the problem space of process supervision and virtual machine state handling, vmstated is constructed of a daemon and vmstatedctl management utility.
It is built with base-only tooling and libraries and leverages FreeBSD specific constructs like kqueue to minimize its resource impact.
vmstated is configured via a UCL configuration file (similar to [.filename]#jails.conf#) and -- in combination with a man:bhyve_config|5] configuration file -- already provides highest flexibility in configuring virtual machines.
vmstatedctl provides a jail-like command set to start, stop, and retrieve status information about guests.
State transitions can easily be hooked via shell scripts and allow running additional commands for network or storage set up and tear down when relevant state changes occur.
An initial release is already in ports as package:sysutils/vmstated[] and updates are pending commit; however, the newest version can be found on GitHub.
We are considering expanding the work; we would also like to invite anyone interested to join us in this work!
Patches, suggestions, feedback, etc. are all very much welcome!
If you want to know more about our work, come join us at one of Michael Dexter's weekly bhyve production users calls or reach me mailto:freebsd@ny-central.org[by email].
==== Documentation ===
We managed to update a few parts of the Handbook and Porter's Handbook (thanks to mailto:emaste@FreeBSD.org[Ed Maste], mailto:jrm@FreeBSD.org[Joseph Mingrone], mailto:pauamma@gundo.com[Pau Amma], and mailto:rgrimes@FreeBSD.org[Rodney W. Grimes]):
* several improvements and expansions to the virtualization chapter in the FreeBSD Handbook
** using a man:bhyve_config|5] configuration file
** jailing bhyve
** experimental snapshot and restore feature
** setting up a Windows guest
* we also have a review (link:https://reviews.freebsd.org/D43940[D43940]) up for an initial step to improving the bhyve man page
** this was intentionally started with a structural update first to separate the many `-s` flag options
** once this lands, we can move to a more widespread update to the overall content
Feedback is obviously very welcome -- on the existing content as well as any additional content we should be looking into!
|
