Running Windows on FreeBSD using Bhyve

Yes, it’s possible since beginning of October if you run FreeBSD 11-CURRENT.

I have waited for quite a while for this. This means no more need for KVM or XEN or what have you.
Bhyve does it all.

It’s a bit rough around the edges. Biggest issue (at least for me) is the lack of video support but serial console is there and that is enough to get going. The documentation on windows installation is a bit lacking, but this will probably change quickly as it only just got pushed to CURRENT.

I have had it running for a day now and it seems stable to me.

Currently all the docs regarding Windows on Bhyve that are available (that I’m aware off) are here: https://people.freebsd.org/~grehan/bhyve_uefi/

And really that is enough to get you going and if you have had some experience with bhyve before then you’re probably going to be ok.

These are my notes on getting Windows 2012 R2 running on Bhyve.

First, make sure bhyve is usable on your machine. Try booting some linux vm as a test.

Get an installation ISO.
I happen to have an MSDN subscription so this is where I got mine.

I used en_windows_server_2012_r2_with_update_x64_dvd_4065220.iso

You will have to modify the ISO, so make sure you have the tools for that:

pkg install cdrtools-devel p7zip  

Since there is no video output, the installation has to happen automatically. Luckily windows does support this. What you need is a Autounattend.xml file to be present in the root of your installation ISO file.

There are excellent examples available on github. So let’s clone that:

git clone https://github.com/nahanni/bhyve-windows-unattend-xml  

I used win2012r2_AutoUnattend.xml file and the only modification I did was adding the product key:
Find the line

<ProductKey />  

And replace it with actual key:

<ProductKey>  
<WillShowUI>OnError</WillShowUI>  
 <Key>YOUR KEY GOES HERE</Key>
</ProductKey>  

Then rename the file to AutoUnattend.xml and keep it as you’ll need it in a moment. I kept mine in ~/

Next step, get the virtio drivers:

fetch https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.96/virtio-win-0.1.96.iso  

Extract the windows installation ISO. I extract mine into destination directory win2k12r2 like so:

7z -owin2k12r2 x en_windows_server_2012_r2_with_update_x64_dvd_4065220.iso  

change into this directory and copy over the prepared AutoUnattend.xml file so that it would now be in the root of the unpacked ISO:

cd win2k12r2  
cp ~/AutoUnattend.xml .  

now, you need to add the virtio drivers.
Again, while being in the root of the unpacked ISO, create a directory virtio and unpack the virtio drivers there:

mkdir virtio  
tar xfv ~/virtio-win-0.1.96.iso  

These are all the modifications needed, so you can go ahead and re-pack the ISO:

mkisofs \  
    -b boot/etfsboot.com -no-emul-boot -c BOOT.CAT \
    -iso-level 4 -J -l -D \
    -N -joliet-long \
    -relaxed-filenames -v \
    -V "Custom" -udf \
    -boot-info-table -eltorito-alt-boot -eltorito-platform 0xEF \
    -eltorito-boot efi/microsoft/boot/efisys_noprompt.bin \
    -no-emul-boot \
    -o ~/win_repack.iso .

Now, one last dependency before you boot it, you need the UEFI binary as modern Windows versions require UEFI:

fetch  http://people.freebsd.org/~grehan/bhyve_uefi/BHYVE_UEFI_20151002.fd  

Create a file that will serve as the disk for your Windows VM

truncate -s 30G win.img  

Make sure that nmdm module is loaded as we’ll use serial console:

kldload nmdm  

You can now start the installation:

bhyve \  
      -c 2 \
      -s 0,hostbridge \
      -s 3,ahci-hd,win.img \
      -s 4,ahci-cd,win_repack.iso \
      -s 10,virtio-net,tap0 \
      -s 31,lpc \
      -l com1,/dev/nmdm0A \
      -l com2,/dev/nmdm1A \
      -l bootrom,BHYVE_UEFI_20151002.fd \
      -m 2G -H -w \
      windows

Once the bhyve process is running, you can connect to the serial port from another console:

cu -l /dev/nmdm0B -s 9600  

Once connected, you’ll find yourself in Special Administration Console. Type ch to get a list of channels and once the installation has started you can follow the progress by connecting to channel #1 by typing:

ch -si 1  

The first stage of installation will copy over the files and eventually the machine will reboot. At this point the bhyve process will exit and you’ll have to restart it. This time, do it without attaching the ISO:

bhyve \  
      -c 2 \
      -s 0,hostbridge \
      -s 3,ahci-hd,win.img \
      -s 10,virtio-net,tap0 \
      -s 31,lpc \
      -l com1,/dev/nmdm0A \
      -l com2,/dev/nmdm1A \
      -l bootrom,BHYVE_UEFI_20151002.fd \
      -m 2G -H -w \
      windows

This stage again takes a bit of time but eventually you are done.

I ran vmstat to see when it settles down.

Once it has, you can type i in the SAC console to find out the IP address that your new Windows VM got from the DHCP and then you can connect over RDP.

The default username is Administrator and the default password is Test123 but you can change that in the AutoUnattend.xml

That’s it. Can’t say that it’s straightforward, but it’s certainly doable and if you’ve done it once then it all makes sense.