Open Stack

On Tue, Jun 09, 2015 at 07:52:39PM +0300, Roman Bogorodskiy wrote:
> Hi,
> 
> Few months ago I've started a discussion on FreeBSD host support for
> OpenStack. [1] At a result of discussion it was figured out that there
> are a number of limitations, mainly on the BHyVe (the FreeBSD hypervisor)
> side, that make the effort not feasible.

Do you still intend to add the missing features to BHyve to let it be
supported in Nova eventually too ?

> However, some things changed since then. Specifically, FreeBSD's got Xen
> dom0 support. [2] In context of OpenStack deployment that has a number of
> benefits over bhyve. Specifically:
> 
>  * The stack is more mature and feature-rich
>  * The toolstack is here already: libxl is available through the FreeBSD
>    ports tree, libvirt/libxl works there with minimal modifications
>    (already available in the git master)
>  * OpenStack libvirt/libxl driver is already here
> 
> I was able to setup a proof-of-concept environment on FreeBSD that
> required quite a small amount of modifications required in OpenStack:
> 
>  * Glance and Keystone didn't require any changes
>  * Nove required some minor modifications mainly in the linux_net.py
>    area
> 
> The summary of Nova modifications:
> 
>  * I had to implement FreeBSD version of linux_net.LinuxNetInterfaceDriver.
>    It currently doesn't support vlans though. 
> 
>    https://github.com/novel/bsdstack/blob/master/bsdstack/nova/network/freebsd_net.py
> 
>    I keep it as an external package and configure Nova to use it through
>    linuxnet_interface_driver config option in nova.conf
>  * I had to create a stub for the IptablesManager class. I also had to
>    add a config option to be able to override class for that in a way
>    similar to interface driver.
>  * I had to fix a minor interface incapability for NullL3 stub, that's
>    already in the Nova tree: https://review.openstack.org/#/c/189001/
>  * I added a hack to use 'phy' driver in domain's xml for disks because
>    for some reason driver='qemu' results in guests not able to access
>    disk devices (tried both FreeBSD and Linux guests). Need to
>    investigate
>  * Dropped some LinuxBridgeInterfaceDriver hardcodes in
>    virt.libvirt.vif.
> 
> Here's a quick overview of my changes:
> 
> https://github.com/novel/nova/compare/stable/kilo...novel:stable/kilo_freebsd?expand=1
> 
> With this changes I was able to get things working, i.e. VM starting,
> obtaining IP addresses (with nova-network configured with FlatDHCP) etc.
> 
> Having that said I'm wondering if community is interested in integrating
> FreeBSD support through libvirt/libxl into mainline? Obviously, the
> changes I provided are shortcuts and the appropriate specs should be
> create with proposals of proper designs, not quick hacks like that.

I'd be happy to see FreeBSD support for any of the libvirt hypervisors
added to Nova. As you point out, the changes to the libvirt code are
going to be pretty minimal for libxl, so there's not going to be any
appreciable ongoing maint burden for this. So I see no serious reason
to reject the changes to Nova libvirt driver code for libxl+FreeBSD.
The fun bit will be the changes to non-libvirt code in nova...

> The biggest part of the unportable code, just like it was in bhyve case,
> is still linux_net.py, so probably it makes sense to revive the old
> spec:
> 
> https://review.openstack.org/#/c/127827/
> 
> TBH, IMHO linux_net.py could have a refactor regardless of FreeBSD
> support. It's approx. 2000 lines long, contains a lot of stuff like
> dnsmasq handling code, interface handing code and firewall management
> that could have their own place.

Yes, that network setup code is really awful and I'd love to see
it refactored regardless of FreeBSD support.

With my crystal ball, probably the main question wrt any kind of
FreeBSD support will be that of 3rd party CI testing. I don't know
whether there is any company backing your work who has resources
to provide a CI system, or whether FreeBSD project can manage it
independently ?

The refactoring of the linux_net.py code could be done even without
CI support of course - that would only block the second part where
you actually add a FreeBSD implementation

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|