Open Stack

On 04/12/2016 09:17 AM, Jim Rollenhagen wrote:
> On Thu, Apr 07, 2016 at 02:42:09AM +0000, Jeremy Stanley wrote:
>> On 2016-04-06 18:33:06 +0300 (+0300), Igor Belikov wrote:
>> [...]
>>> I suppose there are security issues when we talk about running
>>> custom code on bare metal slaves, but I'm not sure I understand
>>> the difference from running custom code on a virtual machine if
>>> bare metal nodes are isolated, don't contain any sensitive data
>>> and follow a regular redeployment procedure.
>> [...]
>>
>> With a virtual machine, you can delete it and create a new one.
>> Nothing remains behind.
>>
>> With a physical machine, arbitrary code running in the scope of a
>> test with root access can do _nasty_ things like backdoor your
>> server firmware with shims that even masquerade as the firmware
>> updater and persist through redeployments that include firmware
>> refreshes.
>>
>> Physical servers persist, and are therefore vulnerable in this
>> scenario in ways which virtual servers are not.
> 
> Right, it's a huge effort to run a secure bare metal cloud running
> arbitrary code. Homogenous hardware and vendor cooperation is a must,
> and that's only part of it.
> 
> I don't foresee the infra team having the resources to take on such a
> task any time soon (but of course, I'm not well-informed on the infra
> team's workload).
> 
> Another option for baremetal in the gate is baremetal flavors in other
> public clouds - Rackspace has one (OnMetal) but doesn't yet support
> custom images, and others have launched or are working on one. Once
> there's two clouds that support baremetal with custom images, we could
> put those resources in the upstream CI pool.

Depending on exactly what you need baremetal for, we're getting very
close to OVB[1] being usable in an unmodified cloud, especially for
one-time-use CI environments.  I just merged [2] from Steve Baker which
enables pxe booting without Nova hacks, and I've done some successful
tests locally using the Neutron port-security extension to allow PXE
deployment of instances.  The port-security stuff isn't in the git repo
yet because we need to make it compatible with Kilo-based clouds, but
Steve tells me has a way to make that work.

This obviously doesn't help with the nested virt problem, if that's what
you need baremetal for, but for testing baremetal-style deployments it
works quite well in my experience.  We've started work to make use of it
for TripleO CI[3], and it's already being used for some of our
downstream testing.

I don't know that we're quite ready to just run in regular infra yet
because we do need the ability to upload our custom ipxe-boot image and
we need a cloud at least new enough for the port-security to work (and I
don't know exactly how new is new enough, other than it worked in a
Neutron build from a couple of weeks ago).  It also deploys the VMs with
Heat, so we need that in addition to all the other usual suspects.

For the moment, our plan in TripleO is to re-deploy our rack with an
OVB-friendly cloud and stay separate, but I believe eventually we'd like
to run in a regular infra environment and throw that hardware into the
infra pool (don't quote me on this, I don't have any direct control over
it, but this is my understanding of the plan).  We're way closer to
being able to do that than I had thought a month ago, so I wanted to
bring it up as part of this discussion.

1: https://github.com/cybertron/openstack-virtual-baremetal
2:
https://github.com/cybertron/openstack-virtual-baremetal/commit/915269adc73475c1ee6ac722534386ef5dc0250c
3: https://review.openstack.org/#/c/295243

-Ben