Open Stack

      injection thread on openstack-dev         
      Jarrod Johnson                            
      to:                                       
      Jarrod B Johnson                          
      05/12/2013 10:17 AM                       
      Hide Details                              
      From: Jarrod Johnson                      
      <jarrod.b.johnson at gmail.com>              
      To: Jarrod B Johnson/Raleigh/IBM at IBMUS    
      Default custom expiration date:           
      05/12/2014                                
                                                



Upon further consideration, I realized I wasn't thinking enough about the
status quo for virtualization and also conflating a few things.  As such,
I'd like to correct, refine, and clarify my thoughts.

I am going to presume that virtualization guests will continue to use a
block-oriented strategy for boot volumes.  This is by far the most
ubiquitous strategy.  In my experience container solutions are not done
this way and I have also helped at least one enterprise implement an
nfsroot option for tenants of their private cloud for their virtualization
platform.  I appreciate the philosophy that fs on block on fs on block that
is the norm is an awkward reality, but I'm going to ignore that viewpoint
for simplicity.

I'm also going to ignore scripted install case as that flow is pretty much
straightforward and the same regardless of a lot of circumstances.  I also
posit that for Xen and ESXi, scripted install makes the most sense.  I will
confess to inadequate knowledge of the BSDs and Solaris.

On the baremetal side, I'm going to assume some sort of BMC.  Conceptually,
the relationship of baremetal to its BMC and a managed switch  (or SDN
enabled switch) can provide something analogous to a guest to its
hypervisor for purposes of this discussion.

So images that apply to baremetal and virtual machines.  I will consider
the flow in three steps:

Getting a new copy of the image into the right place.  For Virtualization
and SAN boot, this is usually done outside the instance.  Whether its CoW
or full copy doesn't matter for this discussion.  For direct attached
storage, it requires that the environment be booted into a
deployment environment   If the payload is block oriented, that deployment
platform need know nothing about the contents of the payload.  However, for
consistency with the next phase, I suggest this environment should be
paired with the class of OS being deployed.  For cases requiring a
deployment environment, file-oriented payload makes the most sense, but for
virtualization and SAN boot it almost never makes sense.  It is less
awkward for direct attached storage to do block imaging than it is for
virtualized and SAN instances to do file-oriented, so block oriented I
suppose wins if you pick just one.

Fixing up the instance to be bootable.  This would include driver
injection.  For UEFI, it can also mean setting boot variables (some do not
opt to create bootx64.efi, and even if they do it might not always be easy
for the platform to find the correct block device, which is a long story).
Here, you must be booting an environment that understands the deployed
target enough to perform those activities, which strongly suggests WinPE
for Windows and Linux for linux distros.  Note this can be of import for
situations that do not require the in-instance deployment (boot form SAN
and Virtual guests) and as such the environment should be capable of being
invoked for driver injection on an already cloned set of contents.  For
example, if someone crafted an image based on vmxnet3 and vmscsi, this
environment would be helpful to modify the instance to run with virtio
storage instead under KVM.

Finally, we come to the more well recognized step: establishing instance
unique configuration and credentials.  This is the one step that requires
the most cooperation from the image contents.  If there is disparity, there
is a high chance an image author fails to test for one or the other scheme,
so ideally this looks as similar as possilbe.  I that the extensible bulk
of it is best done with some sort of SSL protected network scheme (probably
https).  It is the bootstrap of that process that warrants more
consideration.

I consider a reasonable bootstrap payload to be:
current Date and time (certificates are in play after all)
Network configuration
Instance identifier information (whatever you'd want to put in a CSR)
The singular CA certificate the subsequent setup process should trust
(don't trust public CAs when you don't need to).
URL(s) for the instance to check into.
Either:
A certificate and private key
Or (preferably in my mind)
System can generate private key and CSR and push it up the same secure
mechanism the payload come down.  Infrastructure private CA auto-signs CSRs
if they match what was injected.

Everything else could be subsequently handled by less
restricted facilities, meaning this only has to handle a few kilobytes of
well defined data.

The mechanisms I can think of include:
Boot config/ramfs injection:
This really only applies to the relation between the infrastructure and the
deployment/injection environment.  Not directly to the image.  In
environments without either case being applicable, this isn't critical.
This would be a step on the road to file injection for baremetal.


File injection:
For environments that incur the driver injection phase, file injection is
trivial, but otherwise is highly awkward. Requires private key be injected.

Partition injection:
Trivial for scenarios that use either in-instance deployment or
injection, awkward otherwise. Requires private key be injected.

Attaching a config drive:
Trivial for virtualization  possible on select server hardware.  As it
stands, this is not a ubiquitous feature of server hardware, and also it is
implemented in proprietary ways. Requires private key be injected.

Bridge-Filtered Protocol:
Possible today, but requires very peculiar use of protocols that I think
would be unpopular.  Also would not be durable in the face of some
virtualization networking strategies. Can facilitate either private key
strategy.

Having a special purpose dedicated nic with a recognizable fingerprint
(e.g. a certain mac):
Easy in virtualization, possible on select server hardware if the vendors
bother to do it.  Not possible on existing firmware in capable hardware and
impossible on most server hardware designs. Can facilitate either private
key strategy.

Having a special behavior to the networking that assures safety of a layer
3 protocol:
I think this strategy is highly risky.  Possible, sure, but highly error
prone.  Likely unable to do the key publish strategy securely.

Serial port:
I personally think this would be the winner.  In KVM the hypervisor can
host the security bootstrap agent or have it hosted elsewhere.  In server
space, SOL over IPMI 2.0 is bog standard and the bootstrap agent can reside
wherever.  I think for this to not look like a horrible beast, it requires
instrumenting the SOL console handler, which could come cheap if I provide
the native Python IPMI support I'm considering doing.  VMware environments
also provide serial port redirection which would require instrumenting it
off-hypervisor.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130513/3be691b5/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ecblank.gif
Type: image/gif
Size: 45 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130513/3be691b5/attachment.gif>