Open Stack

On 18 May 2013 06:42, Joshua Harlow <harlowja at yahoo-inc.com> wrote:
> I was talking about this problem with my management and had a good analogy
> that might make the flaw a little more visible.
>
> Thought experiment: Imagine a VM as a physical box, imagine
> nova/quantum/cinder/glance as the IT personnel.
>
> It seems like the IT personnel would go through the following likely overly
> simplistic steps when setting up a physical box:
>
> 1. Get physical box from supplier (equivalent to nova reserving a VM with
> libvirt) that matches desired specs (aka flavor).
>    a. Place physical box in rack and ensure available power switches and
> so-on.
> 2. Connect hard-drives to physical box (equivalent to downloading the image
> from glance, or talking to cinder to do the same)
> 3. Connect network-cables to physical box (equivalent to talking to quantum,
> having quantum-agent plug things in…)
>    a. Connect network-cables to router, configure router ports/flows and
> so-on.
> 4.  Turn on said box by pressing the power switch/button.
> 5. Report back to user that box is now ready for use.
>
> From the below it seems to be that the 3,4&5 can happen simultaneously (or
> out of order)? That seems pretty bad and is not something I would expect
> when setting up a physical box.

So to tie this into baremetal - I would say it goes like this:
1. Decide what the box should be: how many NICs, how much disk, how
much CPU etc.
2. Ask network folk if they can handle that many NICs and the network
topology wanted.
2. Ask DC folk if they have space for that size box, power supply, heating etc.
2. [optional] Ask SAN folk if they have enough storage of appropriate
types (nearline etc)
3. Either commit the reservations/requests from 2, or rollback.
4. Wheel box in, rack, plug into the allocated ports.
5. Advise network and SAN and DC ops of the machine MAC addresses and
ILO/DRAC details.

I have points here:
A) None of the suppliers act on the reservation until a COMMIT is
received confirming that all parties are happy with the order. Some
corporate workflow systems do this by pipelining the request through
different departments, others do out-of-order speculative stuff,
others pre-supply hardware and network resources in advance of demand,
based on predicted utilisation.
B) The exact physical details - serial number, MAC address, etc -
these are all late-bound and only established once the resources are
all physically hooked up.

So if we want to pre-check stuff to give better errors to users, I
suggest modelling it on the above process: have nova-api check quotas,
quantum check quotas and acls, cinder check quotas and acls, determine
that it is *permitted* and turn it into a reservation, return to the
user, and then async satisfy it.

However, I think for scaling it is a lot better to just return to the
user immediately, then do scatter-gather checks across all services
from the scheduler, and just fix our failure reporting to clearly
identify the failure that occurred against the reservation.

This is a lot better for scaling because it doesn't require a
simultaneous cross-request lock on *every* scarce resource during the
initial API call.

Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services