Open Stack

(Just a warning, my primary area of research is large scale system resource
management, so i have a tendency to descend into scheduling wonkery at the
drop of a hat ;)

Metadata annotations are a great start, but I think that we need to move
the data somewhere else in the long run. We want to be able to push
decision making into nova eventually. For example, our existing
implementation only provides manual load-shedding; eventually, it would be
good if nova itself could decide when to initiate load-shedding -- in
response to new submissions for example. This is probably the simplest
algorithm you'd want to build with this mechanism.

The main issue with using metadata annotations is that they are basically
the wild west. We'll need a controlled vocabulary for the annotations, and
you'd really want to be able to validate them, and assert defaults, etc. If
we want to be able to make decisions from core components, i'd be a lot
more comfortable if the information was always there, as opposed to being
some sort of sparse, optional annotation.

The metadata namespace is also user mutable. If you wanted to drive
variable pricing from this information, the system instantly becomes easily
gamed. Ideally, most of these would be sticky assertions, where some
aspects were immutable, or perhaps state changes were tracked.

The major reason that we implemented this on top of metadata to start with
is that it was a simple place to prototype. We also expect to need to
evolve the model in response to user feedback, and so the current
wild-westy-ness of the implementation is a plus for the moment.
 -nld



On Thu, May 16, 2013 at 7:44 PM, Jonathan Proulx <jon at jonproulx.com> wrote:

> Poncho leads in a very interesting direction.  Among other things I've
> been wanting a way to have "nice" instances that could use space capacity
> but be suspended or deleted when the capacity was needed by higher priority
> instances.
>
> Though curious why existing metadata implementation is insufficient for
> annotation?
>
> -Jon
>
>
> On Thu, May 16, 2013 at 6:50 PM, Narayan Desai <narayan.desai at gmail.com>wrote:
>
>> We've pushed an updated license in.
>>  -nld
>>
>>
>> On Thu, May 16, 2013 at 5:36 PM, Narayan Desai <narayan.desai at gmail.com>wrote:
>>
>>> In the rush to get the code posted concurrently with the paper
>>> submission, we missed this. It should be BSD licensed; that is our legal
>>> department's license of choice. Other licenses are possible, just not
>>> quickly ;)
>>>
>>> Our main intent here is to get the code and ideas out, so the licensing
>>> was a complete oversight.
>>>  -nld
>>>
>>>
>>> On Thu, May 16, 2013 at 5:00 PM, Jeremy Hanmer <
>>> jeremy.hanmer at dreamhost.com> wrote:
>>>
>>>> Any chance the license could get updated to something that would
>>>> encourage contributions?
>>>>
>>>>
>>>> On Thu, May 16, 2013 at 2:16 PM, Tim Bell <Tim.Bell at cern.ch> wrote:
>>>>
>>>>> ** **
>>>>>
>>>>> Poncho would be my nomination for best project name, although Ironic
>>>>> deserves a special award also..****
>>>>>
>>>>> ** **
>>>>>
>>>>> Looks very interesting …****
>>>>>
>>>>> ** **
>>>>>
>>>>> Tim****
>>>>>
>>>>> ** **
>>>>>
>>>>> *From:* Narayan Desai [mailto:narayan.desai at gmail.com]
>>>>> *Sent:* 16 May 2013 22:52
>>>>> *To:* Tim Bell
>>>>> *Cc:* openstack-operators at lists.openstack.org; Scott Devoid; Lorin
>>>>> Hochstein
>>>>> *Subject:* Re: [Openstack-operators] Graceful VM shutdown****
>>>>>
>>>>> ** **
>>>>>
>>>>> We (Scott Devoid, Lorin Hochstein, and myself) have built something to
>>>>> help with this class of problem, and submitted a paper to LISA about it.
>>>>> The paper is still under review, but the code is already up on github. We'd
>>>>> love feedback on the code or approach. ****
>>>>>
>>>>> ** **
>>>>>
>>>>> Our basic goal here was to start working with a user-annotated
>>>>> approach for load-shedding, which will open the door to a bunch of really
>>>>> interesting resource management techniques and reducing the impact of basic
>>>>> system operations tasks.****
>>>>>
>>>>> ** **
>>>>>
>>>>> The code is called poncho (since we needed it to deal with our "full
>>>>> cloud" ;) and is available here:****
>>>>>
>>>>> https://github.com/magellancloud/poncho****
>>>>>
>>>>> ** **
>>>>>
>>>>> It is definitely still under development, but we hope that it will
>>>>> prove out the concepts before we try to build a version that would be
>>>>> suitable for integration into openstack.****
>>>>>
>>>>> ** **
>>>>>
>>>>> I've attached a preprint of the paper to this mail.****
>>>>>
>>>>>  -nld****
>>>>>
>>>>> ** **
>>>>>
>>>>> ** **
>>>>>
>>>>> ** **
>>>>>
>>>>> On Thu, May 16, 2013 at 1:02 PM, Tim Bell <Tim.Bell at cern.ch> wrote:***
>>>>> *
>>>>>
>>>>>  ****
>>>>>
>>>>> I am interested to see how other service providers handle the cases
>>>>> where there is a need to reboot a hypervisor but it is planned (such as a
>>>>> reboot after OS patching).****
>>>>>
>>>>>  ****
>>>>>
>>>>> We do not want to do live or block migration for these cases as there
>>>>> is no shared storage and block migration can be a bit unreliable at times.
>>>>> ****
>>>>>
>>>>>  ****
>>>>>
>>>>> In my ideal case, we would be able to warn the VMs in some way so they
>>>>> can do a reasonable job of shutting down. In some cases, our users would
>>>>> like many minutes of notice to complete their current transaction.****
>>>>>
>>>>>  ****
>>>>>
>>>>> Does anyone know of a standard mechanism for hypervisor communicating
>>>>> to the guest to warn of an impending shutdown ?****
>>>>>
>>>>>  ****
>>>>>
>>>>> Tim****
>>>>>
>>>>>  ****
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-operators mailing list
>>>>> OpenStack-operators at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>>>> ****
>>>>>
>>>>> ** **
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-operators mailing list
>>>>> OpenStack-operators at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>>>>
>>>>>
>>>>
>>>
>>
>> _______________________________________________
>> OpenStack-operators mailing list
>> OpenStack-operators at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20130517/5e058f21/attachment.html>