Open Stack

Like Gabe, I'd like to also -1 the idea of packing everything new into Heat.

The Snapshot scheduling concern is a feature specific to Nova, and should be handled as such. That's what API extensions are for. If they are demonstrated to be widely used, then they become community extensions, and then finally core. If there are similar solutions requires across projects, they should be moved into Oslo, and when it makes sense, API services should be offered to front-end them.

It would be convenient if we had a generalized scheduling mechanism, but we don't yet. Not even in Heat. For Heat to be successful, we'll need to resist some temptations to creep up scope there. I think this is probably one of those times.

The concept in https://wiki.openstack.org/wiki/Convection expressed as "A persistent job/process (for example an Auto-Scale policy) that remains running until manually terminated" is a component that could be used as the basis for a generalized event scheduler, but it does not yet exist. When that concept materializes, we can collect a few things around Openstack, and consolidate them on a common solution in order to simplify matters overall. This same concept may be used to create an event scheduling API that could be exposed not only to OpenStack projects but to openstack users (along the lines of a Cloud Cron). I see persistent task flows as a reasonable prerequisite for going down that path.

Adrian


On May 1, 2013, at 10:09 AM, Gabe Westmaas <gabe.westmaas at RACKSPACE.COM>
 wrote:

> Thanks Russell,
> 
> However, I'm going to throw in a -1 for this being in Heat.  I definitely understand wanting to keep nova clean, and I could be convinced on that point, but it seems that we are rushing to throw everything in new in Heat, and that doesn't seem like what we want either.
> 
> Scheduled images has three components:
> 1) An API extension
> 2) A scheduling mechanism
> 3) A process for taking snapshots
> 
> The idea is to have an interface to support a feature that has been requested by a few different groups (scheduled images) through a relatively stable API (scheduled images extension) while not locking down the back end implementation (qonos - 2 and 3 above) immediately.  This way, if there is a need for other scheduled actions, we can change the backend implementation.  However, we can keep the scheduled images extension and API as-is, or make sure it only needs to be added to - which is the reason for the extremely minimal interface for the extension.  Another reason to have an API extension is to give operators some control over their nova deployment.  We have found that letting users choose when scheduled snapshots happen mean that everyone picks 2am which leads to one of two scenarios:
> 
> 1) You try to do everything at once, and all snapshots take a long time, and some portion of your network is overloaded.
> 2) You queue things internally in some way and this leads to a bad customer experience.  
> 
> Rather than giving users the impression that they have control over when their snapshot happens, we find value in saying "we will take a snapshot daily, but I can't tell you exactly when it is going to happen."
> 
> We put the extension in nova because it is close to the existing snapshot functionality in nova. I definitely understand the desire to keep things out of nova when it doesn't apply though. 
> 
> As implemented, qonos can be a generic scheduler for anything, not just scheduled images.  It can be replaced with something else that has the same interface, or the interface could change and the extension backend could change as well.  If we think there is value in a separate scheduling service, then lets get a project going to do just that.
> 
> My impression of Heat is that it is for orchestrating user deployments - identifying a number of servers/volumes/etc to be configured, how to configure them, etc.  Which is not really what scheduled images is trying to do.  Heat should be about defining what you want a stack to look like, not about managing complicated workflows (or even, in this case, simple workflows).  If workflow management is what we want, perhaps we should consider this proposal: https://wiki.openstack.org/wiki/Convection (though I don't think workflow implies scheduling either - those should be separated as well).
> 
> Gabe
> 
> On Apr 30, 2013, at 3:23 PM, Russell Bryant <rbryant at redhat.com> wrote:
> 
>> Greetings,
>> 
>> The following blueprint was proposed for the havana series:
>> 
>>   https://blueprints.launchpad.net/nova/+spec/scheduled-images
>> 
>> Based on the current design, I think the nova part of this needs to be
>> deferred.  It does not seem appropriate to add an API extension that
>> talks to a service that is not an integrated project, or at least incubated.
>> 
>> Beyond that, I'm curious about the choice to implement this as a new
>> service.  I definitely agree that this is not something that should be
>> implemented inside of Nova.  However, I wonder if it makes sense as a
>> feature in Heat.  It seems like an orchestration feature.
>> 
>> Thoughts?
>> 
>> -- 
>> Russell Bryant
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev