Open Stack

I guess this isn't a new discussion.  I did some more digging and apparently this is what came out of the last discussion:

https://wiki.openstack.org/wiki/EventScheduler

That definitely seems like it would be something simple we could use, since it only provides scheduling and that's all we need, but it doesn't appear that it's had any traction since that time.  I guess that got absorbed along with Convection into Mistral (is that right?).

I tend to agree with the sentiment that the scheduling component should live outside the workflow service, especially for use cases like this where we just need scheduling and not the workflow portions as we're just scheduling things that are already achievable via single API calls (to my knowledge).

It seems like we basically have 4 options at this point:

1. Wait for/help finish the scheduling component of mistral and use that
2. Build Qonos workers to do the trove needful and integrate with that
3. Build this proposed EventScheduler thing and have trove use that
4. Build something simple internal to trove for now and revisit when things have matured more

Despite my initial enthusiasm about Qonos after it was mentioned earlier today, the more I look into it, the more it seems like the wrong fit.  It could definitely do it, but the way it's structured, it appears that we'd have to add code to qonos/worker for each action we wanted to schedule, which just seems like a pain for what amounts to "make this API call to trove".

My gut says working on EventScheduler is probably the best/most ideal option, but time constraints and what-not make "build it into trove" the most likely course of action for now.

Greg

On Dec 31, 2013, at 10:06 AM, Joshua Harlow <harlowja at yahoo-inc.com<mailto:harlowja at yahoo-inc.com>> wrote:

Agreed taskflow doesn't currently provide scheduling as it was thought that reliable execution that can be restarted and resumed is the foundation that someone using taskflow can easily provide scheduling ontop of... Better IMHO to have a project doing this foundation well (since openstack would benefit from this) than try to pack so many features in that it does none of them well (this kind of kitchen sink approach seems to happen more often than not, sadly).

But in reality it's all about compromises and finding the solution that makes sense and works, and happy new year!! :P

Sent from my really tiny device...

On Dec 30, 2013, at 9:03 PM, "Renat Akhmerov" <rakhmerov at mirantis.com<mailto:rakhmerov at mirantis.com>> wrote:

Greg,

Georgy is right. We’re now actively working on PoC and we’ve already implemented the functionality we initially planned, including cron-based scheduling. You can take a look at our repo and evaluate what we’ve done, we’d be very glad to hear some feedback from anyone potentially interested in Mistral. We were supposed to deliver PoC in the end of December, however, we decided not to rush and include several really cool things that we came up with while working on PoC, they should demonstrate the whole idea of Mistral much better and expose functionality for more potential use cases. A couple of days ago I sent out the information about additional changes in DSL that we want to implement (etherpad: https://etherpad.openstack.org/p/mistral-poc), so if you’d like please join the discussion and let us know how we can evolve the project to better fit your needs. In fact, even though we call it PoC it’s already in a good shape and pretty soon (~1.5 month) is going to be mature enough to use it as a dependency for other projects.

As far as security, we thought about this and and we have a vision of how it could be implemented. Generally, later on we’re planning to implement sort of Role Based Access Control (RBAC) to, first of all, isolate user workbooks (definition of tasks, actions, events) from each other and deal with access patterns to OpenStack services. We would encourage you to file a BP with a description of what would be needed by Trove in that regard.

I looked at https://wiki.openstack.org/wiki/Trove/scheduled-tasks and at the first glance Mistral looks a good fit here, especially if you’re interested in a standalone REST service with its capabilities like execution monitoring, history, language independence and HA (i.e. you schedule backups via Mistral and Trove itself shouldn’t care about availability of any functionality related to scheduling). TaskFlow may also be helpful in case your scheduled jobs are representable as flows using one of TaskFlow patterns. However, in my understanding you’ll have to implement scheduling yourself since TaskFlow does not support it now, at least I didn’t find anything like that (Joshua can provide more details on that).

Thanks.

Renat Akhmerov
@Mirantis Inc.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131231/04577157/attachment.html>