Open Stack

On Dec 12, 2013, at 1:39 PM, devdatta kulkarni <devdatta.kulkarni at rackspace.com> wrote:

> We followed on the Zuul question in this week's git-integration working group meeting.
> 
> mordred has created an etherpad with a high-level description of Zuul and how it might
> fit with Solum't git integration workflow
> 
> https://etherpad.openstack.org/p/ZuulSolum
> 
> The working group seemed to be coming to the consensus that we want to use a single workflow
> engine, as far as possible, for all of Solum's workflow needs.
> This brought up the question about, what are really Solum's workflow requirements. 

Hi

I had a long conversation with Monty yesterday and we flushed out a few things I would like to run by the group.
I have also included answers to the questions below.

> 
> At a high-level, I think that Solum has three different kinds of workflows.
> 
> 1) Workflow around getting user code into Solum
>   - This is the git integration piece being worked out in the git-integration
>     working group.

This is possible using the Zuul workflows. Would potentially require a little work in Zuul.

> 
> 2) Workflow around creating language pack(s).
>   - The main workflow requirement here involves ability to run tests before creating a language pack.
>     There was some discussion in language-pack working group about this requirement.

This is also possible using Zuul and in-fact would benefit Solum by providing config file based build workflows
that could be customized by ops personelle. For e.g.. one DU might require SVN, another might require git 
and a jenkins CI based unit test before triggering Langpack, other DUs might wish to leverage gerrit etc.
This would be possible through Zuul without having to reinvent it on the other workflow engine.

> 
> 3) Workflow around deploying created language pack(s) in order to instantiate an assembly.
>   - The deployment may potentially contain several steps, some of which may be long running, such as
>   populating a database. Further, there may be a need to checkpoint intermediate steps
>   and retry the workflow from the failed point.

This is probably not a very good fit for Zuul. It can handle simple workflow but won’t be able to do the
complex checkpointing, rollback, retry logic etc.

> 
> 
> mordred mentioned that #1 can be achieved by Zuul (both, push-to-solum and pull-by-solum)
> We want to know if #2 and #3 can also be achieved by Zuul.
> If not, we want to know what are the available options.
> 
> mordred, thanks for the etherpad; looking forward to the digram :)


Zuul is workflow engine capable of running simple workflows. It is probably not suitable for all of Solum but would
manage the source -> DU flow quite nicely. Initially my thoughts were that I wanted to avoid having 2 workflow
engines in Solum but there is another way to look at it…

During out F2F, we had said that we should have a Solum API where we could just post DU images. This would
allow someone to build the DU outside Solum and just provide it. We could use this same API as a clean interface to
separated out the DU build flow from the DU deploy flow. Once this is done, the DU build flow (#1, #2 above)
could be cleanly handled by Zuul and the DU deploy flow by whatever complex engine the rest of Solum would
use.

This approach has a few advantages:
	* Re-uses what Openstack already uses but its build & CI process (and potentially makes it better)
	* Allows operations who deploy Solum to customize their build process without having to change Solum
	* Allows us to leverage the Zuul/OpenStack-infra team to help us solve the DU build flow instead of having 
	  to go alone

—Krishna

> 
> 
> thanks,
> devkulkarni
> 
> 
> -----Original Message-----
> From: "Roshan Agrawal" <roshan.agrawal at RACKSPACE.COM>
> Sent: Monday, December 9, 2013 10:57am
> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] [Solum] Using Zuul in the Git-pull blueprint
> 
> 
>> -----Original Message-----
>> From: Krishna Raman [mailto:kraman at gmail.com]
>> Sent: Sunday, December 08, 2013 11:24 PM
>> To: OpenStack Development Mailing List (not for usage questions)
>> Subject: [openstack-dev] [Solum] Using Zuul in the Git-pull blueprint
>> 
>> Hi all,
>> 
>> We had a very good meeting last week around the git-pull blueprint. During
>> the discussion, Monty suggested using Zuul to manage the git repository
>> access and workflow.
>> While he is working on sending the group a diagram and description of what
>> he has in mind, I had a couple of other questions which I am hoping the
>> extended group will be able to answer.
>> 
>> 1) Zuul is currently an infrastructure project.
>> 	- Is there anything that prevents us from using it in Solum?
>> 	- Does it need to be moved to a normal OpenStack project?
>> 
>> 2) Zuul provides a sort of workflow engine. This workflow engine could
>> potentially be used to initiate and manage: API Post -> git flow -> lang pack
>> flow.
>> 	- Have there been any discussion after the F2F where we have
>> discussed using some other workflow engine?
> 
> There hasn't been further discussion since F2F.
> Most of the processes in Solum will really be customizable workflows, and use of a generic workflow engine is definitely worth discussing. We may still use to leverage Zuul for the gerrit/git/checkin piece, but Solum will have workflow needs beyond that. 
> 
>> 	- Is Zuul's engine generic enough to be used in Solum? (Hoping
>> Monty can help with this one)
>> 		- Perhaps only use it to manage the API post -> git flow
>> stages?
>> 
>> Thanks
>> -Krishna
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev