Open Stack

On Dec 13, 2013, at 8:56 AM, devdatta kulkarni <devdatta.kulkarni at rackspace.com> wrote:

> -----Original Message-----
> From: "Krishna Raman" <kraman at gmail.com>
> Sent: Friday, December 13, 2013 9:44am
> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] [Solum] Using Zuul in the Git-pull blueprint
> 
> On Dec 12, 2013, at 1:39 PM, devdatta kulkarni <devdatta.kulkarni at rackspace.com> wrote:
> 
>> We followed on the Zuul question in this week's git-integration working group meeting.
>> 
>> mordred has created an etherpad with a high-level description of Zuul and how it might
>> fit with Solum't git integration workflow
>> 
>> https://etherpad.openstack.org/p/ZuulSolum
>> 
>> The working group seemed to be coming to the consensus that we want to use a single workflow
>> engine, as far as possible, for all of Solum's workflow needs.
>> This brought up the question about, what are really Solum's workflow requirements. 
> 
> Hi
> 
> I had a long conversation with Monty yesterday and we flushed out a few things I would like to run by the group.
> I have also included answers to the questions below.
> 
>> 
>> At a high-level, I think that Solum has three different kinds of workflows.
>> 
>> 1) Workflow around getting user code into Solum
>>  - This is the git integration piece being worked out in the git-integration
>>    working group.
> 
> This is possible using the Zuul workflows. Would potentially require a little work in Zuul.
> 
>> 
>> 2) Workflow around creating language pack(s).
>>  - The main workflow requirement here involves ability to run tests before creating a language pack.
>>    There was some discussion in language-pack working group about this requirement.
> 
> This is also possible using Zuul and in-fact would benefit Solum by providing config file based build workflows
> that could be customized by ops personelle. For e.g.. one DU might require SVN, another might require git 
> and a jenkins CI based unit test before triggering Langpack, other DUs might wish to leverage gerrit etc.
> This would be possible through Zuul without having to reinvent it on the other workflow engine.
> 
>> 
>> 3) Workflow around deploying created language pack(s) in order to instantiate an assembly.
>>  - The deployment may potentially contain several steps, some of which may be long running, such as
>>  populating a database. Further, there may be a need to checkpoint intermediate steps
>>  and retry the workflow from the failed point.
> 
> This is probably not a very good fit for Zuul. It can handle simple workflow but won’t be able to do the
> complex checkpointing, rollback, retry logic etc.
> 
>> 
>> 
>> mordred mentioned that #1 can be achieved by Zuul (both, push-to-solum and pull-by-solum)
>> We want to know if #2 and #3 can also be achieved by Zuul.
>> If not, we want to know what are the available options.
>> 
>> mordred, thanks for the etherpad; looking forward to the digram :)
> 
> 
> Zuul is workflow engine capable of running simple workflows. It is probably not suitable for all of Solum but would
> manage the source -> DU flow quite nicely. Initially my thoughts were that I wanted to avoid having 2 workflow
> engines in Solum but there is another way to look at it…
> 
> During out F2F, we had said that we should have a Solum API where we could just post DU images. This would
> allow someone to build the DU outside Solum and just provide it. We could use this same API as a clean interface to
> separated out the DU build flow from the DU deploy flow. Once this is done, the DU build flow (#1, #2 above)
> could be cleanly handled by Zuul and the DU deploy flow by whatever complex engine the rest of Solum would
> use.
> 
>>> I think this makes sense.
> 
> If I were to tie this discussion back to the various working groups and blueprints, I think
> the git-integration and language-pack working groups are targeting the "DU build flow" (#1 and #2).
> On the other hand, the work being done as part of 'specify-lang-pack' blueprint and 'pluggable-template-generation'
> are targeting parts of #3. There would be additional blueprints for other aspects of #3.

+1

> 
> - Devdatta
> 
> 
> This approach has a few advantages:
> 	* Re-uses what Openstack already uses but its build & CI process (and potentially makes it better)
> 	* Allows operations who deploy Solum to customize their build process without having to change Solum
> 	* Allows us to leverage the Zuul/OpenStack-infra team to help us solve the DU build flow instead of having 
> 	  to go alone
> 
> —Krishna
> 
>> 
>> 
>> thanks,
>> devkulkarni
>> 
>> 
>> -----Original Message-----
>> From: "Roshan Agrawal" <roshan.agrawal at RACKSPACE.COM>
>> Sent: Monday, December 9, 2013 10:57am
>> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
>> Subject: Re: [openstack-dev] [Solum] Using Zuul in the Git-pull blueprint
>> 
>> 
>>> -----Original Message-----
>>> From: Krishna Raman [mailto:kraman at gmail.com]
>>> Sent: Sunday, December 08, 2013 11:24 PM
>>> To: OpenStack Development Mailing List (not for usage questions)
>>> Subject: [openstack-dev] [Solum] Using Zuul in the Git-pull blueprint
>>> 
>>> Hi all,
>>> 
>>> We had a very good meeting last week around the git-pull blueprint. During
>>> the discussion, Monty suggested using Zuul to manage the git repository
>>> access and workflow.
>>> While he is working on sending the group a diagram and description of what
>>> he has in mind, I had a couple of other questions which I am hoping the
>>> extended group will be able to answer.
>>> 
>>> 1) Zuul is currently an infrastructure project.
>>> 	- Is there anything that prevents us from using it in Solum?
>>> 	- Does it need to be moved to a normal OpenStack project?
>>> 
>>> 2) Zuul provides a sort of workflow engine. This workflow engine could
>>> potentially be used to initiate and manage: API Post -> git flow -> lang pack
>>> flow.
>>> 	- Have there been any discussion after the F2F where we have
>>> discussed using some other workflow engine?
>> 
>> There hasn't been further discussion since F2F.
>> Most of the processes in Solum will really be customizable workflows, and use of a generic workflow engine is definitely worth discussing. We may still use to leverage Zuul for the gerrit/git/checkin piece, but Solum will have workflow needs beyond that. 
>> 
>>> 	- Is Zuul's engine generic enough to be used in Solum? (Hoping
>>> Monty can help with this one)
>>> 		- Perhaps only use it to manage the API post -> git flow
>>> stages?
>>> 
>>> Thanks
>>> -Krishna
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131213/0f6b11d7/attachment.html>