[openstack-dev] [Solum] Using Zuul in the Git-pull blueprint

Georgy Okrokvertskhov gokrokvertskhov at mirantis.com
Fri Dec 13 17:32:00 UTC 2013


Hi,

After reading the etherpad for Solum\Zuul integration I feel that I need
more clarity on this. First of all, what is missed is a positioning of Zuul
in overall Solum architecture. Let me explain a bit why I have this
question about positioning:

1. I don't see how Solum entities (Application, Plan, Components) related
to Zuul workflows.

2. The document describes steps starting from git commit event. It is not
clear how workflow appears in Zuul configuration, what are steps which
should be performed by user? During F2F discussion we agreed that user will
pass some parameters required for build process and deployment process. It
is not clear how these parameters will appear in Zuul workflow.

3. From a security perspective it is not clear how Solum and Zuul will
obtain user authentication information if entry point will be a git commit.
Should user invoke Solum API somehow before git commit? Should Solum be an
entry point? If Zuul will invoke Solum API for actual steps it should pass
user authentication parameters too.

4. If we have multiple users with multiple Application does that mean that
we will have multiple Zuul instances, or we will have multiple workflows
configured in Zuul? If it is a single instance will config change trig Zuul
service restart?

Thanks
Georgy


On Fri, Dec 13, 2013 at 8:56 AM, devdatta kulkarni <
devdatta.kulkarni at rackspace.com> wrote:

> -----Original Message-----
> From: "Krishna Raman" <kraman at gmail.com>
> Sent: Friday, December 13, 2013 9:44am
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] [Solum] Using Zuul in the Git-pull blueprint
>
> On Dec 12, 2013, at 1:39 PM, devdatta kulkarni <
> devdatta.kulkarni at rackspace.com> wrote:
>
> > We followed on the Zuul question in this week's git-integration working
> group meeting.
> >
> > mordred has created an etherpad with a high-level description of Zuul
> and how it might
> > fit with Solum't git integration workflow
> >
> > https://etherpad.openstack.org/p/ZuulSolum
> >
> > The working group seemed to be coming to the consensus that we want to
> use a single workflow
> > engine, as far as possible, for all of Solum's workflow needs.
> > This brought up the question about, what are really Solum's workflow
> requirements.
>
> Hi
>
> I had a long conversation with Monty yesterday and we flushed out a few
> things I would like to run by the group.
> I have also included answers to the questions below.
>
> >
> > At a high-level, I think that Solum has three different kinds of
> workflows.
> >
> > 1) Workflow around getting user code into Solum
> >   - This is the git integration piece being worked out in the
> git-integration
> >     working group.
>
> This is possible using the Zuul workflows. Would potentially require a
> little work in Zuul.
>
> >
> > 2) Workflow around creating language pack(s).
> >   - The main workflow requirement here involves ability to run tests
> before creating a language pack.
> >     There was some discussion in language-pack working group about this
> requirement.
>
> This is also possible using Zuul and in-fact would benefit Solum by
> providing config file based build workflows
> that could be customized by ops personelle. For e.g.. one DU might require
> SVN, another might require git
> and a jenkins CI based unit test before triggering Langpack, other DUs
> might wish to leverage gerrit etc.
> This would be possible through Zuul without having to reinvent it on the
> other workflow engine.
>
> >
> > 3) Workflow around deploying created language pack(s) in order to
> instantiate an assembly.
> >   - The deployment may potentially contain several steps, some of which
> may be long running, such as
> >   populating a database. Further, there may be a need to checkpoint
> intermediate steps
> >   and retry the workflow from the failed point.
>
> This is probably not a very good fit for Zuul. It can handle simple
> workflow but won’t be able to do the
> complex checkpointing, rollback, retry logic etc.
>
> >
> >
> > mordred mentioned that #1 can be achieved by Zuul (both, push-to-solum
> and pull-by-solum)
> > We want to know if #2 and #3 can also be achieved by Zuul.
> > If not, we want to know what are the available options.
> >
> > mordred, thanks for the etherpad; looking forward to the digram :)
>
>
> Zuul is workflow engine capable of running simple workflows. It is
> probably not suitable for all of Solum but would
> manage the source -> DU flow quite nicely. Initially my thoughts were that
> I wanted to avoid having 2 workflow
> engines in Solum but there is another way to look at it…
>
> During out F2F, we had said that we should have a Solum API where we could
> just post DU images. This would
> allow someone to build the DU outside Solum and just provide it. We could
> use this same API as a clean interface to
> separated out the DU build flow from the DU deploy flow. Once this is
> done, the DU build flow (#1, #2 above)
> could be cleanly handled by Zuul and the DU deploy flow by whatever
> complex engine the rest of Solum would
> use.
>
> >> I think this makes sense.
>
> If I were to tie this discussion back to the various working groups and
> blueprints, I think
> the git-integration and language-pack working groups are targeting the "DU
> build flow" (#1 and #2).
> On the other hand, the work being done as part of 'specify-lang-pack'
> blueprint and 'pluggable-template-generation'
> are targeting parts of #3. There would be additional blueprints for other
> aspects of #3.
>
> - Devdatta
>
>
> This approach has a few advantages:
>         * Re-uses what Openstack already uses but its build & CI process
> (and potentially makes it better)
>         * Allows operations who deploy Solum to customize their build
> process without having to change Solum
>         * Allows us to leverage the Zuul/OpenStack-infra team to help us
> solve the DU build flow instead of having
>           to go alone
>
> —Krishna
>
> >
> >
> > thanks,
> > devkulkarni
> >
> >
> > -----Original Message-----
> > From: "Roshan Agrawal" <roshan.agrawal at RACKSPACE.COM>
> > Sent: Monday, December 9, 2013 10:57am
> > To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org>
> > Subject: Re: [openstack-dev] [Solum] Using Zuul in the Git-pull blueprint
> >
> >
> >> -----Original Message-----
> >> From: Krishna Raman [mailto:kraman at gmail.com]
> >> Sent: Sunday, December 08, 2013 11:24 PM
> >> To: OpenStack Development Mailing List (not for usage questions)
> >> Subject: [openstack-dev] [Solum] Using Zuul in the Git-pull blueprint
> >>
> >> Hi all,
> >>
> >> We had a very good meeting last week around the git-pull blueprint.
> During
> >> the discussion, Monty suggested using Zuul to manage the git repository
> >> access and workflow.
> >> While he is working on sending the group a diagram and description of
> what
> >> he has in mind, I had a couple of other questions which I am hoping the
> >> extended group will be able to answer.
> >>
> >> 1) Zuul is currently an infrastructure project.
> >>      - Is there anything that prevents us from using it in Solum?
> >>      - Does it need to be moved to a normal OpenStack project?
> >>
> >> 2) Zuul provides a sort of workflow engine. This workflow engine could
> >> potentially be used to initiate and manage: API Post -> git flow ->
> lang pack
> >> flow.
> >>      - Have there been any discussion after the F2F where we have
> >> discussed using some other workflow engine?
> >
> > There hasn't been further discussion since F2F.
> > Most of the processes in Solum will really be customizable workflows,
> and use of a generic workflow engine is definitely worth discussing. We may
> still use to leverage Zuul for the gerrit/git/checkin piece, but Solum will
> have workflow needs beyond that.
> >
> >>      - Is Zuul's engine generic enough to be used in Solum? (Hoping
> >> Monty can help with this one)
> >>              - Perhaps only use it to manage the API post -> git flow
> >> stages?
> >>
> >> Thanks
> >> -Krishna
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Georgy Okrokvertskhov
Technical Program Manager,
Cloud and Infrastructure Services,
Mirantis
http://www.mirantis.com
Tel. +1 650 963 9828
Mob. +1 650 996 3284
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131213/3fe8b98e/attachment.html>


More information about the OpenStack-dev mailing list