Open Stack

----- Original Message -----
> 
> 
> On 11/22/2013 09:51 PM, Adrian Otto wrote:
> > Monty,
> > 
> > On Nov 22, 2013, at 6:24 PM, Monty Taylor <mordred at inaugust.com>
> > wrote:
> > 
> >> On 11/22/2013 05:37 PM, Krishna Raman wrote:
> >>> Hello all,
> >>> 
> >>> I would like to kickoff the Git integration discussion. Goal of
> >>> this subgroup is to go through the git-integration blueprint [1]
> >>> and break it up into smaller blueprints that we can execute on.
> >>> 
> >>> We have to consider 2 workflows: 1) For Milestone 1, pull based
> >>> git workflow where user uses a public git repository (possibly on
> >>> github) to trigger the build 2) For later milestones, a push
> >>> based workflow where the git repository is maintained by Solum
> >> 
> >> Hi!
> > 
> > Hi, thanks for chiming in here.
> > 
> >> I'm a little disappointed that we've decided to base the initial
> >> workflow on something that is not related to the world-class
> >> git-based developer tooling that the OpenStack project has already
> >> produced. We have a GIANT amount of tooling in this space, and it's
> >> all quite scalable. There is also the intent by 3 or 4 different
> >> groups to make it more re-usable/re-consumable, including thoughts
> >> in making sure that we can drive it from and have it consume heat.
> > 
> > The initial work will be something pretty trivial. It's just a web
> > hook on a git push. The workflow in this case is not customizable,
> > and has basically no features. The intent is to iterate on this to
> > make it much more compelling over time, soon after the minimum
> > integration, we will put a real workflow system in place. We did
> > discuss Zuul and Nodepool, and nobody had any objection to learning
> > more about those. This might be a bit early in our roadmap to be
> > pulling them in, but if there is an easy way to use them early in our
> > development, we'd like to explore that.
> 
> A web hook on a git push seems trivial, but is not. Something has to run
> the git push - and as you said, you're planning that thing to be github.
> That means that, out of the gate, solum will not work without deferring
> to a non-free service. Alternately, you could install one of the other
> git servers, such as gitlab - but now you're engineering that.

I think we should have been clearer here - we didn't plan to make GitHub required or specific in any fashion.  In fact, during the discussions we simply said "URL that could be curl'd during Git post-receive hook" - working for GitHub/gitorious/arbitrary would be something much later.  Solum's goal was to be source control agnostic except for two points:

1) an abstraction that lets Solum take the contents of a single source repository and/or binary artifacts and pass that to a mechanism that converts them into a deployable image
2) an endpoint (the webhook) that lets an external caller notify Solum of the need to do #1 for a known repository / artifact

The #1 is actually not intended to imply Solum is in control of the source repository - instead, Solum operates only through "pull" mechanics, and operators/users still get to choose the workflow around each repository.  An example might be an OpenStack gate job that runs a deployment of the HEAT services through Solum - Zuul does a build, tells Solum to go deploy the results of that particular commit or those build artifacts as two isolated simple services, then invokes test cases against that deployed result.  Solum pulls and deploys based on the Zuul notification and build output, but is not in charge of the repository in any way.  The key is that a developer might want to deploy their own version of HEAT the same way that Zuul does - they invoke Solum and tell it to deploy based on commit X of their own particular repository, and Solum pulls and deploys in a very similar fashion.

I think that the ultimate goal of Solum is to abstract how a source/binary input is turned into an image that can be deployed and then pass that off to HEAT.  There's a lot that can happen under that abstraction - the idea of taking a base image and transforming it into an image that can be run as a component of a larger service plays to both HEAT and Nova's strengths, without unnecessarily constraining existing workflows from being used.  

The workflow at the front end (verifying and testing source) can be arbitrarily complex, or organizations can adopt Zuul/Gerrit (which we should definitely enable), or simply be a straight pass through (for developers working in test environments).  Likewise the workflow inside making a deployable image can be arbitrarily complex - simple pip install of a Python source repo, straight up to full fledged base images built using diskimagebuilder for OpenStack, but all Solum has to care about is setting up the environment and triggering the transition, then snapshotting afterwards.

I definitely worry we lack a lot of concrete examples in the wiki that walk through how these flows might exist in different organizations / applications / environments, and the value that the Solum abstraction (changing source/binary/topology into deployed resources) can therefore bring.  I'll take a todo to flush that out so we have concrete flows to talk about both in the Git integration and language pack discussions.