Open Stack

On Feb 16, 2014, at 2:31 AM, Robert Collins <robertc at robertcollins.net>
 wrote:

> On 16 February 2014 17:57, Jay Pipes <jaypipes at gmail.com> wrote:
> 
>> Personally, I believe having Gerrit and Jenkins as the default will turn
>> more people off Solumn than attract them to it.
>> 
>> Just because we in the OpenStack community love our gating workflow and
>> think it's all groovy does not mean that view is common, wanted, or
>> understood by the vast majority of users of Heroku-like solutions.
> 
> There are quite a number of such solutions; do we want to write yet
> another? What makes solum special ? Is it supporting OpenStack? [If so
> - I'd argue that our success elsewhere will guarantee other such
> platforms support OpenStack - and e.g. OpenShift already do AFAIK].
> Why are we writing Solum?

Excellent focusing question. We are focused on the Application Developer persona as a class. We want clouds to be easier to use, and more accessible for them. We'd like to offer streamlined experiences that help them focus on application development, and worry less about how to get their applications to run on the cloud. We expect to be able to achieve this goal by letting the developers connect their code repositories with Solum enabled clouds, and use development best practices to automate the building, testing, and deploying of code.

Why? Because Application Developers have enough to worry about, without contemplating all of the above in a DIY approach. We'd like to address this focus area in a way that offers the developers a sense of "insurance" that when their application becomes successful one day (and they want to begin focusing on the details and potentially customizing the various aspects of how code is handled and deployed, and what environments it it run in) that they will not need to forklift their code from one cloud to another, but can begin to use a wider variety of control system API's to tap into what's already there.

Cloud operators (public and private alike) want to offer these streamlined user experiences to the Application Developers that use their clouds. They want to do so without bringing in the operational complexity and overhead of running a PaaS system that largely overlaps with numerous aspects of what's already offered in OpenStack. By selecting Solum, they leverage what's here already, and add in additional ease of use, and the ability to abstract and simplify a variety of complexity relating to the code->app lifecycle, and gradually adding in new capabilities for managing apps once they are on the cloud.

>> Who is the audience here? It is not experienced developers who already
>> understand things like Gerrit and Jenkins. It's developers who just want
>> to simplify the process of pushing code up to some system other than
>> Github or their workstation. Adding the awkwardness of Gerrit's code
> 
> If thats all, then I don't think we should be writing Solum. We should
> just pick some of the existing tooling for that and say 'use that'.

If we do a good job of offering a sensible set of defaults, balanced with a compelling set of features, then Solum will be very useful. We're not aiming to replace or re-invent every CI tool that was ever invented. To the contrary, we want to allow sophisticated dev shops who have investments in CI  infrastructure already an opportunity to use Solum as a CD tool (submit a plan file and a container image to the Solum API by using a simple Git post-commit hook trigger).

However, we also want to provide developers who possibly have never seen a comprehensive CI/CD setup an opportunity to access one that works for general use, and a place where the existing tool makers can integrate and offer their value-added software and solutions to make those experiences even better. This is where we are right now, deicing how to take an iterative approach to making this default CI/CD setup useful and interesting.

>> review system -- and the associated pain of trying to understand how to
>> define Jenkins jobs -- is something that I don't think the *default*
>> Solum experience should invite.
>> 
>> The default experience should be a simple push code, run merge tests,
> 
> What are merge tests? How are they defined? What about that definition
> makes it impossible for us to run them in Jenkins or similar? What
> makes it impossible to run them before the merge?

Excellent questions again. There's no reason you could not use any variety of existing applications to do this. There is value in allowing developers to start simple with reduced effort, and start to use more advanced CI features when they are ready to opt into them. Focusing on ease of use is key.

>> and deploy into the deployment unit (whatever that is called in Solum
>> nowadays). There should be well-documented ways to add commit hooks into
>> this workflow, but having a complex Gerrit and Jenkins gated workflow is
>> just overkill for the default experience.

We can all agree that we are not aiming to offer something complicated by default. It's more about having the capability to do some advanced features easier than if you try to set this up all by yourself. The question is how much should we optimize for the simple use case versus the comprehensive one. I'm suggesting that we make the default capabilities seem as simple as "drop code and see it run", but having some knobs for switching on other features to start automating more and more aspects of the CI pipeline.

> Even in small teams - 4-5 people - gated workflow and CI/CD is a game
> changer for velocity. If your point is 'doing the sophisticated thing
> OpenStack does is hard and we shouldn't try to sell people on working
> hard' - I agree.

+1

> But your point currently sounds like 'there's no way
> we can make this stuff easy, discoverable, and straight forward, so we
> should focus on something much less capable'. Perhaps thats not what
> you mean?

Precisely. Make it easy, but do it in a way that does *not* limit us to only handling toy applications.

> I think there is a spectrum here. We need to aim high, or we achieve
> low. We also need to deliver incrementally, and we need to provide a
> -really- gentle onramp for users. Make the first basic steps easy and
> simple: but thats not in any way incompatible with gating by default.

+1

> It *might* be incompatible with Jenkins and zuul: but thats a slightly
> different discussion- my +1, which you replied to, was about /gating/
> by default, not about the implementation thereof.

Yes. Evidenced by numerous replies to this thread, gating by default is probably overkill for our target persona. But the *capability* to turn on gating, and make that a "light lift" is very smart. Starting with merge tests as a noop, and being able to change that later is striking a balance between the ease-of-use goal, and having more power within grasp.

Let's revisit the original question of using Zuul, or something simpler to start with. I think this question really boils down to our desire to approach our design and development iteratively. We want to avoid over-engineering something initially. We are wise to raise this question. I am convinced at this point that the choice to use Zuul as a first step is worth the additional work in streamlining our "basic" flow. This would mean putting noop gate jobs in by default, with an easy way to swap those for ones that do something later.

To be clear, I'm not advocating for including every popular open source CI tool as part of the default setup. I'm advocating for using (a slightly enhanced) Zuul as a way to give us more hooks for those sweet features later, as we iterate toward putting more "power" within reach. This advances us toward the short term goal of having a Git trigger to run build and deploy within Solum for our first use case, and does it in a way that doe not limit us in the intermediate term.

Adrian