Open Stack

On 06/27/2014 09:04 AM, Jeremy Stanley wrote:
> On 2014-06-27 14:40:44 +0200 (+0200), Antoine Musso wrote:
> [...]
>> I was wondering whether yet another DSL could be build on top of JJB / Zuul.
>>
>> JJB would provide the templates, macro etc and Zuul the pipelines and
>> templates.  The new DSL would let you, out of a single definition,
>> realise the appropriate JJB and Zuul project sections.
>>
>> From there one could build a Django app on top of the new DSL and let
>> folks easily click the jobs they want on their project.  And you end up
>> with a CI self service.
>>
>> Sorry probably does not make any sense.
> 
> It actually makes a lot of sense. One of the options we discussed
> was to backronym it to be Jim's Job Builder and teach JJB to be able
> to spit out shell suitable for TH or XML for Jenkins (assuming the
> "JJB as a library so TH can read the configs directly" idea proves
> less viable). I could see extending that to also teach it to
> interact with Zuul's layout configuration, potentially.

Right - but actually what I'm suggesting is the opposite. I'm suggesting
not that we write a thing that lets jenkins know about ansible, or a new
meta dsl ...

What I'm suggesting is that we use the ansible dsl. It's 95% of what we
need - and it might actually be 100% of what we need. Just like jjb it
has pluggable modules that know how to do stuff same as we.

BUT - what we would do is, for the case where you wanted to use jenkins
as an engine, jjb would read the ansible yaml and would output jenkins
XML same as it does not. In the case where you want to use ansible to
execute the yaml, you'd just run ansible on it. In the case where
turbo-hipster wants to do things, it could either read and parse and
execute the yaml itself, or it could use ansible as a library and tell
it to run the actions on localhost.

The main difference we have to solve are two things. First:

- sections

In JJB, we have sections that tasks go in. For instance, we have a
-builders section and a -publishers section in the yaml. This is how
jenkins organizes things, but actually isn't necessary to express in the
jobs. If the ssh publisher knows that it's a publisher, and the shell
builder knows that is'a  builder, then instead of:

- job:
    name: storyboard-docs
    node: 'bare-precise || bare-trusty'

    builders:
      - revoke-sudo
      - gerrit-git-prep
      - docs:
          github-org: openstack-infra
          project: storyboard

    publishers:
      - scp:
          site: '173.203.107.207'
          files:
            - target: 'ci/storyboard'
              source: 'doc/build/html/**/*'
              keep-hierarchy: true
      - console-log

We could just have:

- job:
    name: storyboard-docs
    node: 'bare-precise || bare-trusty'

    tasks:
      - revoke-sudo
      - gerrit-git-prep
      - docs:
          github-org: openstack-infra
          project: storyboard
      - scp:
          site: '173.203.107.207'
          files:
            - target: 'ci/storyboard'
              source: 'doc/build/html/**/*'
              keep-hierarchy: true
      - console-log

Or, more succinctly:


- name: storyboard-docs
  hosts: 'bare-precise || bare-trusty'
  tasks:
    - revoke-sudo
    - gerrit-git-prep
    - docs:
        github-org: openstack-infra
        project: storyboard
    - scp:
        site: '173.203.107.207'
        files:
          - target: 'ci/storyboard'
            source: 'doc/build/html/**/*'
            keep-hierarchy: true
    - console-log

Which, btw - is a valid ansible playbook.

Second:

- templating

This is the biggest problem and I don't have an answer for it. But I
figure if we can get the first thing, as a first step, then perhaps in
the "execute with ansible" scenario we can just have jjb be a
pre-compiler which expands our templates and then feeds to ansible.