Open Stack

On 5/26/16, 8:45 PM, "Swapnil Kulkarni (coolsvap)" <me at coolsvap.net> wrote:

>On Fri, May 27, 2016 at 8:35 AM, Steven Dake (stdake) <stdake at cisco.com>
>wrote:
>> Hey folks,
>>
>> While Swapnil has been busy churning the dockerfile.j2 files to all
>>match
>> the same style, and we also had summit where we declared we would solve
>>the
>> plugin problem, I have decided to begin work on a DSL prototype.
>>
>> Here are the problems I want to solve in order of importance by this
>>work:
>>
>> Build CentOS, Ubuntu, Oracle Linux, Debian, Fedora containers
>> Provide a programmatic way to manage Dockerfile construction rather
>>then a
>> manual (with vi or emacs or the like) mechanism
>> Allow complete overrides of every facet of Dockerfile construction, most
>> especially repositories per container (rather than in the base
>>container) to
>> permit the use case of dependencies from one version with dependencies
>>in
>> another version of a different service
>> Get out of the business of maintaining 100+ dockerfiles but instead
>>maintain
>> one master file which defines the data that needs to be used to
>>construct
>> Dockerfiles
>> Permit different types of optimizations or Dockerfile building by
>>changing
>> around the parser implementation ­ to allow layering of each operation,
>>or
>> alternatively to merge layers as we do today
>>
>> I don't believe we can proceed with both binary and source plugins
>>given our
>> current implementation of Dockerfiles in any sane way.
>>
>> I further don't believe it is possible to customize repositories &
>>installed
>> files per container, which I receive increasing requests for offline.
>>
>> To that end, I've created a very very rough prototype which builds the
>>base
>> container as well as a mariadb container.  The mariadb container builds
>>and
>> I suspect would work.
>>
>> An example of the DSL usage is here:
>> https://review.openstack.org/#/c/321468/4/dockerdsl/dsl.yml
>>
>> A very poorly written parser is here:
>> https://review.openstack.org/#/c/321468/4/dockerdsl/load.py
>>
>> I played around with INI as a format, to take advantage of oslo.config
>>and
>> kolla-build.conf, but that didn't work out.  YML is the way to go.
>>
>> I'd appreciate reviews on the YML implementation especially.
>>
>> How I see this work progressing is as follows:
>>
>> A yml file describing all docker containers for all distros is placed in
>> kolla/docker
>> The build tool adds an option ‹use-yml which uses the YML file
>> A parser (such as load.py above) is integrated into build.py to lay
>>down he
>> Dockerfiles
>> Wait 4-6 weeks for people to find bugs and complain
>> Make the ‹use-yml the default for 4-6 weeks
>> Once we feel confident in the yml implementation, remove all
>>Dockerfile.j2
>> files
>> Remove ‹use-yml option
>> Remove all jinja2-isms from build.py
>>
>> This is similar to the work that took place to convert from raw
>>Dockerfiles
>> to Dockerfile.j2 files.  We are just reusing that pattern.  Hopefully
>>this
>> will be the last major refactor of the dockerfiles unless someone has
>>some
>> significant complaints about the approach.
>>
>> Regards
>> -steve
>>
>>
>> 
>>_________________________________________________________________________
>>_
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 
>>OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>The DSL template to generate the Dockerfile seems way better than the
>jinja templates in terms of extension which is currently a major
>bottleneck in the plugin implementation. I am +2+W on this plan of
>action to test it for next 4-6 weeks and see thereon.
>
>Swapnil
>

Agree.

Customization and plugins are the trigger for the work.  I was thinking of
the following:

Elemental.yml (ships with Kolla)
Elemental-merge.yml (operator provides in /etc/kolla, this file is yaml
merged with elemental.yml)
Elemental-override.yml (operator provides in /etc/kolla, this file
overrides any YAML sections defined)

I think merging and overriding the yaml files should be pretty easy,
compared to jinja2, where I don't even know where to begin in a way that
the operator doesn't have to have deep knowledge of Kolla's internal
implementation.

Regards
-steve
  
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev