Open Stack

On Fri, May 27, 2016 at 8:35 AM, Steven Dake (stdake) <stdake at cisco.com> wrote:
> Hey folks,
>
> While Swapnil has been busy churning the dockerfile.j2 files to all match
> the same style, and we also had summit where we declared we would solve the
> plugin problem, I have decided to begin work on a DSL prototype.
>
> Here are the problems I want to solve in order of importance by this work:
>
> Build CentOS, Ubuntu, Oracle Linux, Debian, Fedora containers
> Provide a programmatic way to manage Dockerfile construction rather then a
> manual (with vi or emacs or the like) mechanism
> Allow complete overrides of every facet of Dockerfile construction, most
> especially repositories per container (rather than in the base container) to
> permit the use case of dependencies from one version with dependencies in
> another version of a different service
> Get out of the business of maintaining 100+ dockerfiles but instead maintain
> one master file which defines the data that needs to be used to construct
> Dockerfiles
> Permit different types of optimizations or Dockerfile building by changing
> around the parser implementation – to allow layering of each operation, or
> alternatively to merge layers as we do today
>
> I don't believe we can proceed with both binary and source plugins given our
> current implementation of Dockerfiles in any sane way.
>
> I further don't believe it is possible to customize repositories & installed
> files per container, which I receive increasing requests for offline.
>
> To that end, I've created a very very rough prototype which builds the base
> container as well as a mariadb container.  The mariadb container builds and
> I suspect would work.
>
> An example of the DSL usage is here:
> https://review.openstack.org/#/c/321468/4/dockerdsl/dsl.yml
>
> A very poorly written parser is here:
> https://review.openstack.org/#/c/321468/4/dockerdsl/load.py
>
> I played around with INI as a format, to take advantage of oslo.config and
> kolla-build.conf, but that didn't work out.  YML is the way to go.
>
> I'd appreciate reviews on the YML implementation especially.
>
> How I see this work progressing is as follows:
>
> A yml file describing all docker containers for all distros is placed in
> kolla/docker
> The build tool adds an option —use-yml which uses the YML file
> A parser (such as load.py above) is integrated into build.py to lay down he
> Dockerfiles
> Wait 4-6 weeks for people to find bugs and complain
> Make the —use-yml the default for 4-6 weeks
> Once we feel confident in the yml implementation, remove all Dockerfile.j2
> files
> Remove —use-yml option
> Remove all jinja2-isms from build.py
>
> This is similar to the work that took place to convert from raw Dockerfiles
> to Dockerfile.j2 files.  We are just reusing that pattern.  Hopefully this
> will be the last major refactor of the dockerfiles unless someone has some
> significant complaints about the approach.
>
> Regards
> -steve
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

The DSL template to generate the Dockerfile seems way better than the
jinja templates in terms of extension which is currently a major
bottleneck in the plugin implementation. I am +2+W on this plan of
action to test it for next 4-6 weeks and see thereon.

Swapnil