<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>Hey folks,</div>
<div><br>
</div>
<div>While Swapnil has been busy churning the dockerfile.j2 files to all match the same style, and we also had summit where we declared we would solve the plugin problem, I have decided to begin work on a DSL prototype.</div>
<div><br>
</div>
<div>Here are the problems I want to solve in order of importance by this work:</div>
<ol>
<li>Build CentOS, Ubuntu, Oracle Linux, Debian, Fedora containers</li><li>Provide a programmatic way to manage Dockerfile construction rather then a manual (with vi or emacs or the like) mechanism</li><li>Allow complete overrides of every facet of Dockerfile construction, most especially repositories per container (rather than in the base container) to permit the use case of dependencies from one version with dependencies in another version of a different
service</li><li>Get out of the business of maintaining 100+ dockerfiles but instead maintain one master file which defines the data that needs to be used to construct Dockerfiles</li><li>Permit different types of optimizations or Dockerfile building by changing around the parser implementation – to allow layering of each operation, or alternatively to merge layers as we do today</li></ol>
<div>I don't believe we can proceed with both binary and source plugins given our current implementation of Dockerfiles in any sane way.</div>
<div><br>
</div>
<div>I further don't believe it is possible to customize repositories & installed files per container, which I receive increasing requests for offline.</div>
<div><br>
</div>
<div>To that end, I've created a very very rough prototype which builds the base container as well as a mariadb container. The mariadb container builds and I suspect would work.</div>
<div><br>
</div>
<div>An example of the DSL usage is here:</div>
<div><a href="https://review.openstack.org/#/c/321468/4/dockerdsl/dsl.yml">https://review.openstack.org/#/c/321468/4/dockerdsl/dsl.yml</a></div>
<div><br>
</div>
<div>A very poorly written parser is here:</div>
<div><a href="https://review.openstack.org/#/c/321468/4/dockerdsl/load.py">https://review.openstack.org/#/c/321468/4/dockerdsl/load.py</a></div>
<div><br>
</div>
<div>I played around with INI as a format, to take advantage of oslo.config and kolla-build.conf, but that didn't work out. YML is the way to go.</div>
<div><br>
</div>
<div>I'd appreciate reviews on the YML implementation especially.</div>
<div><br>
</div>
<div>How I see this work progressing is as follows:</div>
<ol>
<li>A yml file describing all docker containers for all distros is placed in kolla/docker</li><li>The build tool adds an option —use-yml which uses the YML file</li><li>A parser (such as load.py above) is integrated into build.py to lay down he Dockerfiles</li><li>Wait 4-6 weeks for people to find bugs and complain</li><li>Make the —use-yml the default for 4-6 weeks</li><li>Once we feel confident in the yml implementation, remove all Dockerfile.j2 files</li><li>Remove —use-yml option</li><li>Remove all jinja2-isms from build.py</li></ol>
<div>This is similar to the work that took place to convert from raw Dockerfiles to Dockerfile.j2 files. We are just reusing that pattern. Hopefully this will be the last major refactor of the dockerfiles unless someone has some significant complaints about
the approach.</div>
<div><br>
</div>
<div>Regards</div>
<div>-steve</div>
<div><br>
</div>
</body>
</html>