Open Stack

I admit I'm not as knowledgable about the Kolla codebase as I'd like to be, so most of what you're saying is going over my head.  I think mainly I don't understand the problem statement.  It looks like you're pulling all the "hard coded" things out of the docker files, and making them user replaceable?  So the dockerfiles just become a list of required steps, and the user can change how each step is implemented?  Would this also unify the dockefiles so there wouldn't be a huge if statements between Centos and Ubuntu?

Thx,
Britt



On 5/27/16, 1:58 AM, "Steven Dake (stdake)" <stdake at cisco.com> wrote:

>
>
>On 5/26/16, 8:45 PM, "Swapnil Kulkarni (coolsvap)" <me at coolsvap.net> wrote:
>
>>On Fri, May 27, 2016 at 8:35 AM, Steven Dake (stdake) <stdake at cisco.com>
>>wrote:
>>> Hey folks,
>>>
>>> While Swapnil has been busy churning the dockerfile.j2 files to all
>>>match
>>> the same style, and we also had summit where we declared we would solve
>>>the
>>> plugin problem, I have decided to begin work on a DSL prototype.
>>>
>>> Here are the problems I want to solve in order of importance by this
>>>work:
>>>
>>> Build CentOS, Ubuntu, Oracle Linux, Debian, Fedora containers
>>> Provide a programmatic way to manage Dockerfile construction rather
>>>then a
>>> manual (with vi or emacs or the like) mechanism
>>> Allow complete overrides of every facet of Dockerfile construction, most
>>> especially repositories per container (rather than in the base
>>>container) to
>>> permit the use case of dependencies from one version with dependencies
>>>in
>>> another version of a different service
>>> Get out of the business of maintaining 100+ dockerfiles but instead
>>>maintain
>>> one master file which defines the data that needs to be used to
>>>construct
>>> Dockerfiles
>>> Permit different types of optimizations or Dockerfile building by
>>>changing
>>> around the parser implementation ­ to allow layering of each operation,
>>>or
>>> alternatively to merge layers as we do today
>>>
>>> I don't believe we can proceed with both binary and source plugins
>>>given our
>>> current implementation of Dockerfiles in any sane way.
>>>
>>> I further don't believe it is possible to customize repositories &
>>>installed
>>> files per container, which I receive increasing requests for offline.
>>>
>>> To that end, I've created a very very rough prototype which builds the
>>>base
>>> container as well as a mariadb container.  The mariadb container builds
>>>and
>>> I suspect would work.
>>>
>>> An example of the DSL usage is here:
>>> https://review.openstack.org/#/c/321468/4/dockerdsl/dsl.yml
>>>
>>> A very poorly written parser is here:
>>> https://review.openstack.org/#/c/321468/4/dockerdsl/load.py
>>>
>>> I played around with INI as a format, to take advantage of oslo.config
>>>and
>>> kolla-build.conf, but that didn't work out.  YML is the way to go.
>>>
>>> I'd appreciate reviews on the YML implementation especially.
>>>
>>> How I see this work progressing is as follows:
>>>
>>> A yml file describing all docker containers for all distros is placed in
>>> kolla/docker
>>> The build tool adds an option ‹use-yml which uses the YML file
>>> A parser (such as load.py above) is integrated into build.py to lay
>>>down he
>>> Dockerfiles
>>> Wait 4-6 weeks for people to find bugs and complain
>>> Make the ‹use-yml the default for 4-6 weeks
>>> Once we feel confident in the yml implementation, remove all
>>>Dockerfile.j2
>>> files
>>> Remove ‹use-yml option
>>> Remove all jinja2-isms from build.py
>>>
>>> This is similar to the work that took place to convert from raw
>>>Dockerfiles
>>> to Dockerfile.j2 files.  We are just reusing that pattern.  Hopefully
>>>this
>>> will be the last major refactor of the dockerfiles unless someone has
>>>some
>>> significant complaints about the approach.
>>>
>>> Regards
>>> -steve
>>>
>>>
>>> 
>>>_________________________________________________________________________
>>>_
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: 
>>>OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>The DSL template to generate the Dockerfile seems way better than the
>>jinja templates in terms of extension which is currently a major
>>bottleneck in the plugin implementation. I am +2+W on this plan of
>>action to test it for next 4-6 weeks and see thereon.
>>
>>Swapnil
>>
>
>Agree.
>
>Customization and plugins are the trigger for the work.  I was thinking of
>the following:
>
>Elemental.yml (ships with Kolla)
>Elemental-merge.yml (operator provides in /etc/kolla, this file is yaml
>merged with elemental.yml)
>Elemental-override.yml (operator provides in /etc/kolla, this file
>overrides any YAML sections defined)
>
>I think merging and overriding the yaml files should be pretty easy,
>compared to jinja2, where I don't even know where to begin in a way that
>the operator doesn't have to have deep knowledge of Kolla's internal
>implementation.
>
>Regards
>-steve
>  
>>__________________________________________________________________________
>>OpenStack Development Mailing List (not for usage questions)
>>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev