[openstack-dev] [all][kolla] Summary from Mitaka Summit for Kolla
paul.bourke at oracle.com
Thu Nov 5 15:45:50 UTC 2015
I noticed Flavio did one of these for Glance, so I said I'd try
summarise the work sessions that took place for Kolla at the Mitaka summit.
These are by no means exhaustive or a complete record of everything that
was discussed, but are more a high level summary of the etherpads from
each session. Others please feel free to follow up with your own.
Hope it's useful.
* There has been feedback that the current Kolla documentation is too
repetitive and not logically structured. Based on this we brainstormed
on a new structure and set of titles that the docs should be made up of.
This can be found in the etherpad link above.
* Decided we should not require documentation to be included with every
patch - patches should not be rejected due to this. However, should make
liberal use of the DocImpact flag, which will auto generate tickets/bugs
in launchpad for the docs to be filled in.
* We want to figure out how to make use of the smart yum/apt feature
used in existing OpenStack docs that allows the same set of docs to be
generated with commands for various distros.
* Currently we have rsyslog containers on each node collecting logs from
each container/service. However, the logging config in these services is
suboptimal, and we're not getting error logs. Also, non OpenStack
services such as mariadb, rabbitmq, etc are not logging to syslog at all.
* Central logging needs to happen in Mitaka. Need to evaluate
elkstack/efkstack for this. Pad highlights various requirements and
caveats to watch out for when implementing central logging.
* People like the idea of a toolbox container, which has a variety of
useful tools for interacting with a cluster. We may also like to
distribute a wrapper script for calling this in a more user friendly way
(e.g. 'docker-ostk nova list' instead of 'docker run -it <toolbox> nova
Bare Metal Deployment
* Currently we suffer from the problem that in order to deploy Kolla you
first need to manually init the host nodes which can take anywhere from
3+ hours. We discussed some existing solutions for this which mainly
boiled down to either bifrost or instack. Both have their pros and cons
outlined in the etherpad.
* Outcome is to evaluate both of these solutions and hope to document
solutions for both. To support the bare metal config we may also aim to
create a separate small playbook which can do basic config on the nodes
once provisioned (installed docker-py, firewall config, etc).
* In this session we brainstormed on ideas and work that we would like
to see scheduled for the Mitaka cycle. Topics included deploying the
rest of the big tent, a better plugin architecture (horizon, neutron,
etc), and alternative forms of deployment, i.e. Apache Mesos.
* We didn't get as many ops as we'd hoped for this session but still
gathered some very useful feedback. Amongst the top requests were a
mechanism for service plugins (which was already in the mitaka roadmap
session), a way to customise service start scripts, and a way to bind
mount local repos into containers to make kolla a more viable dev
* Pain points included the fact that restart the docker daemon restarts
all containers, and some other open bugs in Docker itself that hits
operators such as sometimes not being able to delete containers.
* Source installs were echoed as been a big driver for Kolla adoption -
people don't like packages and even more so when they have to build them
* Currently we have gates for building + deploying "centos
binary+source", and "ubuntu source". We don't deploy every container in
the gate, just a subset to save time. This excercises the Docker build
and Ansible deploy code. We also want to add gates for the rest of our
included OSes which include oraclelinux and RHEL.
* Our next goal is to exercise the OpenStack services/apis themselves,
in the gate. There is work under way to add tempest runs to the gate,
however inc0 also suggested that we should have some more simple smoke
tests present in order to fail more quickly. Frustration can arise from
waiting for a full deploy only to find out that one of the core
containers such as Keystone went into a failed state right at the
beggining of the deploy. Will investigate how we can use the existing
kolla-ansible container possibly along with shade to implement this.
Third Party Integration
I'm missing notes for this, perhaps someone else or tripleO guys can
pitch in here.
Upgrade - Liberty to Mitaka
* This was quite a long and detailed session and lots got done. We broke
it down by service, starting with infrastructure (mariadb, rabbitmq,
memcached, etc). Each of these have their own quirks mostly related to
clustering and how they can be upgraded in a rolling fashion. Details,
concerns, and investigation required around each are in the etherpad.
* After infra we tackled the OpenStack services themselves. These were
mostly broken down into services that required a database migration, and
those that are stateless.
A problem for us is to automatically tell whether an upgrade of a
particular service will require a migration. As a way to approach this,
we plan to pin to particular tags of services in our build config;
similar to how openstack/requirements operates. When a service tags a
new version, it is up to the community to bump the tag in our build, and
submit this for review. If the bump requires a migration, some Ansible
code and/or a script will be supplied to handle the upgrade. If not, it
is a simple case of stopping the old container and starting the new one,
the same as we can do today.
Finally, we don't want to rely on the container versions in order to
find out what version of Kolla a cluster is meant to be running. To
better solve this case we propose to deploy something like
/etc/kolla/release on the hosts, which will contain the current version.
Again, the upgrade session was fairly detailed so recommend people scan
the etherpad for this one, and definitely chat to us in irc (#kolla)
More information about the OpenStack-dev