Open Stack

Hi all,

I noticed Flavio did one of these for Glance, so I said I'd try 
summarise the work sessions that took place for Kolla at the Mitaka summit.

These are by no means exhaustive or a complete record of everything that 
was discussed, but are more a high level summary of the etherpads from 
each session. Others please feel free to follow up with your own.

Hope it's useful.

-Paul

Wednesday:
=========
Documentation
-------------
https://etherpad.openstack.org/p/kolla-mitaka-documentation

* There has been feedback that the current Kolla documentation is too 
repetitive and not logically structured. Based on this we brainstormed 
on a new structure and set of titles that the docs should be made up of. 
This can be found in the etherpad link above.

* Decided we should not require documentation to be included with every 
patch - patches should not be rejected due to this. However, should make 
liberal use of the DocImpact flag, which will auto generate tickets/bugs 
in launchpad for the docs to be filled in.

* We want to figure out how to make use of the smart yum/apt feature 
used in existing OpenStack docs that allows the same set of docs to be 
generated with commands for various distros.

Diagnostics
------------
https://etherpad.openstack.org/p/kolla-mitaka-diagnostics

* Currently we have rsyslog containers on each node collecting logs from 
each container/service. However, the logging config in these services is 
suboptimal, and we're not getting error logs. Also, non OpenStack 
services such as mariadb, rabbitmq, etc are not logging to syslog at all.

* Central logging needs to happen in Mitaka. Need to evaluate 
elkstack/efkstack for this. Pad highlights various requirements and 
caveats to watch out for when implementing central logging.

* People like the idea of a toolbox container, which has a variety of 
useful tools for interacting with a cluster. We may also like to 
distribute a wrapper script for calling this in a more user friendly way 
(e.g. 'docker-ostk nova list' instead of 'docker run -it <toolbox> nova 
list'

Bare Metal Deployment
---------------------
https://etherpad.openstack.org/p/kolla-mitaka-bare-metal-deployment

* Currently we suffer from the problem that in order to deploy Kolla you 
first need to manually init the host nodes which can take anywhere from 
3+ hours. We discussed some existing solutions for this which mainly 
boiled down to either bifrost or instack. Both have their pros and cons 
outlined in the etherpad.

* Outcome is to evaluate both of these solutions and hope to document 
solutions for both. To support the bare metal config we may also aim to 
create a separate small playbook which can do basic config on the nodes 
once provisioned (installed docker-py, firewall config, etc).

Mitaka Roadmap
--------------
https://etherpad.openstack.org/p/Kolla-Mitaka-Roadmap

* In this session we brainstormed on ideas and work that we would like 
to see scheduled for the Mitaka cycle. Topics included deploying the 
rest of the big tent, a better plugin architecture (horizon, neutron, 
etc), and alternative forms of deployment, i.e. Apache Mesos.

Operator Requirements
---------------------
https://etherpad.openstack.org/p/kolla-mitaka-operator-requirements-gathering

* We didn't get as many ops as we'd hoped for this session but still 
gathered some very useful feedback. Amongst the top requests were a 
mechanism for service plugins (which was already in the mitaka roadmap 
session), a way to customise service start scripts, and a way to bind 
mount local repos into containers to make kolla a more viable dev 
environment.

* Pain points included the fact that restart the docker daemon restarts 
all containers, and some other open bugs in Docker itself that hits 
operators such as sometimes not being able to delete containers.

* Source installs were echoed as been a big driver for Kolla adoption - 
people don't like packages and even more so when they have to build them 
themselves.

Thursday
========
Gating Commits
---------------
https://etherpad.openstack.org/p/kolla-mitaka-gating

* Currently we have gates for building + deploying "centos 
binary+source", and "ubuntu source". We don't deploy every container in 
the gate, just a subset to save time. This excercises the Docker build 
and Ansible deploy code. We also want to add gates for the rest of our 
included OSes which include oraclelinux and RHEL.

* Our next goal is to exercise the OpenStack services/apis themselves, 
in the gate. There is work under way to add tempest runs to the gate, 
however inc0 also suggested that we should have some more simple smoke 
tests present in order to fail more quickly. Frustration can arise from 
waiting for a full deploy only to find out that one of the core 
containers such as Keystone went into a failed state right at the 
beggining of the deploy. Will investigate how we can use the existing 
kolla-ansible container possibly along with shade to implement this.


Third Party Integration
-----------------------
https://etherpad.openstack.org/p/kolla-mitaka-third-party-projects

I'm missing notes for this, perhaps someone else or tripleO guys can 
pitch in here.

Friday
======
Upgrade - Liberty to Mitaka
---------------------------
https://etherpad.openstack.org/p/kolla-mitaka-upgrade

* This was quite a long and detailed session and lots got done. We broke 
it down by service, starting with infrastructure (mariadb, rabbitmq, 
memcached, etc). Each of these have their own quirks mostly related to 
clustering and how they can be upgraded in a rolling fashion. Details, 
concerns, and investigation required around each are in the etherpad.

* After infra we tackled the OpenStack services themselves. These were 
mostly broken down into services that required a database migration, and 
those that are stateless.

A problem for us is to automatically tell whether an upgrade of a 
particular service will require a migration. As a way to approach this, 
we plan to pin to particular tags of services in our build config; 
similar to how openstack/requirements operates. When a service tags a 
new version, it is up to the community to bump the tag in our build, and 
submit this for review. If the bump requires a migration, some Ansible 
code and/or a script will be supplied to handle the upgrade. If not, it 
is a simple case of stopping the old container and starting the new one, 
the same as we can do today.

Finally, we don't want to rely on the container versions in order to 
find out what version of Kolla a cluster is meant to be running. To 
better solve this case we propose to deploy something like 
/etc/kolla/release on the hosts, which will contain the current version.

Again, the upgrade session was fairly detailed so recommend people scan 
the etherpad for this one, and definitely chat to us in irc (#kolla) 
with questions/concerns.

---