Open Stack

On 05/02/2017 08:30 AM, Sean Dague wrote:
> We started running systemd for devstack in the gate yesterday, so far so
> good.
> 
> The following patch (which will hopefully land soon), will convert the
> default local use of devstack to systemd as well -
> https://review.openstack.org/#/c/461716/. It also includes substantially
> updated documentation.
> 
> Once you take this patch, a "./clean.sh" is recommended. Flipping modes
> can cause some cruft to build up, and ./clean.sh should be pretty good
> at eliminating them.
> 
> https://review.openstack.org/#/c/461716/2/doc/source/development.rst is
> probably specifically interesting / useful for people to read, as it
> shows how the standard development workflows will change (for the
> better) with systemd.
> 
> 	-Sean

As a follow up, there are definitely a few edge conditions we've hit
with some jobs, so the following is provided as information in case you
have a job that seems to fail in one of these ways.

Doing process stop / start
==========================

The nova live migration job is special, it was restarting services
manually, however it was doing so with some copy / pasted devstack
code, which means it didn't evolve with the rest of devstack. So the
stop code stopped working (and wasn't robust enough to make it clear
that was the issue).

https://review.openstack.org/#/c/461803/ is the fix (merged)

run_process limitations
=======================

When doing the systemd conversion I looked for a path forward which
was going to make 90% of everything just work. The key trick here was
that services start as the "stack" user, and aren't daemonizing away
from the console. We can take the run_process command and make that
the ExecStart in a unit file.

*Except* that only works if the command is specified by an *absolute
path*.

So things like this in kuryr-libnetwork become an issue
https://github.com/openstack/kuryr-libnetwork/blob/3e2891d6fc5d55b3712258c932a5a8b9b323f6c2/devstack/plugin.sh#L148

There is also a second issue there, which is calling sudo in the
run_process line. If you need to run as a user/group different than
the default, you need to specify that directly.

The run_process command now supports that -
https://github.com/openstack-dev/devstack/blob/803acffcf9254e328426ad67380a99f4f5b164ec/functions-common#L1531-L1535

And lastly, run_process really always did expect that the thing you
started remained attached to the console. These are run as "simple"
services in systemd. If you are running a thing which already
daemonizes systemd is going to assume (correctly in this simple mode)
the fact that the process detatched from it means it died, and kill
and clean it up.

This is the issue the OpenDaylight plugin ran
into. https://review.openstack.org/#/c/461889/ is the proposed fix.



If you run into any other issues please pop into #openstack-qa (or
respond to this email) and we'll try to work through them.

	-Sean

-- 
Sean Dague
http://dague.net