Open Stack

4) Write a small daemon that runs as root, accepting commands over a unix
domain socket or similar. Easier to audit, less code running as root.

On 4 February 2015 at 12:58, Thierry Carrez <thierry at openstack.org> wrote:

> Hi,
>
> This is the follow-up of the discussion we started yesterday on rootwrap
> usage at the cross-project meeting.
>
> A bit of history to stage this story first. OpenStack nodes sometimes
> need to run things with elevated privileges. Nova started out as calling
> sudo shell commands to execute those, which basically allowed it to run
> anything as root. Distributions didn't really like that, so packagers
> started to ship sudoers files that restricted the commands Nova could
> run with sudo. It created a maintenance nightmare, where the code and
> the packaging were routinely out of sync. Rootwrap was designed so that
> we would maintain the allowed commands in the code itself, which
> facilitated maintenance. It would also allow more complex rules than
> sudoers allowed, like the ability to only kill a certain executable
> processes.
>
> Rootwrap was then adopted by Cinder and Neutron, and moved to oslo to
> avoid code duplication. There were still two long-standing problems with
> it, though.
>
> The first one is performance -- each call would spawn a Python
> interpreter which would then call the system command. This was fine when
> there were just a few calls here and there, not so much when it's called
> a hundred times in a row. During the Juno cycle, a daemon mode was added
> to solve this issue. It is significantly faster than running sudo
> directly (the often-suggested alternative). Projects still have to start
> adopting it though. Neutron and Cinder have started work to do that in
> Kilo.
>
> The second problem is the quality of the filter definitions. Rootwrap is
> a framework to enable isolation. It's only as good as the filters each
> project defines. Most of them rely on CommandFilters that do not check
> any argument, instead of using more powerful filters (which are arguably
> more painful to maintain). Developers routinely add filter definitions
> that basically remove any isolation that might have been there, like
> allowing blank dd, tee, chown or chmod.
>
> Basically, some nodes run such a variety of commands as root that
> maintaining proper isolation is a non-trivial amount of work: that's the
> case for Nova's compute nodes and Cinder's volume nodes. I would argue
> that for all other types of nodes, rootwrap is still an appropriate way
> to provide isolation. But for nodes which need to make a very wide
> variety of root calls, we pretend to isolate, but the filter definitions
> are so full of holes that we actually don't.
>
> What solutions do we have ?
>
> (1) we could get our act together and audit and fix those filter
> definitions. Remove superfluous usage of root rights, make use of
> advanced filters for where we actually need them. We have been preaching
> for that at many many design summits. This is a lot of work though...
> There were such efforts in the past, but they were never completed for
> some types of nodes. Worse, the bad filter definitions kept coming back,
> since developers take shortcuts, reviewers may not have sufficient
> security awareness to detect crappy filter definitions, and I don't
> think we can design a gate test that would have such awareness.
>
> (2) bite the bullet and accept that some types of nodes actually need
> root rights for so many different things, they should just run as root
> anyway. I know a few distributions which won't be very pleased by such a
> prospect, but that would be a more honest approach (rather than claiming
> we provide efficient isolation when we really don't). An added benefit
> is that we could replace a number of shell calls by Python code, which
> would simplify the code and increase performance.
>
> (3) intermediary solution where we would run as the nova user but run
> sudo COMMAND directly (instead of sudo nova-rootwrap CONFIG COMMAND).
> That would leave it up to distros to choose between a blanket sudoer or
> maintain their own filtering rules. I think it's a bit hypocritical
> though (pretend the distros could filter if they wanted it, when we
> dropped the towel on doing that ourselves). I'm also not convinced it's
> more secure than solution 2, and it prevents from reducing the number of
> shell-outs, which I think is a worthy idea.
>
> In all cases I would not drop the baby with the bath water, and keep
> rootwrap for all the cases where root rights are needed on a very
> specific set of commands (like neutron, or nova's api-metadata). The
> daemon mode should address the performance issue for the projects making
> a lot of calls.
>
> --
> Thierry Carrez (ttx)
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Duncan Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150204/6b3e3b01/attachment.html>