Open Stack

So inspired by the "Rootwrap on root-intensive nodes" thread, I went and
wrote a proof-of-concept privsep daemon for neutron:
https://review.openstack.org/#/c/155631
There's nothing neutron-specific in the core mechanism and it could easily
be moved out into a common (oslo) library and reused across other projects.


The basic principles guiding a few design choices were:
- it had to be obvious what code would run with elevated privileges
- the interface between that and the rest of the system had to be easy to
understand and audit
- it had to be about as easy as just implementing a function to add new
functionality

The current code does the predictable things to get a privileged buddy
process: assumes you started as root, creates a socketpair, forks, keeps
limited perms and setuids the privileged process, then reads commands over
the socket.  When the socket closes (for whatever reason), the daemon exits.

Currently it scrobbles around below the neutron.agent.privileged._commands
namespace and allows you to invoke any normal function that doesn't start
start with an underscore.  I think I will change this to use some sort of
more explicit decorator, but you get the idea.

On the client side, it generates client stubs of all the same functions
below neutron.agent.privileged.commands at import-time.  Using the daemon
is then as simple as just calling the client stub:

    from neutron.agent.privileged import daemon as privsep_daemon
    def main():
        privsep_daemon.start()
        ...

    from neutron.agent.privileged.commands import ip_lib as priv_ip
    def foo():
        # Need to create a new veth interface pair - that usually requires
root/NET_ADMIN
        priv_ip.CreateLink('veth', 'veth0', peer='veth1')

Because we now have elevated privileges directly (on the privileged daemon
side) without having to shell out through sudo, we can do all sorts of
nicer things like just using netlink directly to configure networking.
This avoids the overhead of executing subcommands, the ugliness (and
danger) of generating command lines and regex parsing output, and make us
less reliant on specific versions of command line tools (since the kernel
API should be very stable).
I demonstrate some of that in the above change by a set of privileged ipset
functions that still call out to commands but don't use sh -c or sudo
anywhere (so are immune to shell metacharacters in arguments), and ip_lib
functions that use pyroute2 to just call netlink directly for network
interface create/delete/update.

Please discuss.  I could have done this in a spec, but I felt the basic
concept and motivation was obvious and the code specifics were of such
importance that this was better explored in a poc change.  I can post-facto
write a spec if it turns out folks would prefer that.

 - Gus

(if you're curious, it took about a day to write the code, and then about 3
long days of debugging eventlet-related conflicts with the 3rd party
libraries I'd just pulled in.  +1 to removing eventlet, particularly in
low-concurrent-queries agent processes where we can presumably just remove
it and use system threads without any further thought)

On Fri Feb 06 2015 at 5:52:44 PM Steven Dake (stdake) <stdake at cisco.com>
wrote:

>
>
> On 2/4/15, 10:24 AM, "Daniel P. Berrange" <berrange at redhat.com> wrote:
>
> >On Wed, Feb 04, 2015 at 09:10:06AM -0800, James E. Blair wrote:
> >> Thierry Carrez <thierry at openstack.org> writes:
> >>
> >> > You make a good point when you mention "traditional distro" here. I
> >> > would argue that containers are slightly changing the rules of the
> >> > don't-run-as-root game.
> >> >
> >> > Solution (2) aligns pretty well with container-powered OpenStack
> >> > deployments -- running compute nodes as root in a container (and
> >> > embracing abovementioned simplicity/performance gains) sounds like a
> >> > pretty strong combo.
> >>
> >> This sounds at least a little like a suggestion that containers are a
> >> substitute for the security provided by running non-root.  The security
> >> landscape around containers is complex, and while there are a lot of
> >> benefits, I believe the general consensus is that uid 0 processes should
> >> not be seen as fully isolated.
> >>
> >> From https://docs.docker.com/articles/security/ :
> >>
> >>   Docker containers are, by default, quite secure; especially if you
> >>   take care of running your processes inside the containers as
> >>   non-privileged users (i.e., non-root).
> >>
> >> Which is not to say that using containers is not a good idea, but
> >> rather, if one does, one should avoid running as root (perhaps with
> >> capabilities), and use selinux (or similar).
> >
> >Yep, I've seen attempts by some folks to run nova-compute and libvirtd
> >and QEMU inside a docker container. Because of the inherantly privileged
> >nature of what Nova/libvirt/qemu need to do, you end up having to share
> >all the host namespaces with the docker container, except for the
> >filesystem
> >namespace and even that you need to bind mount a bunch of stuff over. As
> >a result the container isn't really offerring any security benefit over
> >running the things outside the container. IOW the use of containers to
> >confine nova is only solving a managability problem rather than a security
> >problem.
> >
> >Regards,
> >Daniel
>
> Daniel,
>
> Agree 100% - compute in containers is all about an atomic image-based
> upgrade and downgrade process, not about solving security problems.
> Ideally the services that are within containers are as secure as running
> natively without containers on bare metal although this might be a bit of
> assumption since docker does run with all Linux capabilities enabled.
>
> Regards
> -steve
>
> >--
> >|: http://berrange.com      -o-
> >http://www.flickr.com/photos/dberrange/ :|
> >|: http://libvirt.org              -o-
> >http://virt-manager.org :|
> >|: http://autobuild.org       -o-
> >http://search.cpan.org/~danberr/ :|
> >|: http://entangle-photo.org       -o-
> >http://live.gnome.org/gtk-vnc :|
> >
> >___________________________________________________________
> _______________
> >OpenStack Development Mailing List (not for usage questions)
> >Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:
> unsubscribe
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150213/de540dcb/attachment.html>