[openstack-dev] [neutron][rootwrap] Performance considerations, sudo?

Joe Gordon joe.gordon0 at gmail.com
Wed Mar 5 17:44:51 UTC 2014

On Wed, Mar 5, 2014 at 8:51 AM, Miguel Angel Ajo Pelayo
<mangelajo at redhat.com> wrote:
> ----- Original Message -----
>> Miguel Angel Ajo wrote:
>> > [...]
>> >    The overhead comes from python startup time + rootwrap loading.
>> >
>> >    I suppose that rootwrap was designed for lower amount of system calls
>> > (nova?).
>> Yes, it was not really designed to escalate rights on hundreds of
>> separate shell commands in a row.
>> >    And, I understand what rootwrap provides, a level of filtering that
>> > sudo cannot offer. But it raises some question:
>> >
>> > 1) It's actually someone using rootwrap in production?
>> >
>> > 2) What alternatives can we think about to improve this situation.
>> >
>> >    0) already being done: coalescing system calls. But I'm unsure that's
>> > enough. (if we coalesce 15 calls to 3 on this system we get:
>> > 192*3*0.3/60 ~=3 minutes overhead on a 10min operation).
>> >
>> >    a) Rewriting rules into sudo (to the extent that it's possible), and
>> > live with that.
>> We used to use sudo and a sudoers file. The rules were poorly written,
>> and there is just so much you can check in a sudoers file. But the main
>> issue was that the sudoers file lived in packaging
>> (distribution-dependent), and was not maintained in sync with the code.
>> Rootwrap let us to maintain the rules (filters) in sync with the code
>> calling them.
> Yes, from security & maintenance, it was an smart decision. I'm thinking
> of automatically converting rootwrap rules to sudoers, but that's very
> limited, specially for the ip netns exec ... case.
>> To work around perf issues, you still have the option of running with a
>> wildcard sudoer file (and root_wrapper = sudo). That's about as safe as
>> running with a badly-written or badly-maintained sudo rules anyway.
> That's what I used for my "benchmark". I just wonder, the how possible
> is to get command injection from neutron, via API or DB.
>> > [...]
>> >    d) Re-writing rootwrap into C (it's 600 python LOCs now).
>> (d2) would be to explore running rootwrap under Pypy. Testing that is on
>> my TODO list, but $OTHERSTUFF got into the way. Feel free to explore
>> that option.
> I tried in my system right now, it takes more time to boot-up. Pypy JIT
> is awesome on runtime, but it seems that boot time is slower.

That is the wrong pypy! there are some pypy core devs lurking on this
ML so they may correct some of these details but:

It turns out python has a really big startup overhead:

jogo at lappy:~$ time echo true

real    0m0.000s
user    0m0.000s
sys     0m0.000s

jogo at lappy:~$ time python -c "print True"

real    0m0.022s
user    0m0.013s
sys     0m0.009s

And I am not surprised pypy isn't much better, pypy works better with
longer running programs.

But pypy isn't just one thing its two parts:

"In common parlance, PyPy has been used to mean two things. The first
is the RPython translation toolchain, which is a framework for
generating dynamic programming language implementations. And the
second is one particular implementation that is so generated - an
implementation of the Pythonprogramming language written in Python
itself. It is designed to be flexible and easy to experiment with."

So the idea is to rewrite rootwrap in in RPython and use the Rpython
translation toolchain to convert rootwrap into C. That way we keep the
source code in a language more friendly to OpenStack devs, and we
hopefully avoid the overhead assocated with starting python up.

> I also played a little with shedskin (py->c++ converter), but it
> doesn't support all the python libraries, dynamic typing, or parameter unpacking.
> That could be another approach, writing a simplified rootwrap in python, and
> have it automatically converted to C++.
> f) haleyb on IRC is pointing me to another approach Carl Baldwin is
> pushing https://review.openstack.org/#/c/67490/ towards command execution
> coalescing.
>> >    e) Doing the command filtering at neutron-side, as a library and live
>> > with sudo with simple filtering. (we kill the python/rootwrap startup
>> > overhead).
>> That's as safe as running with a wildcard sudoers file (neutron user can
>> escalate to root). Which may just be acceptable in /some/ scenarios.
> I think it can be safer, (from the command injection point of view).
>> --
>> Thierry Carrez (ttx)
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

More information about the OpenStack-dev mailing list