[openstack-dev] [neutron][rootwrap] Performance considerations, sudo?
Miguel Angel Ajo
majopela at redhat.com
Thu Mar 20 09:31:18 UTC 2014
On 03/19/2014 10:54 PM, Joe Gordon wrote:
>
>
>
> On Wed, Mar 19, 2014 at 9:25 AM, Miguel Angel Ajo <majopela at redhat.com
> <mailto:majopela at redhat.com>> wrote:
>
>
>
> An advance on the changes that it's requiring to have a
> py->c++ compiled rootwrap as a mitigation POC for havana/icehouse.
>
> https://github.com/mangelajo/__shedskin.rootwrap/commit/__e4167a6491dfbc71e2d0f6e28ba93b__c8a1dd66c0
> <https://github.com/mangelajo/shedskin.rootwrap/commit/e4167a6491dfbc71e2d0f6e28ba93bc8a1dd66c0>
>
> The current translation output is included.
>
> It looks like doable (almost killed 80% of the translation problems),
> but there are two big stones:
>
> 1) As Joe said, no support for Subprocess (we're interested in popen),
> I'm using a dummy os.system() for the test.
>
> 2) No logging support.
>
> I'm not sure on how complicated could be getting those modules
> implemented for shedkin.
>
>
> Before sorting out of we can get those working under shedskin, any
> preliminary performance numbers from neutron when using this?
>
Sure, totally agree.
I'm trying to put up a conversion without 1 & 2, to run a benchmark on
it, and then I'll post the results.
I suppose, we couldn't use it in neutron itself without Popen support
(not sure) but at least I could get an estimate out of the previous
numbers and the new ones.
Best,
Miguel Ángel.
>
>
> On 03/18/2014 09:14 AM, Miguel Angel Ajo wrote:
>
> Hi Joe, thank you very much for the positive feedback,
>
> I plan to spend a day during this week on the
> shedskin-compatibility
> for rootwrap (I'll branch it, and tune/cut down as necessary) to
> make
> it compile under shedskin [1] : nothing done yet.
>
> It's a short-term alternative until we can have a rootwrap
> agent,
> together with it's integration in neutron (for Juno). As, for the
> compiled rootwrap, if it works, and if it does look good
> (security wise)
> then we'd have a solution for Icehouse/Havana.
>
> help in [1] is really welcome ;-) I'm available in
> #openstack-neutron
> as 'ajo'.
>
> Best regards,
> Miguel Ángel.
>
> [1] https://github.com/mangelajo/__shedskin.rootwrap
> <https://github.com/mangelajo/shedskin.rootwrap>
>
> On 03/18/2014 12:48 AM, Joe Gordon wrote:
>
>
>
>
> On Tue, Mar 11, 2014 at 1:46 AM, Miguel Angel Ajo Pelayo
> <mangelajo at redhat.com <mailto:mangelajo at redhat.com>
> <mailto:mangelajo at redhat.com <mailto:mangelajo at redhat.com>>>
> wrote:
>
>
> I have included on the etherpad, the option to write a sudo
> plugin (or several), specific for openstack.
>
>
> And this is a test with shedskin, I suppose that in
> more complicated
> dependecy scenarios it should perform better.
>
> [majopela at redcylon tmp]$ cat <<EOF >test.py
> > import sys
> > print "hello world"
> > sys.exit(0)
> > EOF
>
> [majopela at redcylon tmp]$ time python test.py
> hello world
>
> real 0m0.016s
> user 0m0.015s
> sys 0m0.001s
>
>
>
> This looks very promising!
>
> A few gotchas:
>
> * Very limited library support
> https://code.google.com/p/__shedskin/wiki/docs#Library___Limitations
> <https://code.google.com/p/shedskin/wiki/docs#Library_Limitations>
> * no logging
> * no six
> * no subprocess
>
> * no *args support
> *
> https://code.google.com/p/__shedskin/wiki/docs#Python___Subset_Restrictions
> <https://code.google.com/p/shedskin/wiki/docs#Python_Subset_Restrictions>
>
> that being said I did a quick comparison with great results:
>
> $ cat tmp.sh
> #!/usr/bin/env bash
> echo $0 $@
> ip a
>
> $ time ./tmp.sh foo bar> /dev/null
>
> real 0m0.009s
> user 0m0.003s
> sys 0m0.006s
>
>
>
> $ cat tmp.py
> #!/usr/bin/env python
> import os
> import sys
>
> print sys.argv
> print os.system("ip a")
>
> $ time ./tmp.py foo bar > /dev/null
>
> min:
> real 0m0.016s
> user 0m0.004s
> sys 0m0.012s
>
> max:
> real 0m0.038s
> user 0m0.016s
> sys 0m0.020s
>
>
>
> shedskin tmp.py && make
>
>
> $ time ./tmp foo bar > /dev/null
>
> real 0m0.010s
> user 0m0.007s
> sys 0m0.002s
>
>
>
> Based in these results I think a deeper dive into making
> rootwrap
> supportshedskin is worthwhile.
>
>
>
>
>
> [majopela at redcylon tmp]$ shedskin test.py
> *** SHED SKIN Python-to-C++ Compiler 0.9.4 ***
> Copyright 2005-2011 Mark Dufour; License GNU GPL
> version 3 (See
> LICENSE)
>
> [analyzing types..]
> ******************************__**100%
> [generating c++ code..]
> [elapsed time: 1.59 seconds]
> [majopela at redcylon tmp]$ make
> g++ -O2 -march=native -Wno-deprecated -I.
> -I/usr/lib/python2.7/site-__packages/shedskin/lib
> /tmp/test.cpp
> /usr/lib/python2.7/site-__packages/shedskin/lib/sys.cpp
> /usr/lib/python2.7/site-__packages/shedskin/lib/re.cpp
>
> /usr/lib/python2.7/site-__packages/shedskin/lib/builtin.__cpp -lgc
> -lpcre -o test
> [majopela at redcylon tmp]$ time ./test
> hello world
>
> real 0m0.003s
> user 0m0.000s
> sys 0m0.002s
>
>
> ----- Original Message -----
> > We had this same issue with the dhcp-agent. Code was
> added that
> paralleled
> > the initial sync here:
> https://review.openstack.org/#__/c/28914/
> <https://review.openstack.org/#/c/28914/>
> that made
> > things a good bit faster if I remember correctly.
> Might be worth
> doing
> > something similar for the l3-agent.
> >
> > Best,
> >
> > Aaron
> >
> >
> > On Mon, Mar 10, 2014 at 5:07 PM, Joe Gordon <
> joe.gordon0 at gmail.com <mailto:joe.gordon0 at gmail.com>
> <mailto:joe.gordon0 at gmail.com
> <mailto:joe.gordon0 at gmail.com>> > wrote:
> >
> >
> >
> >
> >
> >
> > On Mon, Mar 10, 2014 at 3:57 PM, Joe Gordon <
> joe.gordon0 at gmail.com <mailto:joe.gordon0 at gmail.com>
> <mailto:joe.gordon0 at gmail.com
> <mailto:joe.gordon0 at gmail.com>> > wrote:
> >
> >
> >
> > I looked into the python to C options and haven't
> found anything
> promising
> > yet.
> >
> >
> > I tried Cython, and RPython, on a trivial hello
> world app, but
> git similar
> > startup times to standard python.
> >
> > The one thing that did work was adding a '-S' when
> starting
> python.
> >
> > -S Disable the import of the module site and the
> site-dependent
> manipulations
> > of sys.path that it entails.
> >
> > Using 'python -S' didn't appear to help in devstack
> >
> > #!/usr/bin/python -S
> > # PBR Generated from u'console_scripts'
> >
> > import sys
> > import site
> >
> site.addsitedir('/mnt/stack/__oslo.rootwrap/oslo/rootwrap')
> >
> >
> >
> >
> >
> >
> > I am not sure if we can do that for rootwrap.
> >
> >
> > jogo at dev:~/tmp/pypy-2.2.1-src$ time ./tmp-c
> > hello world
> >
> > real 0m0.021s
> > user 0m0.000s
> > sys 0m0.020s
> > jogo at dev:~/tmp/pypy-2.2.1-src$ time ./tmp-c
> > hello world
> >
> > real 0m0.021s
> > user 0m0.000s
> > sys 0m0.020s
> > jogo at dev:~/tmp/pypy-2.2.1-src$ time python -S ./tmp.py
> > hello world
> >
> > real 0m0.010s
> > user 0m0.000s
> > sys 0m0.008s
> >
> > jogo at dev:~/tmp/pypy-2.2.1-src$ time python -S ./tmp.py
> > hello world
> >
> > real 0m0.010s
> > user 0m0.000s
> > sys 0m0.008s
> >
> >
> >
> > On Mon, Mar 10, 2014 at 3:26 PM, Miguel Angel Ajo
> Pelayo <
> > mangelajo at redhat.com <mailto:mangelajo at redhat.com>
> <mailto:mangelajo at redhat.com <mailto:mangelajo at redhat.com>>
> > wrote:
> >
> >
> > Hi Carl, thank you, good idea.
> >
> > I started reviewing it, but I will do it more
> carefully tomorrow
> morning.
> >
> >
> >
> > ----- Original Message -----
> > > All,
> > >
> > > I was writing down a summary of all of this and
> decided to just
> do it
> > > on an etherpad. Will you help me capture the big
> picture
> there? I'd
> > > like to come up with some actions this week to try
> to address
> at least
> > > part of the problem before Icehouse releases.
> > >
> > >
> https://etherpad.openstack.__org/p/neutron-agent-exec-__performance
> <https://etherpad.openstack.org/p/neutron-agent-exec-performance>
> > >
> > > Carl
> > >
> > > On Mon, Mar 10, 2014 at 5:26 AM, Miguel Angel Ajo <
> majopela at redhat.com <mailto:majopela at redhat.com>
> <mailto:majopela at redhat.com <mailto:majopela at redhat.com>> >
> > > wrote:
> > > > Hi Yuri & Stephen, thanks a lot for the
> clarification.
> > > >
> > > > I'm not familiar with unix domain sockets at low
> level, but ,
> I wonder
> > > > if authentication could be achieved just with
> permissions
> (only users in
> > > > group "neutron" or group "rootwrap" accessing
> this service.
> > > >
> > > > I find it an interesting alternative, to the
> other proposed
> solutions,
> > > > but
> > > > there are some challenges associated with this
> solution,
> which could make
> > > > it
> > > > more complicated:
> > > >
> > > > 1) Access control, file system permission based
> or token
> based,
> > > >
> > > > 2) stdout/stderr/return encapsulation/forwarding
> to the
> caller,
> > > > if we have a simple/fast RPC mechanism we can
> use, it's a
> matter
> > > > of serializing a dictionary.
> > > >
> > > > 3) client side implementation for 1 + 2.
> > > >
> > > > 4) It would need to accept new domain socket
> connections in
> green threads
> > > > to
> > > > avoid spawning a new process to handle a new
> connection.
> > > >
> > > > The advantages:
> > > > * we wouldn't need to break the only-python-rule.
> > > > * we don't need to rewrite/translate rootwrap.
> > > >
> > > > The disadvantages:
> > > > * it needs changes on the client side (neutron +
> other
> projects),
> > > >
> > > >
> > > > Cheers,
> > > > Miguel Ángel.
> > > >
> > > >
> > > >
> > > > On 03/08/2014 07:09 AM, Yuriy Taraday wrote:
> > > >>
> > > >> On Fri, Mar 7, 2014 at 5:41 PM, Stephen Gran
> > > >> < stephen.gran at theguardian.com
> <mailto:stephen.gran at theguardian.com>
> <mailto:stephen.gran at __theguardian.com
> <mailto:stephen.gran at theguardian.com>> <mailto:
> stephen.gran at theguardian.com
> <mailto:stephen.gran at theguardian.com>
> <mailto:stephen.gran at __theguardian.com
> <mailto:stephen.gran at theguardian.com>> >>
> > > >> wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> Given that Yuriy says explicitly 'unix socket',
> I dont
> think he
> > > >> means 'MQ' when he says 'RPC'. I think he just
> means a daemon
> > > >> listening on a unix socket for execution
> requests. This
> seems like
> > > >> a reasonably sensible idea to me.
> > > >>
> > > >>
> > > >> Yes, you're right.
> > > >>
> > > >> On 07/03/14 12:52, Miguel Angel Ajo wrote:
> > > >>
> > > >>
> > > >> I thought of this option, but didn't consider
> it, as It's
> somehow
> > > >> risky to expose an RPC end executing
> priviledged (even
> filtered)
> > > >> commands.
> > > >>
> > > >>
> > > >> subprocess module have some means to do RPC
> securely over
> UNIX sockets.
> > > >> I does this by passing some token along with
> messages. It
> should be
> > > >> secure because with UNIX sockets we don't need
> anything
> stronger since
> > > >> MITM attacks are not possible.
> > > >>
> > > >> If I'm not wrong, once you have credentials for
> messaging,
> you can
> > > >> send messages to any end, even filtered, I
> somehow see
> this as a
> > > >> higher
> > > >> risk option.
> > > >>
> > > >>
> > > >> As Stephen noted, I'm not talking about using
> MQ for RPC.
> Just some
> > > >> local UNIX socket with very simple RPC over it.
> > > >>
> > > >> And btw, if we add RPC in the middle, it's
> possible that all
> those
> > > >> system call delays increase, or don't decrease
> all it'll be
> > > >> desirable.
> > > >>
> > > >>
> > > >> Every call to rootwrap would require the following.
> > > >>
> > > >> Client side:
> > > >> - new client socket;
> > > >> - one message sent;
> > > >> - one message received.
> > > >>
> > > >> Server side:
> > > >> - accepting new connection;
> > > >> - one message received;
> > > >> - one fork-exec;
> > > >> - one message sent.
> > > >>
> > > >> This looks like way simpler than passing
> through sudo and
> rootwrap that
> > > >> requires three exec's and whole lot of
> configuration files
> opened and
> > > >> parsed.
> > > >>
> > > >> --
> > > >>
> > > >> Kind regards, Yuriy.
> > > >>
> > > >>
> > > >> _________________________________________________
> > > >> OpenStack-dev mailing list
> > > >> OpenStack-dev at lists.openstack.__org
> <mailto:OpenStack-dev at lists.openstack.org>
> <mailto:OpenStack-dev at lists.__openstack.org
> <mailto:OpenStack-dev at lists.openstack.org>>
> > > >>
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> > > >>
> > > >
> > > > _________________________________________________
> > > > OpenStack-dev mailing list
> > > > OpenStack-dev at lists.openstack.__org
> <mailto:OpenStack-dev at lists.openstack.org>
> <mailto:OpenStack-dev at lists.__openstack.org
> <mailto:OpenStack-dev at lists.openstack.org>>
> > > >
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> > >
> > > _________________________________________________
> > > OpenStack-dev mailing list
> > > OpenStack-dev at lists.openstack.__org
> <mailto:OpenStack-dev at lists.openstack.org>
> <mailto:OpenStack-dev at lists.__openstack.org
> <mailto:OpenStack-dev at lists.openstack.org>>
> > >
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> > >
> >
> > _________________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.__org
> <mailto:OpenStack-dev at lists.openstack.org>
> <mailto:OpenStack-dev at lists.__openstack.org
> <mailto:OpenStack-dev at lists.openstack.org>>
> >
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >
> >
> >
> > _________________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.__org
> <mailto:OpenStack-dev at lists.openstack.org>
> <mailto:OpenStack-dev at lists.__openstack.org
> <mailto:OpenStack-dev at lists.openstack.org>>
> >
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >
> >
> >
> > _________________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.__org
> <mailto:OpenStack-dev at lists.openstack.org>
> <mailto:OpenStack-dev at lists.__openstack.org
> <mailto:OpenStack-dev at lists.openstack.org>>
> >
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> >
>
> _________________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.__org
> <mailto:OpenStack-dev at lists.openstack.org>
> <mailto:OpenStack-dev at lists.__openstack.org
> <mailto:OpenStack-dev at lists.openstack.org>>
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
>
>
> _________________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.__org
> <mailto:OpenStack-dev at lists.openstack.org>
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
> _________________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.__org
> <mailto:OpenStack-dev at lists.openstack.org>
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
> _________________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.__org
> <mailto:OpenStack-dev at lists.openstack.org>
> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
More information about the OpenStack-dev
mailing list