<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Mar 17, 2014 at 4:48 PM, Joe Gordon <span dir="ltr"><<a href="mailto:joe.gordon0@gmail.com" target="_blank">joe.gordon0@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">
<div class="">On Tue, Mar 11, 2014 at 1:46 AM, Miguel Angel Ajo Pelayo <span dir="ltr"><<a href="mailto:mangelajo@redhat.com" target="_blank">mangelajo@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br>
I have included on the etherpad, the option to write a sudo<br>
plugin (or several), specific for openstack.<br>
<br>
<br>
And this is a test with shedskin, I suppose that in more complicated<br>
dependecy scenarios it should perform better.<br>
<br>
[majopela@redcylon tmp]$ cat <<EOF >test.py<br>
> import sys<br>
> print "hello world"<br>
> sys.exit(0)<br>
> EOF<br>
<br>
[majopela@redcylon tmp]$ time python test.py<br>
hello world<br>
<br>
real 0m0.016s<br>
user 0m0.015s<br>
sys 0m0.001s<br></blockquote><div><br></div><div><br></div></div><div>This looks very promising!</div><div><br></div><div>A few gotchas:</div><div><br></div><div>* Very limited library support <a href="https://code.google.com/p/shedskin/wiki/docs#Library_Limitations" target="_blank">https://code.google.com/p/shedskin/wiki/docs#Library_Limitations</a></div>
<div> * no logging</div><div> * no six</div><div> * no subprocess</div><div><br></div><div>* no *args support </div><div> * <a href="https://code.google.com/p/shedskin/wiki/docs#Python_Subset_Restrictions" target="_blank">https://code.google.com/p/shedskin/wiki/docs#Python_Subset_Restrictions</a></div>
<div><br></div><div>that being said I did a quick comparison with great results:</div><div><br></div><div><div>$ cat tmp.sh </div><div>#!/usr/bin/env bash</div><div>echo $0 $@</div><div>ip a</div></div><div><br></div><div>
<div>$ time ./tmp.sh foo bar> /dev/null</div><div><br></div><div>real 0m0.009s</div><div>user 0m0.003s</div><div>sys 0m0.006s</div></div><div><br></div><div><br></div><div><br></div><div><div>$ cat tmp.py </div>
<div>#!/usr/bin/env python</div><div>import os</div><div>import sys</div><div><br></div><div>print sys.argv</div><div>print os.system("ip a")</div><div><br></div><div><div>$ time ./tmp.py foo bar > /dev/null</div>
<div><br></div><div>min:</div><div>real 0m0.016s</div><div>user 0m0.004s</div><div>sys 0m0.012s</div></div></div><div><br></div><div>max:</div><div><div>real 0m0.038s</div><div>user 0m0.016s</div><div>sys 0m0.020s</div>
</div><div><br></div><div><br></div><div><br></div><div>shedskin tmp.py && make<br></div><div><br></div><div><br></div><div><div>$ time ./tmp foo bar > /dev/null</div><div><br></div><div>real 0m0.010s</div>
<div>user 0m0.007s</div><div>sys 0m0.002s</div></div><div><br></div></div></div></div></blockquote><div><br></div><div>for completeness here is the auto generated cpp code: <a href="http://paste.openstack.org/show/73711/">http://paste.openstack.org/show/73711/</a> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div></div><div><br></div><div><br></div><div>Based in these results I think a deeper dive into making rootwrap supportshedskin is worthwhile.</div><div><div class="h5"><div><br>
</div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
<br>
[majopela@redcylon tmp]$ shedskin test.py<br>
*** SHED SKIN Python-to-C++ Compiler 0.9.4 ***<br>
Copyright 2005-2011 Mark Dufour; License GNU GPL version 3 (See LICENSE)<br>
<br>
[analyzing types..]<br>
********************************100%<br>
[generating c++ code..]<br>
[elapsed time: 1.59 seconds]<br>
[majopela@redcylon tmp]$ make<br>
g++ -O2 -march=native -Wno-deprecated -I. -I/usr/lib/python2.7/site-packages/shedskin/lib /tmp/test.cpp /usr/lib/python2.7/site-packages/shedskin/lib/sys.cpp /usr/lib/python2.7/site-packages/shedskin/lib/re.cpp /usr/lib/python2.7/site-packages/shedskin/lib/builtin.cpp -lgc -lpcre -o test<br>
[majopela@redcylon tmp]$ time ./test<br>
hello world<br>
<br>
real 0m0.003s<br>
user 0m0.000s<br>
sys 0m0.002s<br>
<div><div><br>
<br>
----- Original Message -----<br>
> We had this same issue with the dhcp-agent. Code was added that paralleled<br>
> the initial sync here: <a href="https://review.openstack.org/#/c/28914/" target="_blank">https://review.openstack.org/#/c/28914/</a> that made<br>
> things a good bit faster if I remember correctly. Might be worth doing<br>
> something similar for the l3-agent.<br>
><br>
> Best,<br>
><br>
> Aaron<br>
><br>
><br>
> On Mon, Mar 10, 2014 at 5:07 PM, Joe Gordon < <a href="mailto:joe.gordon0@gmail.com" target="_blank">joe.gordon0@gmail.com</a> > wrote:<br>
><br>
><br>
><br>
><br>
><br>
><br>
> On Mon, Mar 10, 2014 at 3:57 PM, Joe Gordon < <a href="mailto:joe.gordon0@gmail.com" target="_blank">joe.gordon0@gmail.com</a> > wrote:<br>
><br>
><br>
><br>
> I looked into the python to C options and haven't found anything promising<br>
> yet.<br>
><br>
><br>
> I tried Cython, and RPython, on a trivial hello world app, but git similar<br>
> startup times to standard python.<br>
><br>
> The one thing that did work was adding a '-S' when starting python.<br>
><br>
> -S Disable the import of the module site and the site-dependent manipulations<br>
> of sys.path that it entails.<br>
><br>
> Using 'python -S' didn't appear to help in devstack<br>
><br>
> #!/usr/bin/python -S<br>
> # PBR Generated from u'console_scripts'<br>
><br>
> import sys<br>
> import site<br>
> site.addsitedir('/mnt/stack/oslo.rootwrap/oslo/rootwrap')<br>
><br>
><br>
><br>
><br>
><br>
><br>
> I am not sure if we can do that for rootwrap.<br>
><br>
><br>
> jogo@dev:~/tmp/pypy-2.2.1-src$ time ./tmp-c<br>
> hello world<br>
><br>
> real 0m0.021s<br>
> user 0m0.000s<br>
> sys 0m0.020s<br>
> jogo@dev:~/tmp/pypy-2.2.1-src$ time ./tmp-c<br>
> hello world<br>
><br>
> real 0m0.021s<br>
> user 0m0.000s<br>
> sys 0m0.020s<br>
> jogo@dev:~/tmp/pypy-2.2.1-src$ time python -S ./tmp.py<br>
> hello world<br>
><br>
> real 0m0.010s<br>
> user 0m0.000s<br>
> sys 0m0.008s<br>
><br>
> jogo@dev:~/tmp/pypy-2.2.1-src$ time python -S ./tmp.py<br>
> hello world<br>
><br>
> real 0m0.010s<br>
> user 0m0.000s<br>
> sys 0m0.008s<br>
><br>
><br>
><br>
> On Mon, Mar 10, 2014 at 3:26 PM, Miguel Angel Ajo Pelayo <<br>
> <a href="mailto:mangelajo@redhat.com" target="_blank">mangelajo@redhat.com</a> > wrote:<br>
><br>
><br>
> Hi Carl, thank you, good idea.<br>
><br>
> I started reviewing it, but I will do it more carefully tomorrow morning.<br>
><br>
><br>
><br>
> ----- Original Message -----<br>
> > All,<br>
> ><br>
> > I was writing down a summary of all of this and decided to just do it<br>
> > on an etherpad. Will you help me capture the big picture there? I'd<br>
> > like to come up with some actions this week to try to address at least<br>
> > part of the problem before Icehouse releases.<br>
> ><br>
> > <a href="https://etherpad.openstack.org/p/neutron-agent-exec-performance" target="_blank">https://etherpad.openstack.org/p/neutron-agent-exec-performance</a><br>
> ><br>
> > Carl<br>
> ><br>
> > On Mon, Mar 10, 2014 at 5:26 AM, Miguel Angel Ajo < <a href="mailto:majopela@redhat.com" target="_blank">majopela@redhat.com</a> ><br>
> > wrote:<br>
> > > Hi Yuri & Stephen, thanks a lot for the clarification.<br>
> > ><br>
> > > I'm not familiar with unix domain sockets at low level, but , I wonder<br>
> > > if authentication could be achieved just with permissions (only users in<br>
> > > group "neutron" or group "rootwrap" accessing this service.<br>
> > ><br>
> > > I find it an interesting alternative, to the other proposed solutions,<br>
> > > but<br>
> > > there are some challenges associated with this solution, which could make<br>
> > > it<br>
> > > more complicated:<br>
> > ><br>
> > > 1) Access control, file system permission based or token based,<br>
> > ><br>
> > > 2) stdout/stderr/return encapsulation/forwarding to the caller,<br>
> > > if we have a simple/fast RPC mechanism we can use, it's a matter<br>
> > > of serializing a dictionary.<br>
> > ><br>
> > > 3) client side implementation for 1 + 2.<br>
> > ><br>
> > > 4) It would need to accept new domain socket connections in green threads<br>
> > > to<br>
> > > avoid spawning a new process to handle a new connection.<br>
> > ><br>
> > > The advantages:<br>
> > > * we wouldn't need to break the only-python-rule.<br>
> > > * we don't need to rewrite/translate rootwrap.<br>
> > ><br>
> > > The disadvantages:<br>
> > > * it needs changes on the client side (neutron + other projects),<br>
> > ><br>
> > ><br>
> > > Cheers,<br>
> > > Miguel Ángel.<br>
> > ><br>
> > ><br>
> > ><br>
> > > On 03/08/2014 07:09 AM, Yuriy Taraday wrote:<br>
> > >><br>
> > >> On Fri, Mar 7, 2014 at 5:41 PM, Stephen Gran<br>
> > >> < <a href="mailto:stephen.gran@theguardian.com" target="_blank">stephen.gran@theguardian.com</a> <mailto: <a href="mailto:stephen.gran@theguardian.com" target="_blank">stephen.gran@theguardian.com</a> >><br>
> > >> wrote:<br>
> > >><br>
> > >> Hi,<br>
> > >><br>
> > >> Given that Yuriy says explicitly 'unix socket', I dont think he<br>
> > >> means 'MQ' when he says 'RPC'. I think he just means a daemon<br>
> > >> listening on a unix socket for execution requests. This seems like<br>
> > >> a reasonably sensible idea to me.<br>
> > >><br>
> > >><br>
> > >> Yes, you're right.<br>
> > >><br>
> > >> On 07/03/14 12:52, Miguel Angel Ajo wrote:<br>
> > >><br>
> > >><br>
> > >> I thought of this option, but didn't consider it, as It's somehow<br>
> > >> risky to expose an RPC end executing priviledged (even filtered)<br>
> > >> commands.<br>
> > >><br>
> > >><br>
> > >> subprocess module have some means to do RPC securely over UNIX sockets.<br>
> > >> I does this by passing some token along with messages. It should be<br>
> > >> secure because with UNIX sockets we don't need anything stronger since<br>
> > >> MITM attacks are not possible.<br>
> > >><br>
> > >> If I'm not wrong, once you have credentials for messaging, you can<br>
> > >> send messages to any end, even filtered, I somehow see this as a<br>
> > >> higher<br>
> > >> risk option.<br>
> > >><br>
> > >><br>
> > >> As Stephen noted, I'm not talking about using MQ for RPC. Just some<br>
> > >> local UNIX socket with very simple RPC over it.<br>
> > >><br>
> > >> And btw, if we add RPC in the middle, it's possible that all those<br>
> > >> system call delays increase, or don't decrease all it'll be<br>
> > >> desirable.<br>
> > >><br>
> > >><br>
> > >> Every call to rootwrap would require the following.<br>
> > >><br>
> > >> Client side:<br>
> > >> - new client socket;<br>
> > >> - one message sent;<br>
> > >> - one message received.<br>
> > >><br>
> > >> Server side:<br>
> > >> - accepting new connection;<br>
> > >> - one message received;<br>
> > >> - one fork-exec;<br>
> > >> - one message sent.<br>
> > >><br>
> > >> This looks like way simpler than passing through sudo and rootwrap that<br>
> > >> requires three exec's and whole lot of configuration files opened and<br>
> > >> parsed.<br>
> > >><br>
> > >> --<br>
> > >><br>
> > >> Kind regards, Yuriy.<br>
> > >><br>
> > >><br>
> > >> _______________________________________________<br>
> > >> OpenStack-dev mailing list<br>
> > >> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> > >> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> > >><br>
> > ><br>
> > > _______________________________________________<br>
> > > OpenStack-dev mailing list<br>
> > > <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> > > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> ><br>
> > _______________________________________________<br>
> > OpenStack-dev mailing list<br>
> > <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
> ><br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div></div>