Open Stack

Excerpts from Jianghua Wang's message of 2016-11-02 15:52:22 +0000:
> Thanks Doug. Please see my response inline starts with <jianghuaw>.
> 
> Jianghua
> 
> -----Original Message-----
> From: Doug Hellmann [mailto:doug at doughellmann.com] 
> Sent: Wednesday, November 2, 2016 9:31 PM
> To: openstack-dev <openstack-dev at lists.openstack.org>
> Subject: Re: [openstack-dev] [neutron][oslo] proposal to resolve a rootwrap problem for XenServer
> 
> Excerpts from Jianghua Wang's message of 2016-11-02 04:14:48 +0000:
> > Ihar and Tony,
> >  Thanks for the input.
> >  In order to run command in dom0, it uses XenAPI to create a session which can be used to remotely call a plugin - netwrap which is located in dom0. The netwrap plugin is executed as root. It will validate the command basing on the allowed command list and execute it.
> > The source code for netwrap is in neutron project: 
> > https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/d
> > rivers/openvswitch/agent/xenapi/etc/xapi.d/plugins/netwrap
> > 
> > So at least we can see there are two dependences: 
> > 1. it depends on XenAPI which is XenServer specific.
> > 2. it depends on Neutron's plugin netwrap.
> > Is it acceptable to add such dependences in this common library of oslo.rootwrap? 
> 
> Why would they need to be dependencies of oslo.rootwrap? They are dependencies of the driver, not the base library, right?
> 
> <jianghuaw> With a second thought, I think we can pass the plugin name netwrap as a parameter to the rootwrap; so maybe not a dependence. But if we host the XenAPI session creation in oslo.rootwrap, I think we should import XenAPI in oslo.rootwrap. Then it is a dependence in the base library, isn't it?

I don't think we want to build Xen-specific features or dependencies
into any of the Oslo libraries unless we absolutely can't avoid it.

> 
> > And most of the code in oslo.rootwrap is to:
> > 1. spawn a daemon process and maintain the connection between the 
> > client and daemon; 2. filter commands in the daemon process.
> > But both can't be re-used for this XenAPI/XenServer case as the daemon process is already running in dom0; the command filtering is done in dom0's netwrap plugin. In order to hold this in oslo.rootwrap, it requires some refactoring work to make it looks reasonable. Is it worthy to do that? Specially by considering it has determined to replace oslo.wrap with oslo.provsep?
> > 
> > Maybe it's a good option to cover this dom0 case in oslo.provsep at the beginning. But it becomes more complicated. Maybe we can run a daemon process in dom0 with the privileges set properly and listening on a dedicated tcp port . But that's much different from the initial provsep design [1]. And also it makes the mechanism very different from the current XenServer OpenStack which is using XenAPI plugin. Anyway, I think we have long to go with a good solution to cover it in provsep.
> 
> What sort of refactoring do you have in mind for privsep? I could see something analogous to the preexec_fn argument to subprocess.Popen() to let the XenServer driver ensure that its privileged process is running in dom0.
> 
> <jianghuaw>Sorry, I don't have a clear idea on refactorying in privsep now. It seems privsep attempts to create a daemon process and set caps for this daemon. But for XenServer/XenAPI, the daemon and caps in daemon seems useless. As it sends all commands to the a common XAPI daemon running in dom0. No additional daemon is needed. TBH I don't know how to apply caps at here. But to make it to resolve the current issue, what I'm thinking is to create a XenAPI session and keep it in the rootwrap's client; then each command will be passed to dom0 via this same session.

OK. I think Thierry's question in the other thread (about why the
XenAPI calls have to be made from a privileged process at all) is
useful for thinking about any API changes. Let's keep the discussion
over there to avoid drift or confusion.

Doug

> 
> Doug
> 
> > 
> > But this issue is urgent for XenAPI/XenServer OpenStack. Please the details described in the bug[2]. So I still think the PoC is a better option, unless both oslo and Neutron guys agree it's acceptable to refactor oslo.rootwrap and allow the above dependences introduced to this library.
> > 
> > [1]https://specs.openstack.org/openstack/oslo-specs/specs/liberty/priv
> > sep.html [2] https://bugs.launchpad.net/neutron/+bug/1585510
> > 
> > Regards,
> > Jianghua
> > 
> > On Tue, Nov 01, 2016 at 12:45:43PM +0100, Ihar Hrachyshka wrote:
> > 
> > > I suggested in the bug and the PoC review that neutron is not the 
> > > right project to solve the issue. Seems like oslo.rootwrap is a 
> > > better place to maintain privilege management code for OpenStack. 
> > > Ideally, a solution would be found in scope of the library that 
> > > would not require any changes per-project.
> > 
> > With the change of direction from oslo.roowrap to oslo.provsep I doubt that there is scope to land this in oslo.rootwarp.
> > 
> > Yours Tony.
> > 
> > -----Original Message-----
> > From: Ihar Hrachyshka [mailto:ihrachys at redhat.com]
> > Sent: Tuesday, November 01, 2016 7:46 PM
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [neutron] proposal to resolve a rootwrap 
> > problem for XenServer
> > 
> > Jianghua Wang <jianghua.wang at citrix.com> wrote:
> > 
> > > Hi Neutron guys,
> > >
> > > I’m trying to explain a problem with the XenServer rootwrap and give 
> > > a proposal to resolve it. I need some input on how to proceed with 
> > > this
> > > proposal: e.g. if requires a spec? Any concerns need further 
> > > discussion or clarification?
> > >
> > > Problem description:
> > > As we’ve known, some neutron services need run commands with root 
> > > privileges and it’s achieved by running commands via the rootwrap. 
> > > And in order to resolve performance issue, it has been improved to 
> > > support daemon mode for the rootwrap [1]. Either way has the 
> > > commands running on the same node/VM which has relative neutron services running on.
> > >
> > > But as a type-1 hypervisor, XenServer OpenStack has different behavior.  
> > > Neutron’s compute agent neutron-openvswitch-agent need run commands 
> > > in dom0, as the tenants’ interfaces are plugged in an integration 
> > > OVS which locates in Dom0. Currently the script of
> > > https://github.com/openstack/neutron/blob/master/bin/neutron-rootwra
> > > p- xen-dom0is used as XenServer OpenStack’s rootwrap. This script 
> > > will create a XenAPI session with dom0 and passes the commands to 
> > > dom0 for the real execution.
> > > Each command execution will run this script once. So it has the 
> > > similar performance issue as the non-daemon mode of rootwrap on 
> > > other
> > > hypervisors:  For each command, it has to parse the
> > > neutron-rootwrap-xen-dom0 script and the rootwrap configure file.  
> > > Furthermore, this rootwrap script will create a XenAPI for each 
> > > command execution and XenServer by default will log the XenAPI 
> > > session creation events. It will cause frequent log file rotation 
> > > and so other real useful log is lost.
> > >
> > > Proposal:
> > > The os.rootwrap support daemon mode for other hypervisors; but 
> > > XenServer’s compute agent can’t use that as again it need run 
> > > commands in Dom0. But we can refer to that design and implement the 
> > > daemon mode for XenServer. After creating a XenAPI session, Dom0’s 
> > > XAPI will accept the command running requests from the session and 
> > > reply with the running result. So logically we’ve had a daemon in 
> > > dom0. So we can support daemon mode rootwrap with the following design:
> > > 1. Develop a daemon client module for XenServer: The agent service 
> > > will use this client module to create a XenAPI session, and keep 
> > > this session during the service’s whole life.
> > > 2. once need run command on dom0, use the above client to runs 
> > > commands in dom0.
> > > It should be able to result the issues mentioned above, as the 
> > > client module need import only once for each agent service and only 
> > > use a single session for all commands. The prototype code[3] works well.
> > >
> > > Any concern or comments for the above proposal? And how I can 
> > > proceed with solution? We’ve filed a RFE bug[2] which is in 
> > > wishlist&incomplete status. Per the neutron policy[4], it seems need 
> > > neutron-drivers team to evaluate the RFE and determine if a spec is 
> > > required. Could anyone help to evaluate this proposal and tell me 
> > > how I should proceed? And I’m also open and happy for any comments. Thanks very much.
> > >
> > > [1]
> > > https://specs.openstack.org/openstack/oslo-specs/specs/juno/rootwrap
> > > -daemon-mode.html [2] 
> > > https://bugs.launchpad.net/neutron/+bug/1585510
> > > [3]prototype code: https://review.openstack.org/#/c/390931/
> > > [4] 
> > > http://docs.openstack.org/developer/neutron/policies/blueprints.html
> > >
> > 
> > I suggested in the bug and the PoC review that neutron is not the 
> > right project to solve the issue. Seems like oslo.rootwrap is a better 
> > place to maintain privilege management code for OpenStack. Ideally, a 
> > solution would be found in scope of the library that would not require 
> > any changes per-project.
> > 
> > I moved the bug to Opinion since I don’t believe it’s in scope for 
> > neutron; I also added oslo.rootwrap to the list of affected projects 
> > to collect feedback from oslo folks. Finally, I blocked the PoC patch 
> > with -2 until we agree on how to scope the feature for neutron.
> > 
> > I hope it helps,
> > Ihar
> > 
> > ______________________________________________________________________
> > ____ OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: 
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > ______________________________________________________________________
> > ____ OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: 
> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev