Open Stack

Hi All:

In digging thru this, yes with the neutron change, it changed the MRO as below and thus the issue.


(<class 'neutron_fwaas.services.firewall.agents.varmour.varmour_router.vArmourL3NATAgent'>, <class 'neutron.agent.l3.agent.L3NATAgent'>, <class 'neutron.agent.l3.ha.AgentMixin'>, <class 'neutron.agent.l3.dvr.AgentMixin'>, <class 'neutron.manager.Manager'>, <class 'oslo_service.periodic_task.PeriodicTasks'>,

<—— the issue is at this point where we have a mismatch with args

<class 'neutron_fwaas.services.firewall.agents.l3reference.firewall_l3_agent.FWaaSL3AgentRpcCallback'>, <class 'neutron_fwaas.services.firewall.agents.firewall_agent_api.FWaaSAgentRpcCallbackMixin'>, <type 'object’>)


Nate, Margaret – thanks for digging thru this – lets get together during the day to discuss this more. Margaret, to answer ur question – it worked before due to a favorable ordering with the older hacked inheritance relationship. We can find a way to fix this in fwaas but more importantly need to get some missing pieces in to the Observer Hierarchy patch set as well.


Thanks


Sridar

From: Doug Wiegley <dougwig at parksidesoftware.com<mailto:dougwig at parksidesoftware.com>>
Reply-To: OpenStack List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Thursday, May 5, 2016 at 9:40 PM
To: OpenStack List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [neutron][FWaaS] __init__ arguments issue status

This break is almost certainly because of the following neutron change, to unwind the incestuous inheritance that was in neutron (dependency arrow was circular):

https://review.openstack.org/#/c/223343/

I don’t expect there will be a lot of appetite to revert that, so it will need to be addressed in neutron-fwaas. Likely it should’ve had an ML warning first, sorry about that, this has been a longstanding issue.

doug



On May 5, 2016, at 7:00 PM, Frances, Margaret <Margaret_Frances at cable.comcast.com<mailto:Margaret_Frances at cable.comcast.com>> wrote:

Hi Doug.

The old and new MROs are both pretty complicated, and it’s not entirely clear to me yet why the original one worked. (The MROs are included below for reading pleasure; they're embellished to show the incoming args to self’s init and outgoing args to super’s init in each case.)

I’m fairly sure the APIs for the mixins can be made the same, and I’ll try that.  But I still wonder if in fact the problem is a base class ordering issue.

The error that 223343 produced occurs in method call #6 in the "AFTER" MRO, where we get the following trace:

super(PeriodicTasks, self).__init__()
TypeError: __init__() takes exactly 2 arguments (1 given)


For grins, we changed PeriodicTasks’s call to super init as suggested by the trace:

super(PeriodicTasks, self).__init__(conf)


At this point FWaaSAgentRpcCallbackMixin (AFTER, #8) complained:

super(FWaaSAgentRpcCallbackMixin, self).__init__(host)
TypeError: object.__init__() takes no parameters


Changing *that* class as suggested elicited the following (to me baffling) result:

super(FWaaSAgentRpcCallbackMixin, self).__init__()
TypeError: __init__() takes exactly 2 arguments (1 given)


I find it baffling because FWaaSAgentRpcCallbackMixin is the end of the line, it’s a subclass of object, and object doesn’t allow arguments to init (so whose init is that? that’s the next thing I’m going to look at).  (It’s for these same reasons that I don’t understand why things worked before the 223343 change.)

I’m still looking at things.  (And learning about MRO, which I’ve never really dealt with before.)  Will run pdb and see what surfaces.

Thanks for your help.  Thoughts, comments, suggestions all welcome.
Margaret


BEFORE 223343
 1. varmour_router_vArmourL3NATAgent (host, conf)-->(host, conf)
 2. agent_L3NATAgent  (host, conf)-->(conf)
 3. firewall_l3_agent_FWaaSL3AgentRpcCallback (conf)-->(host)
 4. api_FWaaSAgentRpcCallbackMixin (host)-->(host)
 5. ha_AgentMixin (host)-->(host)
 6. dvr_AgentMixin (host)-->(host)
 7. manager_Manager (host)-->(conf)
 8. periodic_task_PeriodicTasks(conf)-->()
 9. firewall_l3_agent_FWaaSL3AgentRpcCallback(conf)-->(host)
10. api_FWaaSAgentRpcCallbackMixin(host)-->(host)
11. object

AFTER 223343
 1. varmour_router_vArmourL3NATAgent (host, conf)-->(host, conf)
 2. agent_L3NATAgent  (host, conf)-->(host)
 3. ha_AgentMixin (host)-->(host)
 4. dvr_AgentMixin (host)-->(host)
 5. manager_Manager (host)-->(conf)
 6. periodic_task_PeriodicTasks(conf)-->()
 7. firewall_l3_agent_FWaaSL3AgentRpcCallback (conf)-->(host)
 8. api_FWaaSAgentRpcCallbackMixin(host)-->(host)
 9. object

--
Margaret Frances
Eng 4, Prodt Dev Engineering



On May 5, 2016, at 7:06 PM, Doug Hellmann <doug at doughellmann.com<mailto:doug at doughellmann.com>> wrote:

Excerpts from Nate Johnston's message of 2016-05-05 17:40:13 -0400:
FWaaS team,

After a day of looking at the tests currently failing in the FWaaS repo, I
believe I have the issue narrowed down considerably. First, to restate what
is going on.  If you check out the neutron-fwaas repository and run `tox -e
py27` in it, you will get six errors all in the
neutron_fwaas.tests.unit.services.firewall.agents.varmour.test_varmour_router.TestVarmourRouter
section.
Running the py34 tests results in similar problems.  The failures follow
the following form:

Captured traceback:

~~~~~~~~~~~~~~~~~~~

   Traceback (most recent call last):

     File
"neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
line 190, in test_agent_external_gateway

       router = self._create_router()

     File
"neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
line 87, in _create_router

       router = varmour_router.vArmourL3NATAgent(HOSTNAME, self.conf)

     File
"neutron_fwaas/services/firewall/agents/varmour/varmour_router.py", line
54, in __init__

       super(vArmourL3NATAgent, self).__init__(host, conf)

     File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/agent.py<http://github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/agent.py>",
line 244, in __init__

       super(L3NATAgent, self).__init__(host=self.conf.host)

     File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/ha.py<http://github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/ha.py>",
line 93, in __init__

       super(AgentMixin, self).__init__(host)

     File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/dvr.py<http://github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/dvr.py>",
line 30, in __init__

       super(AgentMixin, self).__init__(host)

     File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/agent.py<http://github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/agent.py>",
line 70, in __init__

       super(FWaaSAgentRpcCallbackMixin, self).__init__(host)

     File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/manager.py<http://github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/manager.py>",
line 44, in __init__

       super(Manager, self).__init__(conf)

     File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/lib/python2.7/site-packages/oslo_service/periodic_task.py<http://github.com/openstack/neutron-fwaas-311159/.tox/py27/lib/python2.7/site-packages/oslo_service/periodic_task.py>",
line 177, in __init__

       super(PeriodicTasks, self).__init__()

   TypeError: __init__() takes exactly 2 arguments (1 given)

I pinged the Oslo folks, and they were able to help me look into the issue,
which cleared oslo.service of any role.  The change that introduced this is
actually a neutron change - https://review.openstack.org/#/c/223343/ - and
I could experimentally test for it by doing the following, which checks out
the change before the problem one, "Remove BGP code from neutron".  At that
point `tox -e py27` could complete successfully.

Everything works with the older commit:   cd .tox/py27/src/neutron && git
checkout fe702f8f2af265554c7ff6f464b99562f8c54254 && cd - && tox -e py27
Things break with commit 223343:          cd .tox/py27/src/neutron && git
checkout f31861843d90e013d31fb76fc576b49a35e218aa4  && cd - && tox -e py27

My guess on this is that the reason for the breakage is due to multiple
inheritance and the changing of the ancestry for the L3NATAgent object.  So
the focus of my effort (with Margaret Frances providing crucial insight) is
determining what precisely needs to be fixed or reverted to make this work,
while in keeping with the removal of FWaaS code from Neutron.  I shall
continue to look at this tomorrow, but if anyone wishes to pick up the
torch and figure this out then you should feel free to do so.  If not, I
shall resume tomorrow.

Thanks,

--N.

Based on looking at the class hierarchy for vArmourL3NATAgent, I think
the problem is that the mixin classes up and down that hierarchy don't
take the same arguments for __init__(). That's going to make using super()
difficult. Normally one would just need to switch the order of the base
classes so all of the mixins are initialized before the Manager and
PeriodicTask base classes, but doing that doesn't fix the problem in
this case.

Is it possible to make all of the mixin APIs the same?

Doug

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org<mailto:OpenStack-dev-request at lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org<mailto:OpenStack-dev-request at lists.openstack.org>?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160506/686dc779/attachment.html>