Open Stack

I think the first workaround is the solution we're looking for as it better
reflects the fact that opencontrail is a db-less plugin.
I hope it will be the easier too, but you can never be too sure with
neutron unit tests.

Salvatore

On 4 May 2015 at 12:56, Pavel Bondar <pbondar at infoblox.com> wrote:

> Hi Kevin,
>
> Thanks for your answer, that is what I was looking for!
> I'll check with you in irc to decide which workaround is better:
> 1. Mocking NeutronDbSubnet fetch_subnet for opencontrail tests.
> 2. Using session.query() directly in NeutronDbSubnet fetch_subnet.
>
> - Pavel Bondar
>
> On 30.04.2015 22:46, Kevin Benton wrote:
> > The OpenContrail plugin itself doesn't even use the Neutron DB. I
> > believe what you are observing is a side effect of the fake server they
> > have for their tests, which does inherit the neutron DB.
> >
> > When you call a method on the core plugin in the contrail unit test
> > case, it will go through their request logic and will be piped into the
> > fake server. During this time, the db session that was associated with
> > the original context passed to the core plugin will be lost do to its
> > conversion to a dict.[1, 2]
> >
> > So I believe what you're seeing is this.
> >
> > 1. The FakeServer gets create_port called and starts its transactions.
> > 2. It now hits the ipam driver which calls out to the neutron manager to
> > get the core plugin handle, which is actually the contrail plugin and
> > not the FakeServer.
> > 3. IPAM calls _get_subnet on the contrail plugin, which serializes the
> > context[1] and sends it to the FakeServer.
> > 4. The FakeServer code receives the request and deserializes the
> > context[2], which no longer has the db session.
> > 5. The FakeServer then ends up starting a new session to read the
> > subnet, which will interfere with the transaction you created the port
> > under since they are from the same engine.
> >
> > This is why you can query the DB directly rather than calling the core
> > plugin. The good news is that you don't have to worry because the actual
> > contrail plugin won't be using any of this logic so you're not actually
> > breaking anything.
> >
> > I think what you'll want to do is add a mock.patch for the
> > NeutronDbSubnet fetch_subnet method to monkey patch in a reference to
> > their FakeServer's _get_subnet method. Ping me on IRC (kevinbenton) if
> > you need help.
> >
> > 1.
> >
> https://github.com/openstack/neutron/blob/master/neutron/plugins/opencontrail/contrail_plugin.py#L111
> > 2.
> >
> https://github.com/openstack/neutron/blob/master/neutron/tests/unit/plugins/opencontrail/test_contrail_plugin.py#L121
> >
> > On Thu, Apr 30, 2015 at 6:37 AM, Pavel Bondar <pbondar at infoblox.com
> > <mailto:pbondar at infoblox.com>> wrote:
> >
> >     Hi,
> >
> >     I am debugging issue observed in OpenContrail tests[1] and so far it
> >     does not look obvious.
> >
> >     Issue:
> >
> >     In create_port[2] new transaction is started.
> >     Port gets created, but disappears right after reading subnet from
> plugin
> >     in reference ipam driver[3]:
> >
> >     >    plugin = manager.NeutronManager.get_plugin()
> >     >    return plugin._get_subnet(context, id)
> >
> >     Port no longer seen in transaction, like it never existed before
> >     (magic?). As a result inserting IPAllocation fails with foreing key
> >     constraint error:
> >
> >     DBReferenceError: (IntegrityError) FOREIGN KEY constraint failed
> >     u'INSERT INTO ipallocations (port_id, ip_address, subnet_id,
> network_id)
> >     VALUES (?, ?, ?, ?)' ('aba6eaa2-2b2f-4ab9-97b0-4d8a36659363',
> >     u'10.0.0.2', u'be7bb05b-d501-4cf3-a29a-3861b3b54950',
> >     u'169f6a61-b5d0-493a-b7fa-74fd5b445c84')
> >     }}}
> >
> >     Only OpenContrail tests fail with that error (116 failures[1]). Tests
> >     for other plugin passes fine. As I see OpenContrail is different from
> >     other plugins: each call to plugin is wrapped into http request, so
> >     getting subnet happens in another transaction. In tests
> requests.post()
> >     is mocked and http call gets translated into self.get_subnet(...).
> >     Stack trace from plugin._get_subnet() to db_base get_subnet() in open
> >     contrail tests looks next[4].
> >
> >     Also single test failure with full db debug was uploaded for
> >     investigation[5]:
> >     - Port is inserted at 362.
> >     - Subnet is read by plugin at 384.
> >     - IPAllocation was tried to be inserted at 407.
> >     Between Port and IPAllocation insert no COMMIT/ROLLBACK or delete
> >     statement were issued, so can't find explanation why port no longer
> >     exists on IPAllocation insert step.
> >     Am I missing something obvious?
> >
> >     For now I have several workarounds, which are basically do not use
> >     plugin._get_subnet(). Direct session.query() works without such side
> >     effects.
> >     But this issue bothers me much since I can't explain why it even
> happens
> >     in OpenContrail tests.
> >     Any ideas are welcome!
> >
> >     My best theory for now: OpenContrail silently wipes currently running
> >     transaction in tests (in this case it doesn't sound good).
> >
> >     Anyone can checkout and debug patch set 50 (where issue is observed)
> >     from review page[6].
> >
> >     Thank you in advance.
> >
> >     - Pavel Bondar
> >
> >     [1]
> >
> http://logs.openstack.org/36/153236/50/check/gate-neutron-python27/dd83d43/testr_results.html.gz
> >     [2]
> >
> https://review.openstack.org/#/c/153236/50/neutron/db/db_base_plugin_v2.py
> >     line 1578 / line 1857
> >     [3]
> >
> https://review.openstack.org/#/c/153236/50/neutron/ipam/drivers/neutrondb_ipam/driver.py
> >     line 50
> >     [4] http://pastebin.com/n0AqhC5x
> >     [5] http://pastebin.com/BDBAXFy9
> >     [6] http://logs.openstack.org/36/153236/
> >
> >
>  __________________________________________________________________________
> >     OpenStack Development Mailing List (not for usage questions)
> >     Unsubscribe:
> >     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >     <
> http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> >     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> >
> > --
> > Kevin Benton
> >
> >
> >
> __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150504/94f3658f/attachment.html>