[openstack-dev] [Neutron] _get_subnet() in OpenContrail tests results in port deletion

Pavel Bondar pbondar at infoblox.com
Thu Apr 30 13:37:31 UTC 2015


I am debugging issue observed in OpenContrail tests[1] and so far it
does not look obvious.


In create_port[2] new transaction is started.
Port gets created, but disappears right after reading subnet from plugin
in reference ipam driver[3]:

>    plugin = manager.NeutronManager.get_plugin()
>    return plugin._get_subnet(context, id)

Port no longer seen in transaction, like it never existed before
(magic?). As a result inserting IPAllocation fails with foreing key
constraint error:

DBReferenceError: (IntegrityError) FOREIGN KEY constraint failed
u'INSERT INTO ipallocations (port_id, ip_address, subnet_id, network_id)
VALUES (?, ?, ?, ?)' ('aba6eaa2-2b2f-4ab9-97b0-4d8a36659363',
u'', u'be7bb05b-d501-4cf3-a29a-3861b3b54950',

Only OpenContrail tests fail with that error (116 failures[1]). Tests
for other plugin passes fine. As I see OpenContrail is different from
other plugins: each call to plugin is wrapped into http request, so
getting subnet happens in another transaction. In tests requests.post()
is mocked and http call gets translated into self.get_subnet(...).
Stack trace from plugin._get_subnet() to db_base get_subnet() in open
contrail tests looks next[4].

Also single test failure with full db debug was uploaded for
- Port is inserted at 362.
- Subnet is read by plugin at 384.
- IPAllocation was tried to be inserted at 407.
Between Port and IPAllocation insert no COMMIT/ROLLBACK or delete
statement were issued, so can't find explanation why port no longer
exists on IPAllocation insert step.
Am I missing something obvious?

For now I have several workarounds, which are basically do not use
plugin._get_subnet(). Direct session.query() works without such side
But this issue bothers me much since I can't explain why it even happens
in OpenContrail tests.
Any ideas are welcome!

My best theory for now: OpenContrail silently wipes currently running
transaction in tests (in this case it doesn't sound good).

Anyone can checkout and debug patch set 50 (where issue is observed)
from review page[6].

Thank you in advance.

- Pavel Bondar

line 1578 / line 1857
line 50
[4] http://pastebin.com/n0AqhC5x
[5] http://pastebin.com/BDBAXFy9
[6] http://logs.openstack.org/36/153236/

More information about the OpenStack-dev mailing list