Open Stack

On 11/18/2014 10:17 AM, Victor Sergeyev wrote:
> Matt,
>
> As for race in Nova - it caused by deprecated is_backend_avail()
> function, which calls _ensure_backenv_available() method, which creates
> a SQLAlchemy engine and opens a test connection, but doesn't call
> engine.dispose(). Depending on Python interpreter version, this
> connection may remain open for some time.
>
> So there are such ways to fix Nova:
> - wait for oslo.db 1.1.1 which will include fix for this method - see
> patch [1]
> - remove is_backend_avail() helper usage in Nova - patch [2] refactor
> Nova opportunistic DB tests and remove that method.
>
> [1] https://review.openstack.org/#/c/135293/
> [2] https://review.openstack.org/#/c/103920/
>
>
> On Tue, Nov 18, 2014 at 5:22 AM, Matt Riedemann
> <mriedem at linux.vnet.ibm.com <mailto:mriedem at linux.vnet.ibm.com>> wrote:
>
>
>
>     On 11/17/2014 9:36 AM, Victor Sergeyev wrote:
>
>         Hello All!
>
>         Oslo team is pleased to announce the new release of Oslo database
>         handling library - oslo.db 1.1.0
>
>         List of changes:
>         $ git log --oneline --no-merges  1.0.2..master
>         1b0c2b1 Imported Translations from Transifex
>         9aa02f4 Updated from global requirements
>         766ff5e Activate pep8 check that _ is imported
>         f99e1b5 Assert exceptions based on API, not string messages
>         490f644 Updated from global requirements
>         8bb12c0 Updated from global requirements
>         4e19870 Reorganize DbTestCase to use provisioning completely
>         2a6dbcd Set utf8 encoding for mysql and postgresql
>         1b41056 ModelsMigrationsSync: Add check for foreign keys
>         8fb696e Updated from global requirements
>         ba4a881 Remove extraneous vim editor configuration comments
>         33011a5 Remove utils.drop_unique_constraint()
>         64f6062 Improve error reporting for backend import failures
>         01a54cc Ensure create_engine() retries the initial connection test
>         26ec2fc Imported Translations from Transifex
>         9129545 Use fixture from oslo.config instead of oslo-incubator
>         2285310 Move begin ping listener to a connect listener
>         7f9f4f1 Create a nested helper function that will work on py3.x
>         b42d8f1 Imported Translations from Transifex
>         4fa3350 Start adding a environment for py34/py33
>         b09ee9a Explicitly depend on six in requirements file
>         7a3e091 Unwrap DialectFunctionDispatcher from itself.
>         0928d73 Updated from global requirements
>         696f3c1 Use six.wraps instead of functools.wraps
>         8fac4c7 Update help string to use database
>         fc8eb62 Use __qualname__ if we can
>         6a664b9 Add description for test_models_sync function
>         8bc1fb7 Use the six provided iterator mix-in
>         436dfdc ModelsMigrationsSync:add correct server_default check
>         for Enum
>         2075074 Add history/changelog to docs
>         c9e5fdf Add run_cross_tests.sh script
>
>         Thanks Andreas Jaeger, Ann Kamyshnikova, Christian Berendt, Davanum
>         Srinivas, Doug Hellmann, Ihar Hrachyshka, James Carey, Joshua
>         Harlow,
>         Mike Bayer, Oleksii Chuprykov, Roman Podoliaka for contributing
>         to this
>         release.
>
>         Please report issues to the bug tracker:
>         https://bugs.launchpad.net/__oslo.db
>         <https://bugs.launchpad.net/oslo.db>
>
>
>         _________________________________________________
>         OpenStack-dev mailing list
>         OpenStack-dev at lists.openstack.__org
>         <mailto:OpenStack-dev at lists.openstack.org>
>         http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
>         <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
>     And...the nova postgresql opportunistic DB tests are failing quite
>     frequently due to some race introduced by the new library version [1].
>
>     [1] https://bugs.launchpad.net/__oslo.db/+bug/1393633
>     <https://bugs.launchpad.net/oslo.db/+bug/1393633>
>
>     --
>
>     Thanks,
>
>     Matt Riedemann
>
>
>
>     _________________________________________________
>     OpenStack-dev mailing list
>     OpenStack-dev at lists.openstack.__org
>     <mailto:OpenStack-dev at lists.openstack.org>
>     http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Could https://review.openstack.org/#/c/135293/ have any impact on unit 
tests that hit a backing sqlite database, even non-db migration unit tests?

The reason I ask is running the full nova tox -re py27 job on my 4 VCPUs 
Ubuntu Trusty (python 2.7.6) is taking noticeably longer recently. I 
don't see any recent changes to the nova.tests.integrated tree that 
might have caused a problem, that doesn't mean it might not be something 
else that recently changed in nova, just grasping at straws right now 
(for all I know I've got some rogue process on my system eating CPU and 
RAM too).

-- 

Thanks,

Matt Riedemann