Open Stack

On Fri, May 23, 2014, Rick Harris <rconradharris at gmail.com> wrote:
> On Thu, May 22, 2014 at 7:31 PM, Johannes Erdfelt <johannes at erdfelt.com>wrote:
> 
> > I noticed recently that some tests are being skipped in the Nova gate.
> >
> > Some will always be skipped, but others are conditional.
> 
> I'd like to hear a bit more about why some will always be skipped.
> 
> If it's a Python 2.6 vs Python 2.7 thing, perhaps we should forgo the
> conveniences of 2.7 in places so that we can avoid skipping _any_ tests.

As an example of a test skipped on Python 2.6:

image/test_glance.py:TestGlanceApiServers.test_get_ipv6_api_servers

    # Python 2.6 can not parse ipv6 address correctly
    @testtools.skipIf(sys.version_info < (2, 7), "py27 or greater only")

Python 2.6 certainly has ipv6 support (just tested it) so I'm guessing
this is because of a third-party library or something? Could maybe be a
problem parsing IPv6 addresses from URLs (so in httplib?).

db/test_db_api.py:ArchiveTestCase.test_archive_deleted_rows_fk_constraint

This is an indirect Python 2.6 problem. The version of sqlite3 shipping
with Python 2.6 doesn't support foreign key constraints reliably (from
the comments).


There are some tests that get skipped if the host doesn't have IPv6
support at all:

test_service.py:TestWSGIService.test_service_random_port_with_ipv6


Some are tests are skipped only on Mac OS X. Is that a supported
configuration?

virt/test_virt_disk_vfs_localfs.py:VirtDiskVFSLocalFSTestPaths.test_check_safe_path
virt/test_virt_disk_vfs_localfs.py:VirtDiskVFSLocalFSTestPaths.test_check_unsafe_path


And of course the ZooKeeper example I mentioned:

servicegroup/test_zk_driver.py:ZKServiceGroupTestCase.setUp


There are some tests which are skipped unconditionally. These are
probably not as big of a risk since the behavior would be identical if
the test just wasn't there at all.

Some tests are skipped because of known bugs:

db/test_db_api.py:InstanceSystemMetadataTestCase.test_instance_system_metadata_update_nonexistent

This is because foreign key constraints are missing in sqlite so the
test can erroneously pass with that setup.


Some tests are skipped because of known missing features:

compute/test_compute_cells.py:CellsComputeAPITestCase.test_instance_metadata
compute/test_compute_cells.py:CellsComputeAPITestCase.test_evacuate


One skipped test is a little confusing. This test seems to be skipped
because it was too hard to write?

virt/test_virt_drivers.py:LibvirtConnTestCase.test_migrate_disk_and_power_off

> > Any opinions on this?
> 
> I'm in favor of asserting num_skipped_tests == 0 at the gate.
> 
> A gate with a side-door that's always open isn't much of a gate.

That's a much better way of wording it :)

JE