[openstacksdk] Flakey unit test makes assumptions about the speed of API results
So I've spent a day or so debugging an occasionally flakey unit test that I see in the py3 test suite for openstacksdk. I'm at the point that I want to check that I actually understand what it's doing before proposing a fix. Specifically, openstack.tests.unit.test_stats.TestStats.test_servers_no_detail periodically fails to find a statsd value: self.assert_reported_stat( 'openstack.api.compute.GET.servers.200', value='0', kind='ms' ) This assertion is effectively saying that a GET to the servers compute API should return a HTTP 200, and take 0 milliseconds to execute. The unit test is failing because sometimes it doesn't take 0 milliseconds. In my debugging for example, it takes 4 milliseconds in my environment every now and then: Considering key openstack.api.compute.GET.servers.200, value 4.000000|ms Value 4.000000, kind ms Value 0 != 4.000000 Am I understanding this correctly? It seems to me that perhaps we should be asserting the call took less than a threshold (5 milliseconds) instead of just insisting it always be instantaneous? Michael
participants (1)
-
Michael Still