Open Stack

On 12/12/2018 2:00 PM, Matt Riedemann wrote:
> I wanted to send this separate from the latest gate status update [1] 
> since it's primarily about latent cinder bugs causing failures in the 
> gate for which no one is really investigating.
> 
> Running down our tracked gate bugs [2] there are several related to 
> cinder-backup testing:
> 
> * http://status.openstack.org/elastic-recheck/#1483434
> * http://status.openstack.org/elastic-recheck/#1745168
> * http://status.openstack.org/elastic-recheck/#1739482
> * http://status.openstack.org/elastic-recheck/#1635643
> 
> All of those bugs were reported a long time ago. I've done some 
> investigation into them (at least at the time of reporting) and some are 
> simply due to cinder-api using synchronous RPC calls to cinder-volume 
> (or cinder-backup) and that doesn't scale. This bug isn't a backup 
> issue, but it's definitely related to using RPC call rather than cast:
> 
> http://status.openstack.org/elastic-recheck/#1763712
> 
> Regarding the backup tests specifically, I don't see a reason why they 
> need to be run in the integrated gate jobs, e.g. tempest-full(-py3). 
> They don't involve other services, so in my opinion we should move the 
> backup tests to a separate job which only runs on cinder changes to 
> alleviate these latent bugs failing jobs for unrelated changes and 
> resetting the entire gate.
> 
> I would need someone from the cinder team that is more involved in 
> knowing what their job setup looks like to identify a candidate job for 
> these tests if this is something everyone can agree on doing.
> 
> [1] 
> http://lists.openstack.org/pipermail/openstack-discuss/2018-December/000867.html 
> 
> [2] http://status.openstack.org/elastic-recheck/

This is an old thread but gmann recently skipping a cinder backup test 
which was failing a lot [1] prompted me to revisit this.

As such I've proposed a change [2] which will disable the cinder-backup 
service in the tempest-full job which is in the integrated-gate project 
template and run by most projects.

There is a voting job running against cinder changes named 
"cinder-tempest-dsvm-lvm-lio-barbican" which will still test the backup 
service but it's not gating - it's up to the cinder team if they want to 
make that job gating. The other thing is it doesn't look like that job runs
on glance (or swift) changes so if the cinder team is interested in 
co-gating changes between at least cinder and glance, they could add 
cinder-tempest-dsvm-lvm-lio-barbican to glance so it runs there and/or 
create a new cinder-backup job which just runs backup tests and gate on 
that in both cinder and glance.

[1] https://review.openstack.org/#/c/651660/
[2] https://review.openstack.org/#/c/651865/

-- 

Thanks,

Matt