[cinder][nova] Running parallel iSCSI/LVM c-vol backends is causing random failures in CI

Luigi Toscano ltoscano at redhat.com
Tue Mar 9 23:07:52 UTC 2021


On Wednesday, 10 March 2021 00:00:29 CET Sean Mooney wrote:
> On Tue, 2021-03-09 at 23:50 +0100, Luigi Toscano wrote:
> > On Tuesday, 9 March 2021 23:40:08 CET Lee Yarwood wrote:
> > > On Tue, 9 Mar 2021 at 22:24, Jeremy Stanley <fungi at yuggoth.org> wrote:
> > > > On 2021-03-09 22:19:48 +0000 (+0000), Jeremy Stanley wrote:
> > > > > On 2021-03-09 22:18:12 +0000 (+0000), Jeremy Stanley wrote:
> > > > > > On 2021-03-09 22:08:18 +0000 (+0000), Lee Yarwood wrote:
> > > > > > > I reported the following bug last week but I've yet to get any
> > > > > > > real
> > > > > > > feedback after asking a few times in irc.
> > > > > > > 
> > > > > > > Running parallel iSCSI/LVM c-vol backends is causing random
> > > > > > > failures
> > > > > > > in CI
> > > > > > > https://bugs.launchpad.net/cinder/+bug/1917750
> > > > > > > 
> > > > > > > AFAICT tgtadm is causing this behaviour. As I've stated in the
> > > > > > > bug
> > > > > > > with Fedora 32 and lioadm I don't see the WWN conflict between
> > > > > > > the
> > > > > > > two
> > > > > > > backends. Does anyone know if using lioadm is an option on
> > > > > > > Focal?
> > > > > > 
> > > > > > https://docs.openstack.org/cinder/latest/admin/blockstorage-lio-is
> > > > > > csi-> > > > support.html seems to indicate that you just need to
> > > > > > set it in configuration. The package that document mentions looks
> > > > > > like a distro package
> > > > > > recommendation (does not exist under that name on PyPI) and the
> > > > > > equivalent lib on PyPI is included in cinder's requirements.txt
> > > > > > file, but I don't see either mentioned in the devstack source tree
> > > > > > so maybe that needs to be installed for DevStack-based jobs to
> > > > > > take
> > > > > > advantage of it.
> > > > > 
> > > > > Oh, and this would be the package to add from Ubuntu Focal:
> > > > > 
> > > > > https://packages.ubuntu.com/focal/python3-rtslib-fb
> > > > 
> > > > Nevermind, DevStack installs the projects with pip, so the one in
> > > > cinder's requirements.txt should already be present. In that case,
> > > > yeah, just set it in the config?
> > > 
> > > Yes correct, my question was more to see if there were any known
> > > issues with using lioadm on Focal. Anyway I've pushed the following
> > > WIP change for devstack to switch over to lioadm when using Focal:
> > > 
> > > WIP cinder: Default CINDER_ISCSI_HELPER to lioadm on Ubuntu
> > > https://review.opendev.org/c/openstack/devstack/+/779624
> > 
> > For the record, we use the default (tgt) on the main cinder gates, as well
> > as a lioadm job (defined in cinder-tempest-plugin). If I read the code
> > correctly, that change would break tgt for everyone on ubuntu.
> > 
> > Please raise this in the cinder meeting (Wednesday).
> 
> i dont think we can wait that long im pretty sure this is causing this error
> form talking to lee earlier to day
> http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:
> %5C%22Unable%20to%20detach%20the%20device%20from%20the%20live%20config%5C%22
> %20AND%20loglevel:%20ERROR
> 
> i guess it almost wednesday already but its startin to casue issues in
> multiple poject gates right before code freeze so we need to adress this
> before we end up rechecking things over and over on many patches.

But then don't error out if someone tries to use tgtadm, which is what would 
happen if that patch was merged (if I didn't misread the code).

-- 
Luigi 





More information about the openstack-discuss mailing list