Open Stack

Hey all,

The nova-ceph-multistore job (devstack-plugin-ceph-tempest-py3 + tweaks 
to make it run with multiple glance stores) is failing at around a 80% 
rate as of today. We are tracking the work in this bug:

https://bugs.launchpad.net/devstack-plugin-ceph/+bug/1888895

The TL;DR on this is that the ceph bluestore backend when backed by a 
file will create the file if it doesn't already exist and will create it 
with a default size. Prior to today, we were pulling ceph version 
14.2.10 which defaults the file size to 100G. Then today, we started 
pulling ceph version 14.2.2 which defaults the file size to 10G which 
isn't enough space and we're getting NoValidHost with no allocation 
candidates being returned from placement.

We don't know yet what caused us to start pulling an older version tag 
for ceph.

We are currently trying out a WIP fix in the devstack-plugin-ceph repo 
to configure the bluestore_block_file_size to a reasonable value instead 
of relying on the default:

https://review.opendev.org/742961

We'll keep you updated on the progress as we work on it.

Cheers,
-melanie