Open Stack

On 7/24/20 12:51, melanie witt wrote:
> Hey all,
> 
> The nova-ceph-multistore job (devstack-plugin-ceph-tempest-py3 + tweaks 
> to make it run with multiple glance stores) is failing at around a 80% 
> rate as of today. We are tracking the work in this bug:
> 
> https://bugs.launchpad.net/devstack-plugin-ceph/+bug/1888895
> 
> The TL;DR on this is that the ceph bluestore backend when backed by a 
> file will create the file if it doesn't already exist and will create it 
> with a default size. Prior to today, we were pulling ceph version 
> 14.2.10 which defaults the file size to 100G. Then today, we started 
> pulling ceph version 14.2.2 which defaults the file size to 10G which 
> isn't enough space and we're getting NoValidHost with no allocation 
> candidates being returned from placement.
> 
> We don't know yet what caused us to start pulling an older version tag 
> for ceph.
> 
> We are currently trying out a WIP fix in the devstack-plugin-ceph repo 
> to configure the bluestore_block_file_size to a reasonable value instead 

s/bluestore_block_file_size/bluestore_block_size/

> of relying on the default:
> 
> https://review.opendev.org/742961
> 
> We'll keep you updated on the progress as we work on it.

Updating this to [all][gate] because it appears it's not only the 
nova-ceph-multistore job that's affected but all 
devstack-plugin-ceph-tempest-py3 jobs. We found NoValidHost failures on 
patches proposed to openstack/glance and openstack/tempest as well.

The fix for all should be the same (patch in devstack-plugin-ceph) so 
once we get that working well, the ceph jobs should be fixed for 
[all][gate].

-melanie