Open Stack

> On 21 Nov 2014, at 17:15, Aleksandr Didenko <adidenko at mirantis.com> wrote:
> 
> Hi,
> 
> following our docs/workflow plus writing rspec tests for every new option we add/modify in our manifests could help with regressions. For example:
> 	• we add new keystone config option in openstack::keystone class - keystone_config {'cache/backend': value => 'keystone.cache.memcache_pool';}
> 	• we create new test for openstack::keystone class, something like this:
> 		• should contain_keystone_config("cache/backend").with_value('keystone.cache.memcache_pool')
> So with such test, if for some reason we lose keystone_config("cache/backend") option, 'rake spec' would alert us about it right away and we'll get "-1" from CI. Of course we should also implement 'rake spec' CI gate for this.
> 
> But from the other hand, if someone changes option in manifests and updates rspec tests accordingly, then such commit will pass 'rake spec' test and we can still lose some specific option.
> 
> > We should speed up development of some modular testing framework that will check that corresponding change affects only particular pieces.
> 
> Such test would not catch this particular regressions with "keystone_config {'cache/backend': value => 'keystone.cache.memcache_pool';}", because even with regression (i.e. dogpile backend) keystone was working OK. It has passed several BVTs and custom system tests, because 'dogpile' cache backend was working just fine while all memcached servers are up and running. So it looks like we need some kind of tests that will ensure that particular config options (or particular puppet resources) have some particular values (like "backend = keystone.cache.memcache_pool" in [cache] block of keystone.conf).
> 
> So I would go with rspec testing for specific resources but I would write them in 'openstack' module. Those tests should check that needed (nova/cinder/keystone/glance)_config resources have needed values in the puppet catalog. Since we're not going to sync 'openstack' module with the upstream, such tests will remain intact until we change them, and they won't be affected by other modules sync/merge (keystone, cinder, nova, etc).

I totally agree, but we need to remember to introduce tests in separate commits, otherwise loosing commit ID we would also lose tests ;)

Also I’m just wondering how do we keep upstream modules in our repo? They are not submodules, so how is it organized?

Regards,
-- 
Tomasz 'Zen' Napierala
Sr. OpenStack Engineer
tnapierala at mirantis.com