Open Stack

> Also I’m just wondering how do we keep upstream modules in our repo? They
are not submodules, so how is it organized?

Currently, we don't have any automatic tracking system for changes we apply
to the community/upstream modules, that could help us to re-apply them
during the sync. Only git or diff comparison between original module and
our copy. But that should not be a problem when we finish current sync and
switch to the new contribution workflow described in the doc, Vladimir has
mentioned in the initial email [1].

Also, in the nearest future we're planning to add unit tests (rake spec)
and puppet noop tests into our CI. I think we should combine noop tests
with "regression" testing by using 'rake spec'. But this time I mean RSpec
tests for puppet host, not for specific classes as I suggested in the
previous email. Such tests would compile a complete catalog using our
'site.pp' for specific astute.yaml settings and it will check that needed
puppet resources present in the catalog and have needed attributes. Here's
a draft - [2]. It checks catalog compilation for a controller node and runs
few checks for 'keystone' class and keystone cache driver settings.

Since all the test logic is outside of our puppet modules directory, it
won't be affected by any further upstream syncs or changes we apply in our
modules. So in case some commit removes anything critical that is covered
by "regression"/noop tests, then it will get '-1' from CI and attract our
attention :)

[1]
http://docs.mirantis.com/fuel-dev/develop/module_structure.html#contributing-to-existing-fuel-library-modules
[2] https://review.openstack.org/141022

Regards,
Aleksandr


On Fri, Nov 21, 2014 at 8:07 PM, Tomasz Napierala <tnapierala at mirantis.com>
wrote:

>
> > On 21 Nov 2014, at 17:15, Aleksandr Didenko <adidenko at mirantis.com>
> wrote:
> >
> > Hi,
> >
> > following our docs/workflow plus writing rspec tests for every new
> option we add/modify in our manifests could help with regressions. For
> example:
> >       • we add new keystone config option in openstack::keystone class -
> keystone_config {'cache/backend': value => 'keystone.cache.memcache_pool';}
> >       • we create new test for openstack::keystone class, something like
> this:
> >               • should
> contain_keystone_config("cache/backend").with_value('keystone.cache.memcache_pool')
> > So with such test, if for some reason we lose
> keystone_config("cache/backend") option, 'rake spec' would alert us about
> it right away and we'll get "-1" from CI. Of course we should also
> implement 'rake spec' CI gate for this.
> >
> > But from the other hand, if someone changes option in manifests and
> updates rspec tests accordingly, then such commit will pass 'rake spec'
> test and we can still lose some specific option.
> >
> > > We should speed up development of some modular testing framework that
> will check that corresponding change affects only particular pieces.
> >
> > Such test would not catch this particular regressions with
> "keystone_config {'cache/backend': value =>
> 'keystone.cache.memcache_pool';}", because even with regression (i.e.
> dogpile backend) keystone was working OK. It has passed several BVTs and
> custom system tests, because 'dogpile' cache backend was working just fine
> while all memcached servers are up and running. So it looks like we need
> some kind of tests that will ensure that particular config options (or
> particular puppet resources) have some particular values (like "backend =
> keystone.cache.memcache_pool" in [cache] block of keystone.conf).
> >
> > So I would go with rspec testing for specific resources but I would
> write them in 'openstack' module. Those tests should check that needed
> (nova/cinder/keystone/glance)_config resources have needed values in the
> puppet catalog. Since we're not going to sync 'openstack' module with the
> upstream, such tests will remain intact until we change them, and they
> won't be affected by other modules sync/merge (keystone, cinder, nova, etc).
>
> I totally agree, but we need to remember to introduce tests in separate
> commits, otherwise loosing commit ID we would also lose tests ;)
>
> Also I’m just wondering how do we keep upstream modules in our repo? They
> are not submodules, so how is it organized?
>
> Regards,
> --
> Tomasz 'Zen' Napierala
> Sr. OpenStack Engineer
> tnapierala at mirantis.com
>
>
>
>
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141211/306c6cd0/attachment.html>