Open Stack

On 30.11.2015 13:03, Bogdan Dobrelya wrote:
> On 20.11.2015 17:41, Bogdan Dobrelya wrote:
>>> Hi,
>>>
>>> let me try to rephrase this a bit and Bogdan will correct me if I'm wrong
>>> or missing something.
>>>
>>> We have a set of top-scope manifests (called Fuel puppet tasks) that we use
>>> for OpenStack deployment. We execute those tasks with "puppet apply". Each
>>> task supposed to bring target system into some desired state, so puppet
>>> compiles a catalog and applies it. So basically, puppet catalog = desired
>>> system state.
>>>
>>> So we can compile* catalogs for all top-scope manifests in master branch
>>> and store those compiled* catalogs in fuel-library repo. Then for each
>>> proposed patch CI will compare new catalogs with stored ones and print out
>>> the difference if any. This will pretty much show what is going to be
>>> changed in system configuration by proposed patch.
>>>
>>> We were discussing such checks before several times, iirc, but we did not
>>> have right tools to implement such thing before. Well, now we do :) I think
>>> it could be quite useful even in non-voting mode.
>>>
>>> * By saying compiled catalogs I don't mean actual/real puppet catalogs, I
>>> mean sorted lists of all classes/resources with all parameters that we find
>>> during puppet-rspec tests in our noop test framework, something like
>>> standard puppet-rspec coverage. See example [0] for networks.pp task [1].
>>>
>>> Regards,
>>> Alex
>>>
>>> [0] http://paste.openstack.org/show/477839/
>>> [1] 
>>> https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular/openstack-network/networks.pp
>>
>> Thank you, Alex.
>> Yes, the composition layer is a top-scope manifests, known as a Fuel
>> library modular tasks [0].
>>
>> The "deployment data checks", is nothing more than comparing the
>> committed vs changed states of fixtures [1] of puppet catalogs for known
>> deployment paths under test with rspecs written for each modular task [2].
>>
>> And the *current status* is:
>> - the script for data layer checks now implemented [3]
>> - how-to is being documented here [4]
>> - a fix to make catalogs compilation idempotent submitted [5]
> 
> The status update:
> - the issue [0] is the data regression checks blocker and is only the
> Noop tests specific. It has been reworked to not use custom facts [1].
> New uuid will be still generated each time in the catalog, but the
> augeas ensures it will be processed in idempotent way. Let's make this
> change [2] to the upstream puppet-nova as well please.
> 
> [0] https://bugs.launchpad.net/fuel/+bug/1517915
> [1] https://review.openstack.org/251314
> [2] https://review.openstack.org/131710
> 
> - pregenerated catalogs for the Noop tests to become the very first
> committed state in the data regression process has to be put in the
> *separate repo*. Otherwise, the stackalytics would go mad as that would
> be a 600k-liner patch to an OpenStack project, which is the Fuel-library
> now :)
> 
> So, I'm planning to use the separate repo for the templates. Note, we
> could as well move the tests/noop/astute.yaml/ there. Thoughts?
> 
>> - and there is my WIP branch [6] with the initial committed state of
>> deploy data pre-generated. So, you can checkout, make any test changes
>> to manifests and run the data check (see the README [4]). It works for
>> me, there is no issues with idempotent re-checks of a clean committed
>> state or tests failing when unexpected.
>>
>> So the plan is to implement this noop tests extention as a non-voting CI
>> gate after I make an example workflow update for developers to the
>> Fuel wiki. Thoughts?

Folks, here is another example patch [0] with 4k lines of pure fixtures.
That is why we should not keep astute.yaml fixtures (as well as
pregenerated catalogs for the data regression checks being discussed in
this topic) in the main fuel-library repo.

Instead, all of the fixtures and such must be pulled in from an external
repo, like openstack/fuel-noop-fixtures (which is out of the BigTent
projects listed perhaps) at the rake spec_prep stage of the noop tests.

[0] https://review.openstack.org/246358

>>
>> [0]
>> https://github.com/openstack/fuel-library/blob/master/deployment/puppet/osnailyfacter/modular
>> [1]
>> https://github.com/openstack/fuel-library/tree/master/tests/noop/astute.yaml
>> [2] https://github.com/openstack/fuel-library/tree/master/tests/noop/spec
>> [3] https://review.openstack.org/240015
>> [4]
>> https://github.com/openstack/fuel-library/blob/master/tests/noop/README.rst
>> [5] https://review.openstack.org/247989
>> [6] https://github.com/bogdando/fuel-library-1/commits/data_checks
>>
>>
> 
> 


-- 
Best regards,
Bogdan Dobrelya,
Irc #bogdando