Open Stack

Folks

I am writing this email to discuss very important topic which appeared to
be rather hot with several bugs associated with it.

For example these ones:
https://bugs.launchpad.net/fuel/+bug/1472018
https://bugs.launchpad.net/fuel/+bug/1468053

There were several fixes applied. One of them was merged (unfortunately)
and one was stopped by me and other reviewers of stable branches
(fortunately),

I want to  describe why these bugs happened, why such fixes should not be
applied and which solution I envision the best for such cases.

The key issue here is that our CI tests are using some code not from
packages of particular releases, but from other sources, e.g. pypi for
python.
Libraries being installed are determined by using particular
requirements.txt/test-requirements.txt files. These files usually do not
contain upper bound for libraries versions, thus the newest versions are
getting installed. This leads to the issues which are especially no good
for already released code:

1) if there is a new conflicting version, you need to set this upper-bound,
thus you need to modify bits which get released
2) you are actually testing your code by linking it with libraries which
are different from those that users are really using when running your code
3) even if you specify an upper bound (or even fix the version) for this
particular library, you may still fetch its newer dependency implicitly (by
traversing indirect dependencies) with which you will be linking your
libraries and which will actually be different from the code that you (and
your users) use
4) you may also break production installation if you fix some library
version as it may not exist in the code bundle which gets delivered to your
users as a set of packages

All this means that you will get false-negative (you are lucky in this
case) -1's from CI or false-positive (oops, it can ruin the production
installation) +1's from CI.

So far I suggest that we do the following (which, strictly speaking, seems
to be the only reasonable one): in order to do good testing of components,
we need to do complete freeze of all other libraries and components code we
are linking/running against. Especially, this should happen for already
released code which gets into state of being maintained after that.

This means that we need to not only freeze our repositories, but use a
frozen copy of rubygems/pypi/nodejs repositories for all types of tests we
run. This will require certain amount of effort from our Infra teams to
build up a mechanism of freezing/unfreezing the mirrors, determining how we
should add new pieces of code in such mirrors, may be, add some additional
code-review policies, create real mirroring scripts and so on.

But I think we should start doing this now in order to finish it at least
before 8.0 timeframe.

As a temporary solution, I suggest that we apply a series of changes into
these requirements.txt files before running actual tests instead of
modifying product code.

Please provide your comments and thoughts.


-- 
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
35bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com <http://www.mirantis.ru/>
www.mirantis.ru
vkuklin at mirantis.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150710/a5484327/attachment.html>