<div dir="ltr">I guess it would depend on how many docker containers are running on master node and if we are able to pull off such stunt :).<div><br></div><div>I am not familiar with the amount of work needed to do sth like that, so the proposition may be silly. Just let me know if it is.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 19, 2015 at 11:51 AM, Dmitry Burmistrov <span dir="ltr"><<a href="mailto:dburmistrov@mirantis.com" target="_blank">dburmistrov@mirantis.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Folks,<br>
<span class=""><br>
>>> Correct me if I am wrong, but isn't it what we have containers on master node for?<br>
</span><span class="">>>> On the master node itself conflicts won’t happen because the components are run in their containers.<br>
<br>
</span>Do you propose to use _separate_ package repository for each docker<br>
container? (It means separate gerrit project for each package of each<br>
container, including openstack projects)<br>
<div class="HOEnZb"><div class="h5"><br>
On Thu, Mar 19, 2015 at 1:16 PM, Roman Prykhodchenko <<a href="mailto:me@romcheg.me">me@romcheg.me</a>> wrote:<br>
> Folks,<br>
><br>
>> I assume you meant: "If a requirement that previously was only in Fuel<br>
>> Requirements is merged to Global Requirements, it should be removed<br>
>> from *Fuel* Requirements».<br>
><br>
> Exactly.<br>
><br>
>> I'm not sure it's good idea.<br>
>> We should stay so close to upstream distro as we can. It should be<br>
>> very important reason to update package against it's upstream distro<br>
><br>
><br>
> The issue is the following: OpenStack’s components are tested against those versions of dependencies, that are specified in their requirements. IIRC those requirements are set up by pip so CI nodes contain latest versions of python packages that match their specs.<br>
><br>
> The question is whether it’s required to ship OpenStack services with packages from a distro or with packages, that are used for testing.<br>
><br>
>> Splitting of repositories doesn't help to solve python packages<br>
>> conflicts because master node uses a number of openstack components.<br>
><br>
> On the master node itself conflicts won’t happen because the components are run in their containers.<br>
><br>
><br>
> - romcheg<br>
><br>
><br>
>> 19 бер. 2015 о 10:47 Dmitry Burmistrov <<a href="mailto:dburmistrov@mirantis.com">dburmistrov@mirantis.com</a>> написав(ла):<br>
>><br>
>> Roman, all<br>
>><br>
>>>>> - OSCI mirror should contain the maximum version of a requirement that matches its version specification.<br>
>> I'm not sure it's good idea.<br>
>> We should stay so close to upstream distro as we can. It should be<br>
>> very important reason to update package against it's upstream distro<br>
>> version.<br>
>> If we update some package, we should maintain it too. Tracking bugs,<br>
>> CVEs and so on. The more packages, the more efforts to support it.<br>
>><br>
>>>>> - Set up CI jobs to notify OSCI team if either Global Requirements or<br>
>>>>> Fuel Requirements are changed.<br>
>>>>> - Set up requirements proposal jobs that will automatically propose<br>
>>>>> changes to all fuel projects once either of requirements lists was changed<br>
>> Need to be discussed and investigated.<br>
>><br>
>> Sebastian, Dmitry,<br>
>><br>
>><br>
>>>>> There are some plans (unfortunately I do not know details, so maybe someone from OSCI could tell more) to split those repositories<br>
>> Splitting of repositories doesn't help to solve python packages<br>
>> conflicts because master node uses a number of openstack components.<br>
>> Anyway it should be done for 7.0 milestone in order to separatre<br>
>> master-node distro from environment one. (e.g. if we going to move<br>
>> master-node to debian)<br>
>><br>
>><br>
>><br>
>> On Thu, Mar 19, 2015 at 12:01 AM, Dmitry Borodaenko<br>
>> <<a href="mailto:dborodaenko@mirantis.com">dborodaenko@mirantis.com</a>> wrote:<br>
>>> Roman,<br>
>>><br>
>>> I like this proposal very much, thanks for the idea and for putting<br>
>>> together a straightforward process.<br>
>>><br>
>>> I assume you meant: "If a requirement that previously was only in Fuel<br>
>>> Requirements is merged to Global Requirements, it should be removed<br>
>>> from *Fuel* Requirements".<br>
>>><br>
>>> Sebastian,<br>
>>><br>
>>> We have found ways to resolve the conflicts between clvm and docker,<br>
>>> and between ruby versions 1.8 and 2.1, without introducing a separate<br>
>>> package repo for Fuel master. I've updated the blueprint to make note<br>
>>> of that:<br>
>>> <a href="https://blueprints.launchpad.net/fuel/+spec/separate-repo-for-master-node" target="_blank">https://blueprints.launchpad.net/fuel/+spec/separate-repo-for-master-node</a><br>
>>><br>
>>><br>
>>><br>
>>><br>
>>> On Wed, Mar 18, 2015 at 9:25 AM, Sebastian Kalinowski<br>
>>> <<a href="mailto:skalinowski@mirantis.com">skalinowski@mirantis.com</a>> wrote:<br>
>>>> I assume that you considered a situation when we have a common repository<br>
>>>> with RPMs for Fuel master and for nodes.<br>
>>>> There are some plans (unfortunately I do not know details, so maybe someone<br>
>>>> from OSCI could tell more) to split those repositories. How this workflow<br>
>>>> will work with those separated repos? Will we still need it?<br>
>>>><br>
>>>> Thanks!<br>
>>>> Sebastian<br>
>>>><br>
>>>> 2015-03-18 11:04 GMT+01:00 Roman Prykhodchenko <<a href="mailto:me@romcheg.me">me@romcheg.me</a>>:<br>
>>>>><br>
>>>>> Hi folks,<br>
>>>>><br>
>>>>> before you say «romcheg, go away and never come back again!», please read<br>
>>>>> the story that caused me to propose this and the proposed solution. Perhaps<br>
>>>>> it makes you reconsider :)<br>
>>>>><br>
>>>>> As you know for different reasons, among which are being able to set up<br>
>>>>> everything online and bringing up-to-date packages, we maintain an OSCI<br>
>>>>> repository which is used for building ISOs and deploying OpenStack services.<br>
>>>>> Managing that repo is a pretty hard job. Thus a dedicated group of people is<br>
>>>>> devoted to perform that duty, they are always busy because of a lot of<br>
>>>>> responsibilities they have.<br>
>>>>><br>
>>>>> At the same time Fuel’s developers are pretty energetic and always want to<br>
>>>>> add new features to Fuel. For that they love to use different libraries,<br>
>>>>> many of which aren’t in the OSCI mirror yet. So they ask OSCI guys to add<br>
>>>>> more and more of those and I guess that’s pretty fine except one little<br>
>>>>> thing — sometimes those libraries conflict with ones, required by OpenStack<br>
>>>>> services.<br>
>>>>><br>
>>>>> To prevent that from happening someone has to check every patch against<br>
>>>>> the OSCI repo and OpenStack’s global requirements, to detect whether a<br>
>>>>> version bump or adding a new library is required an whether it can be<br>
>>>>> performed. As you can guess, there’s too much of a human factor so<br>
>>>>> statistically no one does that until problems appear. Moreover, theres’<br>
>>>>> nothing but a «it’s not compatible with OpenStack» yelling from OSCI team<br>
>>>>> that stops developers to change dependencies in Fuel.<br>
>>>>><br>
>>>>> All the stuff described above causes sometimes tremendous time losses and<br>
>>>>> is very problem-prone.<br>
>>>>><br>
>>>>> I’d like to propose to make everyone’s life easier by following these<br>
>>>>> steps:<br>
>>>>><br>
>>>>> - Create a new project called Fuel Requirements, all changes to it should<br>
>>>>> go through a standard review procedure<br>
>>>>> - We strict ourselves to use only packages from both Fuel Requirements<br>
>>>>> and Global Requirements for the version of OpenStack, Fuel is installing in<br>
>>>>> the following manner:<br>
>>>>> - If a requirement is in Global Requirements, the version spec in all<br>
>>>>> Fuel’s components should be exactly like that.<br>
>>>>> - OSCI mirror should contain the maximum version of a requirement<br>
>>>>> that matches its version specification.<br>
>>>>> - If a requirement is not in the global requirements list, then Fuel<br>
>>>>> Requirements list should be used to check whether all Fuel’s components<br>
>>>>> require the same version of a library/package.<br>
>>>>> - OSCI mirror should contain the maximum version of a requirement<br>
>>>>> that matches its version specification.<br>
>>>>> - If a requirement that previously was only in Fuel Requirements is<br>
>>>>> merged to Global Requirements, it should be removed from Global Requirements<br>
>>>>> - Set up CI jobs in both OpenStack CI and FuelCI to check all patches<br>
>>>>> against both Global Requirements and Fuel Requirements and block, if either<br>
>>>>> of checks doesn’t pass<br>
>>>>> - Set up CI jobs to notify OSCI team if either Global Requirements or<br>
>>>>> Fuel Requirements are changed.<br>
>>>>> - Set up requirements proposal jobs that will automatically propose<br>
>>>>> changes to all fuel projects once either of requirements lists was changed,<br>
>>>>> just like it’s done for OpenStack projects.<br>
>>>>><br>
>>>>><br>
>>>>> These steps may look terribly hard, but most of the job is already done in<br>
>>>>> OpenStack projects and therefore it can be reused for Fuel.<br>
>>>>> Looking forward for your feedback folks!<br>
>>>>><br>
>>>>><br>
>>>>> - romcheg<br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> __________________________________________________________________________<br>
>>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>>> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>>><br>
>>>><br>
>>>><br>
>>>> __________________________________________________________________________<br>
>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>><br>
>>><br>
>>><br>
>>><br>
>>> --<br>
>>> Dmitry Borodaenko<br>
>>><br>
>>> __________________________________________________________________________<br>
>>> OpenStack Development Mailing List (not for usage questions)<br>
>>> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>><br>
>> __________________________________________________________________________<br>
>> OpenStack Development Mailing List (not for usage questions)<br>
>> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
><br>
> __________________________________________________________________________<br>
> OpenStack Development Mailing List (not for usage questions)<br>
> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div>