[openstack-dev] [Fuel] Relieving CI/gate jenkins bottleneck
Bogdan Dobrelya
bdobrelia at mirantis.com
Fri Jan 29 09:13:55 UTC 2016
On 28.01.2016 17:48, Alex Schultz wrote:
Please let's continue discussion here [0].
[0]
http://lists.openstack.org/pipermail/openstack-dev/2016-January/085310.html
> On Thu, Jan 28, 2016 at 9:31 AM, Sergii Golovatiuk
> <sgolovatiuk at mirantis.com> wrote:
>> Hi,
>>
>> I disagree with Bogdan. We have the same case in OpenStack components.
>>
>> 1. As a compony we may have own patches on top of 'master'.
>> 2. There is no way to use tags on upstream modules so it will be very hard
>> to support it. If we need to deliver fix in previous release there won't be
>> simple way to create tag, and cherry-pick the particular commit.
>>
>> So I support Alex to continue the way we have right now.
>>
>
> 2 is not completely true, but it does rely on upstream to provide tags
> (not all do or are responsive). For instance, the puppet openstack
> modules do not provide updated tags for patches to the previous
> released version. Personally I agree that the Puppetfile should point
> to clean upstream versions for the upstream fuel-library. But I think
> we need to work out how to properly separate upstream/downstream
> fuel-library prior to switching out the Puppetfile with one that only
> consumes the complete upstream versions. For now, we have a policy in
> place around where the modules we pull in should be located and that
> it must point to a tag. Once we've solidified what the
> upstream/downstream fuel-library looks like, then we can adjust the
> upstream policy to not be so stringent and let the upstream
> fuel-library rely on pure upstream modules and perhaps drop the tag
> requirement while still allowing downstream to continue with custom
> tags and modules.
>
> -Alex
>
>>
>>
>>
>>
>> --
>> Best regards,
>> Sergii Golovatiuk,
>> Skype #golserge
>> IRC #holser
>>
>> On Thu, Jan 28, 2016 at 5:07 PM, Bogdan Dobrelya <bdobrelia at mirantis.com>
>> wrote:
>>>
>>> On 22.01.2016 13:56, Bogdan Dobrelya wrote:
>>>> On 22.01.2016 12:19, Matthew Mosesohn wrote:
>>>>> +1 for defaulting to upstream for CI. If we have a strong case where we
>>>>> need to make a patch in order to make unit tests pass, we could switch
>>>>> a
>>>>> module back to review.fuel-infra.org/puppet-modules/..
>>>>> <http://review.fuel-infra.org/puppet-modules/..>.. with a FIXME and a
>>>>> LP
>>>>> bug ID so we can switch it back quickly once the upstream issue is
>>>>> resolved. As far as I know, we don't have to worry about that
>>>>> scenario.
>>>>
>>>> Yes, exactly so. Switching upstream/downstream links of modules in the
>>>> Puppetfile back and forth can be done as often as we need it, with no
>>>> issues at all. With "free bonuses" though! Which is, firstly, it would
>>>> be easier to estimate upstream-to-downstream sync status by just looking
>>>> at the Puppetfile. Secondly, each time one's switching an upstream link
>>>> to a downstream, reviewers may treat this as a "tech dept growing
>>>> alarm!" and perhaps motivate patch authors to contribute changes
>>>> upstream instead (or *faster*) rather than just stashing them
>>>> accumulated downstream.
>>>
>>> We have a disagreement for the patches [0], [1] related to this topic.
>>> My point is that we should use direct upstream hashtags/releases as
>>> early as possible. Nothing prevents us from switching to a downstream
>>> one in case of emergency.
>>>
>>> So donwstream maintaining efforts shall not be done from the very
>>> beginning, if possible to save infra/engineering resource for something
>>> more useful.
>>>
>>> [0] https://review.openstack.org/#/c/271217
>>> [1] https://review.openstack.org/#/c/273036/
>>>
>>>>
>>>>>
>>>>> -Matthew
>>>>>
>>>>> On Fri, Jan 22, 2016 at 12:51 PM, Bogdan Dobrelya
>>>>> <bdobrelia at mirantis.com <mailto:bdobrelia at mirantis.com>> wrote:
>>>>>
>>>>> Another point is to use upstream links for modules w/o downstream
>>>>> patches.
>>>>> I noticed we *always* put it to the deployment/Puppetfile [0] as
>>>>> "https://review.fuel-infra.org/puppet-modules/...". Why should we?
>>>>> Let's just do the best to reuse upstream modules as is, eventually.
>>>>>
>>>>> [0]
>>>>> https://github.com/openstack/fuel-library/blob/master/deployment/Puppetfile
>>>>>
>>>>> Regards,
>>>>> Bogdan Dobrelya.
>>>>> Irc #bogdando
>>>>>
>>>>> 2016-01-21 11:09 GMT+01:00 Bartlomiej Piotrowski
>>>>> <bpiotrowski at mirantis.com <mailto:bpiotrowski at mirantis.com>>:
>>>>>
>>>>> Let's drop 3.3 as well. 3.4 is oldschool enough for vintage
>>>>> lovers.
>>>>>
>>>>> BP
>>>>>
>>>>> On Thu, Jan 21, 2016 at 11:03 AM, Aleksandr Didenko
>>>>> <adidenko at mirantis.com <mailto:adidenko at mirantis.com>> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> > I also think 3.3 is the version that ships with 14.04.
>>>>>
>>>>> 3.4.3 is shipped with Ubuntu-14.04. I think 3.4, 3.8 and 4
>>>>> should be enough.
>>>>>
>>>>> Regards,
>>>>> Alex
>>>>>
>>>>> On Wed, Jan 20, 2016 at 6:38 PM, Sergii Golovatiuk
>>>>> <sgolovatiuk at mirantis.com
>>>>> <mailto:sgolovatiuk at mirantis.com>>
>>>>> wrote:
>>>>>
>>>>> +1 for 3.3, 3.4, 3.8 and 4
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Sergii Golovatiuk,
>>>>> Skype #golserge
>>>>> IRC #holser
>>>>>
>>>>> On Wed, Jan 20, 2016 at 6:12 PM, Alex Schultz
>>>>> <aschultz at mirantis.com <mailto:aschultz at mirantis.com>>
>>>>> wrote:
>>>>>
>>>>> On Wed, Jan 20, 2016 at 9:02 AM, Matthew Mosesohn
>>>>> <mmosesohn at mirantis.com
>>>>> <mailto:mmosesohn at mirantis.com>> wrote:
>>>>> > Hi all,
>>>>> >
>>>>> > Unit tests on CI and gate bottleneck are really
>>>>> slowing down commit
>>>>> > progress. We recently had a meeting to discuss
>>>>> possible ways to improve
>>>>> > this, including symlinks, caching git
>>>>> repositories, etc, but one thing we
>>>>> > can do much faster is to simply disable 3.3-3.7
>>>>> puppet jobs. We don't deploy
>>>>> > Fuel 9.0 (or 8.0) on earlier Puppet versions, so
>>>>> what value is there to the
>>>>> > checks? I propose we remove these tests, and
>>>>> hopefully we will see some
>>>>> > immediate relief.
>>>>> >
>>>>>
>>>>> How about we reduce to 3.3, 3.4, 3.8 and 4? We
>>>>> would remove 3.6 and
>>>>> 3.7 which would reduce the number of jobs by a
>>>>> third The goal of
>>>>> keeping the others was to ensure that if/when we
>>>>> are
>>>>> able to install
>>>>> fuel-library without our version of puppet that a
>>>>> user could use
>>>>> whatever version their environment has. There were
>>>>> some changes
>>>>> between 3.3 and 3.4 (if I remember correctly) so we
>>>>> should keep
>>>>> checking that as it's also the oldest version
>>>>> supported by the
>>>>> upstream puppet openstack modules. I also think
>>>>> 3.3
>>>>> is the version
>>>>> that ships with 14.04. Additionally we used 3.4 in
>>>>> fuel 7 and below
>>>>> so we should keep those around.
>>>>>
>>>>> -Alex
>>>>>
>>>>> > Best Regards,
>>>>> > Matthew Mosesohn
>>>>> >
>>>>> >
>>>>>
>>>>> __________________________________________________________________________
>>>>> > OpenStack Development Mailing List (not for usage
>>>>> questions)
>>>>> > Unsubscribe:
>>>>>
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>
>>>>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>>>> >
>>>>>
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>> >
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage
>>>>> questions)
>>>>> Unsubscribe:
>>>>>
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>
>>>>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>>>>
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage
>>>>> questions)
>>>>> Unsubscribe:
>>>>>
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>
>>>>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>>>>
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage
>>>>> questions)
>>>>> Unsubscribe:
>>>>>
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>>>>
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>
>>>>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>>>>
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>>
>>>>> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe:
>>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Bogdan Dobrelya,
>>> Irc #bogdando
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
--
Best regards,
Bogdan Dobrelya,
Irc #bogdando
More information about the OpenStack-dev
mailing list