[openstack-dev] [Fuel] Relieving CI/gate jenkins bottleneck
Alex Schultz
aschultz at mirantis.com
Thu Jan 28 16:48:09 UTC 2016
On Thu, Jan 28, 2016 at 9:31 AM, Sergii Golovatiuk
<sgolovatiuk at mirantis.com> wrote:
> Hi,
>
> I disagree with Bogdan. We have the same case in OpenStack components.
>
> 1. As a compony we may have own patches on top of 'master'.
> 2. There is no way to use tags on upstream modules so it will be very hard
> to support it. If we need to deliver fix in previous release there won't be
> simple way to create tag, and cherry-pick the particular commit.
>
> So I support Alex to continue the way we have right now.
>
2 is not completely true, but it does rely on upstream to provide tags
(not all do or are responsive). For instance, the puppet openstack
modules do not provide updated tags for patches to the previous
released version. Personally I agree that the Puppetfile should point
to clean upstream versions for the upstream fuel-library. But I think
we need to work out how to properly separate upstream/downstream
fuel-library prior to switching out the Puppetfile with one that only
consumes the complete upstream versions. For now, we have a policy in
place around where the modules we pull in should be located and that
it must point to a tag. Once we've solidified what the
upstream/downstream fuel-library looks like, then we can adjust the
upstream policy to not be so stringent and let the upstream
fuel-library rely on pure upstream modules and perhaps drop the tag
requirement while still allowing downstream to continue with custom
tags and modules.
-Alex
>
>
>
>
> --
> Best regards,
> Sergii Golovatiuk,
> Skype #golserge
> IRC #holser
>
> On Thu, Jan 28, 2016 at 5:07 PM, Bogdan Dobrelya <bdobrelia at mirantis.com>
> wrote:
>>
>> On 22.01.2016 13:56, Bogdan Dobrelya wrote:
>> > On 22.01.2016 12:19, Matthew Mosesohn wrote:
>> >> +1 for defaulting to upstream for CI. If we have a strong case where we
>> >> need to make a patch in order to make unit tests pass, we could switch
>> >> a
>> >> module back to review.fuel-infra.org/puppet-modules/..
>> >> <http://review.fuel-infra.org/puppet-modules/..>.. with a FIXME and a
>> >> LP
>> >> bug ID so we can switch it back quickly once the upstream issue is
>> >> resolved. As far as I know, we don't have to worry about that
>> >> scenario.
>> >
>> > Yes, exactly so. Switching upstream/downstream links of modules in the
>> > Puppetfile back and forth can be done as often as we need it, with no
>> > issues at all. With "free bonuses" though! Which is, firstly, it would
>> > be easier to estimate upstream-to-downstream sync status by just looking
>> > at the Puppetfile. Secondly, each time one's switching an upstream link
>> > to a downstream, reviewers may treat this as a "tech dept growing
>> > alarm!" and perhaps motivate patch authors to contribute changes
>> > upstream instead (or *faster*) rather than just stashing them
>> > accumulated downstream.
>>
>> We have a disagreement for the patches [0], [1] related to this topic.
>> My point is that we should use direct upstream hashtags/releases as
>> early as possible. Nothing prevents us from switching to a downstream
>> one in case of emergency.
>>
>> So donwstream maintaining efforts shall not be done from the very
>> beginning, if possible to save infra/engineering resource for something
>> more useful.
>>
>> [0] https://review.openstack.org/#/c/271217
>> [1] https://review.openstack.org/#/c/273036/
>>
>> >
>> >>
>> >> -Matthew
>> >>
>> >> On Fri, Jan 22, 2016 at 12:51 PM, Bogdan Dobrelya
>> >> <bdobrelia at mirantis.com <mailto:bdobrelia at mirantis.com>> wrote:
>> >>
>> >> Another point is to use upstream links for modules w/o downstream
>> >> patches.
>> >> I noticed we *always* put it to the deployment/Puppetfile [0] as
>> >> "https://review.fuel-infra.org/puppet-modules/...". Why should we?
>> >> Let's just do the best to reuse upstream modules as is, eventually.
>> >>
>> >> [0]
>> >> https://github.com/openstack/fuel-library/blob/master/deployment/Puppetfile
>> >>
>> >> Regards,
>> >> Bogdan Dobrelya.
>> >> Irc #bogdando
>> >>
>> >> 2016-01-21 11:09 GMT+01:00 Bartlomiej Piotrowski
>> >> <bpiotrowski at mirantis.com <mailto:bpiotrowski at mirantis.com>>:
>> >>
>> >> Let's drop 3.3 as well. 3.4 is oldschool enough for vintage
>> >> lovers.
>> >>
>> >> BP
>> >>
>> >> On Thu, Jan 21, 2016 at 11:03 AM, Aleksandr Didenko
>> >> <adidenko at mirantis.com <mailto:adidenko at mirantis.com>> wrote:
>> >>
>> >> Hi,
>> >>
>> >> > I also think 3.3 is the version that ships with 14.04.
>> >>
>> >> 3.4.3 is shipped with Ubuntu-14.04. I think 3.4, 3.8 and 4
>> >> should be enough.
>> >>
>> >> Regards,
>> >> Alex
>> >>
>> >> On Wed, Jan 20, 2016 at 6:38 PM, Sergii Golovatiuk
>> >> <sgolovatiuk at mirantis.com
>> >> <mailto:sgolovatiuk at mirantis.com>>
>> >> wrote:
>> >>
>> >> +1 for 3.3, 3.4, 3.8 and 4
>> >>
>> >>
>> >> --
>> >> Best regards,
>> >> Sergii Golovatiuk,
>> >> Skype #golserge
>> >> IRC #holser
>> >>
>> >> On Wed, Jan 20, 2016 at 6:12 PM, Alex Schultz
>> >> <aschultz at mirantis.com <mailto:aschultz at mirantis.com>>
>> >> wrote:
>> >>
>> >> On Wed, Jan 20, 2016 at 9:02 AM, Matthew Mosesohn
>> >> <mmosesohn at mirantis.com
>> >> <mailto:mmosesohn at mirantis.com>> wrote:
>> >> > Hi all,
>> >> >
>> >> > Unit tests on CI and gate bottleneck are really
>> >> slowing down commit
>> >> > progress. We recently had a meeting to discuss
>> >> possible ways to improve
>> >> > this, including symlinks, caching git
>> >> repositories, etc, but one thing we
>> >> > can do much faster is to simply disable 3.3-3.7
>> >> puppet jobs. We don't deploy
>> >> > Fuel 9.0 (or 8.0) on earlier Puppet versions, so
>> >> what value is there to the
>> >> > checks? I propose we remove these tests, and
>> >> hopefully we will see some
>> >> > immediate relief.
>> >> >
>> >>
>> >> How about we reduce to 3.3, 3.4, 3.8 and 4? We
>> >> would remove 3.6 and
>> >> 3.7 which would reduce the number of jobs by a
>> >> third The goal of
>> >> keeping the others was to ensure that if/when we
>> >> are
>> >> able to install
>> >> fuel-library without our version of puppet that a
>> >> user could use
>> >> whatever version their environment has. There were
>> >> some changes
>> >> between 3.3 and 3.4 (if I remember correctly) so we
>> >> should keep
>> >> checking that as it's also the oldest version
>> >> supported by the
>> >> upstream puppet openstack modules. I also think
>> >> 3.3
>> >> is the version
>> >> that ships with 14.04. Additionally we used 3.4 in
>> >> fuel 7 and below
>> >> so we should keep those around.
>> >>
>> >> -Alex
>> >>
>> >> > Best Regards,
>> >> > Matthew Mosesohn
>> >> >
>> >> >
>> >>
>> >> __________________________________________________________________________
>> >> > OpenStack Development Mailing List (not for usage
>> >> questions)
>> >> > Unsubscribe:
>> >>
>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >>
>> >> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>> >> >
>> >>
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >> >
>> >>
>> >>
>> >> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage
>> >> questions)
>> >> Unsubscribe:
>> >>
>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >>
>> >> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>> >>
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >>
>> >>
>> >>
>> >> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage
>> >> questions)
>> >> Unsubscribe:
>> >>
>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >>
>> >> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>> >>
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >>
>> >>
>> >>
>> >> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage
>> >> questions)
>> >> Unsubscribe:
>> >>
>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>> >>
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >>
>> >>
>> >>
>> >> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >>
>> >> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>> >>
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >>
>> >>
>> >>
>> >> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >>
>> >> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >>
>> >
>> >
>>
>>
>> --
>> Best regards,
>> Bogdan Dobrelya,
>> Irc #bogdando
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
More information about the OpenStack-dev
mailing list