[openstack-dev] [TripleO] Delorean fail blocks CI for stable branches

Derek Higgins derekh at redhat.com
Thu Jul 21 10:42:00 UTC 2016


Trying to catch up here and summarizing as a top post

so as I see it we have a number of problems that all need sorting out

Regardless of the branch being built, tripleo-ci is currently building
it with the packaging for master
 - in most cases this has worked out ok because most projects don't
have separate packaging for stable branches but it is now biting us
because t-h-t stable can no longer build with the packaging for master
 - This situation started about a month ago, when a commit[1] moved
the code that sets the branch to use in repositories to only happen if
--local isn't present
   - whether this is the correct behavior or not depends on how you
read the description of --local "Use local git repos if possible.", my
original intention for --local was that delorean wouldn't do a new
fetch of the git repositories being used (i.e. only use whats local if
possible), --dev is different it is what intended to be used if you
want the various git repositories to remain as is without being reset.
If the consensus with everybody is the same behavior for --local then
the old behavior is correct and the delorean patch that changed the
behavior of --local was a regression. But before labeling it as one we
need to discuss people expectations of --local as the description
could be more detailed.

Having said all that, in tripleo-ci iirc we need a combination of
--dev and --local and not having an exact match for what we need in
delorean, we've instead relied on knowledge of delorean internals to
put a hack in place so that what we need to happen, happens. It's not
surprising we've gotten broken.

--local alone wont work for us because it has always (and I think
still should) reset local repositories
--dev alone wont work for us because it doesn't update the delorean
database and as a result in cases where we're building multiple
packages during the same CI job we wont end up with a single repo at
the end containing all the packages

so we used --local and worked around the git reset by presetting all
the branch names that our source repository could be set to to match
what zuul told us to use[2]

this is brittle, difficult to follow and something I did a long time
ago expecting us to improve but then never did anything about

So Here is what I propose,

we now have 2 patches that essentially do the same thing[3][4], merge
either of these, based only on which was proposed first i'd go for
345070

Start a discussion in rdo about how exactly people expect --local and
--dev to behave and if there is need for a 3rd option to cover the
tripleo use case, If we find any changes are needed to delorean, make
them and then remove the hacks from tripleo-ci so we're less likely to
hit problems in future.

thanks,
Derek.

[1] https://review.rdoproject.org/r/#/c/1500/
[2] http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/tripleo.sh#n359
[3] https://review.openstack.org/#/c/345106/1
[4] https://review.openstack.org/#/c/345070/1

On 21 July 2016 at 07:08, Sagi Shnaidman <sshnaidm at redhat.com> wrote:
>
>
> On Thu, Jul 21, 2016 at 3:11 AM, Alan Pevec <apevec at redhat.com> wrote:
>>
>> On Wed, Jul 20, 2016 at 7:49 PM, Sagi Shnaidman <sshnaidm at redhat.com>
>> wrote:
>> > How then it worked before? Can you show me the patch that broke this
>> > functionality in delorean? It should be about 15 Jul when jobs started
>> > to
>> > fail.
>>
>> commented in lp
>>
>> > How then master branch works? It also runs on patched repo and succeeds.
>>
>> I explained that but looks like we're talking past each other.
>>
>> > I don't think we can use this workaround, each time this source file
>> > will
>> > change - all our jobs will fail again? It's not even a workaround.
>> > Please let's stop discussing and let's solve it finally, it blocks our
>> > CI
>> > for stable patches.
>>
>> Sure, I've assigned https://bugs.launchpad.net/tripleo/+bug/1604039 to
>> myself and proposed a patch.
>>
>
> It's a workaround for short time range, but NOT a solution, if you change
> something in this one file, it'll be broken again. But it does NOT solve the
> main issue - after recent changes in dlrn and specs we can't build repo with
> delorean on stable branches.
> I think it should be solved on DLRN side and should be provided a
> appropriate interface to use it for CI purposes.
> I opened an issue there:
> https://github.com/openstack-packages/DLRN/issues/22
> But you closed it, so I suppose we will not get any solution and help for it
> from your side?
>
> Should we move to other packaging tool?
>
>>
>> Alan
>
>
>
>
> --
> Best regards
> Sagi Shnaidman
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



More information about the OpenStack-dev mailing list