Open Stack

On 08/11/2014 09:36 AM, Tom Fifield wrote:
> On 11/08/14 13:20, Andreas Jaeger wrote:
>> On 08/11/2014 05:26 AM, Tom Fifield wrote:
>>> On 10/08/14 22:21, Andreas Jaeger wrote:
>>>> With the addition of a non-voting check for linked URLs, we need to
>>>> touch Jenkins jobs and our tox.ini everywhere. I suggest to do this
>>>> differently: Combine the tiny voting jobs in one job and do the same
>>>> for
>>>> non-voting jobs.
>>>>
>>>> So, instead of:
>>>> * check-syntax
>>>> * check-niceness
>>>> * check-deletions
>>>> * check-links
>>>>
>>>> we would have:
>>>> * check-voting
>>>> * check-non-voting
>>>
>>> May I ask what would be posted on the review comment? Would we still
>>> have a posting of all of the various job results {syntax, niceness,
>>> deletions, links} and FAIL/PASS next to them? Or would it be just
>>> {voting, non-voting} PASS/FAIL.
>>
>> It would be
>> voting: PASS
>> non-voting: PASS
>> check-build: PASS
>> check-lang: PASS
>>
>> And then you need to click on the link for details - like you do today -
>> but also to read out which check failed,
> 
> Ok, if that's the case, my preference would be to leave it as it is -
> with each check giving its own explicit pass/fail.
> 
> Aside from my personal preference, I believe explicitly separating the
> tests is more friendly for new users.

Let me cite from the blueprint for the reasons:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Reorganize the gating setup to combine the tiny jobs into a single
voting and a single non-voting check. This will give more flexibility in
adding new jobs and reduce load of Jenkins.
Problem description

Most of the current jobs are very small and it takes far longer to setup
the virtual environment than running the tests. For example the niceness
check runs in two seconds - while Jenkins needs at least 3 minutes for
the whole job. The difference is that Jenkins needs to start a new
virtual machine, install the git repository and all packages.

With the addition of checking for links, we would have the following
tiny jobs where each normally runs in less than 2 seconds:

    niceness check
    syntax check
    deletions check
    linked URL check

Let’s combine these tiny jobs into voting and non-voting jobs.

This gives us also the flexibility to add additional jobs easily and
decide per repository on which of these jobs are voting and which are
non-voting.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The flexibility and the reduced work load are nice - but indeed this is
nothing we have to change.

That's why I wanted the discussion - to weight the pros and cons.

Andreas
-- 
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Jeff Hawn,Jennifer Guild,Felix Imendörffer,HRB16746 (AG Nürnberg)
    GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126