Open Stack

Hi David,

Thanks for commenting!

Sorry below is going to be a bit long but I hope my examples makes the
point clear that we can see very inconsistent behaviour for configuration
sections and it's difficult to tell which one to expect.

On 15 February 2016 at 10:02, David Caro <dcaro at redhat.com> wrote:

>
> What happens then when you want to leave the setting with it's current
> value?
>

Personally I feel like this shouldn't ever be the case. It's better to be
consistent than inconsistent in my opinion and the way it is today with JJB
it's inconsistent what the meaning of removing a section from your YAML
means. For some values it leaves it as the current value. For other values
it reverts it to the default value.

Let me put an example, in my team we use to leave notes in the jobs
> descriptions and also manually disable/enable jobs. That is useful when
> troubleshooting jobs, imagine that you have a job that cleans up the
> workspace
> on start, and it's failing, what we do is disable the job, so it does not
> cleanup the workspace and then we add a small comment in the description
> with a
> note on who's working on the job and why (if the reason for the failure is
> known).
>

I'm not saying this is the best way to do this but my team has a separate
Jenkins instance we call a Sandbox. We do all of our testing and
troubleshooting on the Sandbox instance rather than testing on our
Production instance. The Sandbox is managed with a similar puppet
configuration as the Production system so our upgrade path is to test it on
sandbox and copy the configuration over to production when it's ready.

Another option is to setup a local Jenkins instance for testing. I often
use the official Jenkins Docker container to spin up an environment on my
laptop. You can use the jenkins-jobs command to push up only a single job
to your test instance using:

    jenkins-jobs update /path/to/yamls 'job-name-to-push'


In any case I'm curious to know if the "disabled" button is the only one
you're concerned about in your use case?

If so perhaps we should make an exception for that one and document it's
behaviour.

We don't want to do so by yaml, as most probably is just one of the jobs
> generated by a matrix of combinations and that means that we would have to
> add a lot of boilerplate code just to be able to disable that specific
> job. And the debugging is transient, not meant to last more than a few
> hours.
>
> In that case, if you force jjb to send a default value all the time, that
> flow
> would stop working.
>
> I think that if you want to set defaults, you should do so in the default
> construct instead, that also prevents that any change in the defaults on
> jjb
> code from changing the generated xml.


My main concern is inconsistency in what removing a section of YAML means.
Sometimes it means leave what exists in place, while at other times it
means revert to default. Even the documentation is inconsistent. In the
example of the "disabled" parameter the documentation for it states:

     *disabled:* Boolean value to set whether or not this job should be
disabled in Jenkins. Defaults to false (job will be enabled).

However that is not true, if you do not pass disabled as a parameter the
job does not revert back to the false state.

The use case that's really hurting us right now is disabling Gerrit
Trigger. There is currently no possible way to do this via YAML. Once you
enable the Gerrit Trigger setting for any of your jobs you cannot turn it
off. There is no value to set to say you no longer want it. The intuitive
thinking would be to remove the trigger from the triggers section but it's
current behaviour is to keep that option enabled. We have almost 2000 jobs
that we need to remove the Gerrit Trigger from and currently there's no way
to do it via JJB.

While I'm using Gerrit Trigger as an example here. This case applies to
many other JJB configurations which we've been hit with in the past where
we found configurations offered no way to turn them off (the worst case
scenario) or requiring a 2 steps in YAML to remove a configuration we are
no longer interested in.

Steps
1. Flip the boolean value of option of a plugin and save it in YAML and
update
2. Remove the YAML for section you're no longer interested in and update

My opinion is that this workflow is not what a user would naturally expect
to happen and would cause confusion. As a user my expectation is that if I
remove something from my YAML then JJB should remove the configuration by
reverting to whatever is the default behaviour is for that plugin (often
the behaviour is to turn it off).

For good measure here's a few more examples of varying behaviours for
plugins in hopes that it shows why I think JJB needs to be more consistent:

*Logrotate*
1. Configure logrotate section & Update job
2. Remove logrotate section & Update job
Expected: logrotate is no longer configured in Jenkins
Result: logrotate is still configured. Why was it not removed?

*Git-SCM*
1. Configure git-scm & Update job
2. Remove git-scm & Update job
Expected: git-scm is no longer configured in Jenkins
Result: git-scm is no longer configured in Jenkins (yay)

Here's an interesting situation I found where a plugin can have
inconsistent results results depending on how you remove the configuration.
Say you have the following configuration:

    reporters:
        - findbugs
        - email:
            recipients: breakage at example.com

We're interested in the findbugs part in this example. So if you push a
test job with that configuration and check Jenkins. You should see that
findbugs is configured.

*Case 1*
1. Delete only the "- findbugs" section & Update
Expected: findbugs is disabled in Jenkins
Result: findbugs is disabled. (yay)

*Case 2*
1. Delete the entire "reporters" section including the email & Update
Expected: findbugs is disabled in Jenkins
Result: findbugs is still configured

In this case we can see that if you have multiple reporters configured and
remove one reporter by itself. Then the expected happens, findbugs is
disabled. However if you remove the entire reporters section it keeps the
configuration as is. Which behaviour is the correct behaviour here?

Thanks for reading. I hope this helps point out the issues we are seeing.

Thanh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20160216/f08ee7c2/attachment.html>