Open Stack

In-line.

On 14 March 2016 at 10:53, Darragh Bailey <daragh.bailey at gmail.com> wrote:

>
> Sorry forgot use reply-all.
>
> Also slight mistake below, ignore my comment about needing to look at
> deep_format. I'm pretty certain that it's just the dimensions piece you
> want to look at. Assuming I understand your intent correctly.
>
> ---------- Forwarded message ----------
> From: Darragh Bailey <daragh.bailey at gmail.com>
> Date: 14 March 2016 at 14:51
> Subject: Re: [OpenStack-Infra] JJB nested template variables
> To: Thanh Ha <thanh.ha at linuxfoundation.org>
>
>
>
> Hi Thanh,
>
> On 9 Mar 2016 22:47, "Thanh Ha" <thanh.ha at linuxfoundation.org> wrote:
>
>> Hi Everyone,
>>
>> I'm trying to nest template variables and discovered that JJB behaves in
>> a way I didn't expect when a template variable is nested. For example:
>>
>> - project:
>>     name: test
>>     jobs:
>>         - '{name}-verify-{value}-{jdk}'
>>
>>     value:
>>         - a:
>>             jdk:
>>                 - openjdk8
>>
>> - job-template:
>>     name: '{name}-verify-{value}-{jdk}'
>>
>> When jdk is nested under the value a. It generates a job with the name:
>>
>>     *'test-verify-a-['\''openjdk8'\'']'*
>>
>
> Possibly we should through an exception and complain that it's not
> acceptable to have jdk resolve to a list instead of a string? Or is it you
> want to expand this to be an additional dimension upon which to generate
> more templates based on jdk being a list of possible values?
>
> I noticed that if I switch the above to:
>
>
>     value:
>         - a:
>             jdk: openjdk8
>
> That job is expanded as: test-verify-a-openjdk8
>


That's interesting so there's a method that works. Unfortunately in my use
case I need the list.



>
>> You'll notice the name also includes extra single quotes at the beginning
>> and end of the job name itself in addition to a python list being passed
>> out in place of the jdk variable. However if you don't nest JDK and put it
>> by itself you get the expected name of *test-verify-a-openjdk8 *without
>> the extra single quotes and python list.
>>
>
> It's a little unclear what you formats you were referring to.
>

I was referring to JJB generating *'test-verify-a-['\''openjdk8'\'']'*
instead of *test-verify-a-['\''openjdk8'\'']* it seems something is also
inserting single quotes to the name when we nest a list variable.


> I'd be interested in attempting to fix this since I have a use case for
>> having job templates with nested variables. Would this be something that's
>> easy to fix or would this have some larger affect on the code base?
>>
>
>> Can someone point me to the code files related to the template name
>> scheme. I can have a poke at it and see if it's something that can be fixed.
>>
>
> I believe this involves looking at how the deep_format function works. The
> various values for jdk get stored in the dimensions variable in the
> jenkins_jobs/parser.py code see
> https://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/jenkins_jobs/parser.py?id=ccf682934d14f4a2395c7b9f6d463b9667b938c8#n253
>
> Assuming what you want to do is allow the template to be expanded to
> different job names based on nested variables being a list, I think the
> exact piece of code to be changed is the loop that constructs the
> dimensions that is then iterated over to generate the jobs. See
> https://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/jenkins_jobs/parser.py?id=ccf682934d14f4a2395c7b9f6d463b9667b938c8#n258
>
> This snippet is where you need to focus:
>
>         for (k, v) in project.items():
>             tmpk = '{{{0}}}'.format(k)
>             if tmpk not in template_name:
>                 continue
>             if type(v) == list:
>                 dimensions.append(zip([k] * len(v), v))
>
> I discovered this on the weekend too and tried to poke at it a bit. I
think adding support for deep variable expansion may be a tough problem and
may require rewriting a few things since we need to figure out which
variables are nested before we can evaluate them correctly.

I'll try to poke at this if I can find some spare cycles.

Thanh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20160314/136b3aa0/attachment.html>