<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">In-line.</div><div class="gmail_quote"><br></div><div class="gmail_quote">On 14 March 2016 at 10:53, Darragh Bailey <span dir="ltr"><<a href="mailto:daragh.bailey@gmail.com" target="_blank">daragh.bailey@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><br></div>Sorry forgot use reply-all.<br><br></div>Also slight mistake below, ignore my comment about needing to look at deep_format. I'm pretty certain that it's just the dimensions piece you want to look at. Assuming I understand your intent correctly.<br><div><div><div><div class="h5"><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Darragh Bailey</b> <span dir="ltr"><<a href="mailto:daragh.bailey@gmail.com" target="_blank">daragh.bailey@gmail.com</a>></span><br>Date: 14 March 2016 at 14:51<br>Subject: Re: [OpenStack-Infra] JJB nested template variables<br>To: Thanh Ha <<a href="mailto:thanh.ha@linuxfoundation.org" target="_blank">thanh.ha@linuxfoundation.org</a>><br><br><br><div dir="ltr"><div><br></div>Hi Thanh,<br><div><br><div class="gmail_quote"><span>On 9 Mar 2016 22:47, "Thanh Ha" <<a href="mailto:thanh.ha@linuxfoundation.org" target="_blank">thanh.ha@linuxfoundation.org</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Everyone,<div><br></div><div>I'm trying to nest template variables and discovered that JJB behaves in a way I didn't expect when a template variable is nested. For example:</div><div><br></div><div><div><font face="monospace, monospace">- project:</font></div><div><font face="monospace, monospace"> name: test</font></div><div><font face="monospace, monospace"> jobs:</font></div><div><font face="monospace, monospace"> - '{name}-verify-{value}-{jdk}'</font></div><div><font face="monospace, monospace"><br></font></div><div><font face="monospace, monospace"> value:</font></div><div><font face="monospace, monospace"> - a:</font></div><div><font face="monospace, monospace"> jdk:</font></div><div><font face="monospace, monospace"> - openjdk8</font></div><div><font face="monospace, monospace"><br></font></div><div><font face="monospace, monospace">- job-template:</font></div><div><font face="monospace, monospace"> name: '{name}-verify-{value}-{jdk}'</font></div></div><div><br></div><div>When jdk is nested under the value a. It generates a job with the name:</div><div><br></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0)"> <b>'test-verify-a-['\''openjdk8'\'']'</b></span><br></span></div></div></blockquote><div><br></div><div></div></span><div>Possibly we should through an exception and complain that it's not acceptable to have jdk resolve to a list instead of a string? Or is it you want to expand this to be an additional dimension upon which to generate more templates based on jdk being a list of possible values?<br><br></div><div>I noticed that if I switch the above to:<br><br><div><font face="monospace, monospace"><br></font></div><div><font face="monospace, monospace"> value:</font></div><div><font face="monospace, monospace"> - a:</font></div><div><font face="monospace, monospace"> jdk: openjdk8</font></div><br></div><div>That job is expanded as: <span style="font-family:monospace"><span style="color:rgb(0,0,0)">test-verify-a-openjdk8</span></span></div></div></div></div></div></div></div></div></div></div></blockquote><div><br></div><div><br></div><div>That's interesting so there's a method that works. Unfortunately in my use case I need the list.</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="h5"><div class="gmail_quote"><div dir="ltr"><div class="gmail_quote"><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><span style="font-family:monospace"></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0)"><b></b></span></span></div><div><font face="monospace" color="#000000">You'll notice the name also includes extra single quotes at the beginning and end of the job name itself in addition to a python list being passed out in place of the jdk variable. However if you don't nest JDK and put it by itself you get the expected name of <b>test-verify-a-openjdk8 </b>without the extra single quotes and python list.</font></div></div></blockquote><div><br></div></span><div>It's a little unclear what you formats you were referring to.</div></div></div></div></div></div></blockquote><div><br></div><div>I was referring to JJB generating <b>'test-verify-a-['\''openjdk8'\'']'</b> instead of <b>test-verify-a-['\''openjdk8'\'']</b> it seems something is also inserting single quotes to the name when we nest a list variable.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div><div class="h5"><div class="gmail_quote"><div dir="ltr"><div><div class="gmail_quote"><span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><font face="monospace" color="#000000"></font></div><div><font face="monospace" color="#000000">I'd be interested in attempting to fix this since I have a use case for having job templates with nested variables. Would this be something that's easy to fix or would this have some larger affect on the code base?</font></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Can someone point me to the code files related to the template name scheme. I can have a poke at it and see if it's something that can be fixed.</div></div></blockquote><div><br></div></span><div>I believe this involves looking at how the deep_format function works. The various values for jdk get stored in the dimensions variable in the jenkins_jobs/parser.py code see <a href="https://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/jenkins_jobs/parser.py?id=ccf682934d14f4a2395c7b9f6d463b9667b938c8#n253" target="_blank">https://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/jenkins_jobs/parser.py?id=ccf682934d14f4a2395c7b9f6d463b9667b938c8#n253</a><br><br></div><div>Assuming what you want to do is allow the template to be expanded to different job names based on nested variables being a list, I think the exact piece of code to be changed is the loop that constructs the dimensions that is then iterated over to generate the jobs. See <a href="https://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/jenkins_jobs/parser.py?id=ccf682934d14f4a2395c7b9f6d463b9667b938c8#n258" target="_blank">https://git.openstack.org/cgit/openstack-infra/jenkins-job-builder/tree/jenkins_jobs/parser.py?id=ccf682934d14f4a2395c7b9f6d463b9667b938c8#n258</a><br><br></div><div>This snippet is where you need to focus:<br></div><div><pre><code> for (k, v) in project.items():
tmpk = '{{{0}}}'.format(k)
if tmpk not in template_name:
continue
if type(v) == list:
dimensions.append(zip([k] * len(v), v))</code></pre></div></div></div></div></div></div></div></div></div></div></blockquote></div></div><div class="gmail_extra">I discovered this on the weekend too and tried to poke at it a bit. I think adding support for deep variable expansion may be a tough problem and may require rewriting a few things since we need to figure out which variables are nested before we can evaluate them correctly.</div><div class="gmail_extra"><br></div><div class="gmail_extra">I'll try to poke at this if I can find some spare cycles.</div><div class="gmail_extra"><br></div><div class="gmail_extra">Thanh</div><div class="gmail_extra"><br></div></div>