<div dir="ltr"><div>Hi Anil,<br><br><br></div>Sorry I didn't pick on this before.<br><div><div><div class="gmail_extra"><br><div class="gmail_quote">On 23 February 2017 at 10:03, Anil SB <span dir="ltr"><<a href="mailto:askb23@gmail.com" target="_blank">askb23@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi,<br>
<br>
We use JJB version (1.6.1) and Jenkins 2.32.1 for all our jobs on ODL<br>
infrastructure. While testing the below code, I am trying to understand<br>
the expected behavior / differences of custom yaml tag<br>
`!include-raw-escape` (and !include-raw). From docs in [1.] suggests<br>
that escaping of any shell variables (${{VAR}} in the below script) is<br>
not required when custom tags !include-raw-escape used in builder.<br>
<br>
<br>
For instance, below code is taken from the example in docs from [1.]:<br>
<br>
// yaml file<br>
- job:<br>
name: raw-test<br>
builders:<br>
- shell:<br>
!include-raw-escape: include-raw001-vars.sh<br></blockquote><div><br></div><div>The short answer is that "!include-raw-escape" is for use inside template definitions or macros that require parameters, in which case variable substitution is performed and the use of "!include-raw-escape" prevents undesired substitution of anything inside curly braces.<br><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
// include-raw001-vars.sh<br>
#!/bin/bash<br>
#<br>
# sample script to check that brackets aren't escaped<br>
# when using the include-raw application yaml tag<br>
<br>
VAR1="hello"<br>
VAR2="world"<br>
VAR3="${VAR1} ${VAR2}"<br>
<br>
[[ -n "${VAR3}" ]] && {<br>
# this next section is executed as one<br>
echo "${VAR3}"<br>
exit 0<br>
}<br>
<br>
<br>
Once the above job is pushed into Jenkins, this gets translated to:<br>
<br>
#!/bin/bash<br>
#<br>
# sample script to check that brackets aren't escaped<br>
# when using the include-raw application yaml tag<br>
<br>
VAR1="hello"<br>
VAR2="world"<br>
VAR3="${{VAR1}} ${{VAR2}}"<br>
<br>
[[ -n "${{VAR3}}" ]] && {{<br>
# this next section is executed as one<br>
echo "${{VAR3}}"<br>
exit 0<br>
}}<br>
<br>
Above script when run returns variable substitution errors when running<br>
the job. Is this a expected behavior or known issue ? We also noticed<br>
that the escaping of variables works only for some of the scripts and<br>
not all.<br>
<br>
Thanks,<br>
Anil<br>
<br>
[1.]<br>
<a href="https://docs.openstack.org/infra/jenkins-job-builder/definition.html?highlight=job%20definition" rel="noreferrer" target="_blank">https://docs.openstack.org/<wbr>infra/jenkins-job-builder/<wbr>definition.html?highlight=job%<wbr>20definition</a><br>
<br>
"<br>
<br>
The tag !include-raw-escape: treats the given string or list of strings<br>
as filenames to be opened as one or more data blobs, which should be<br>
escaped before being read in as string data. This allows job-templates<br>
to use this tag to include scripts from files without needing to escape<br>
braces in the original file.<br>
<br>
"<br></blockquote><div><br></div><div>Thanh mentioned in IRC that this needs clearing up to note the behaviour around macros, since a macro that does not take parameters does not have the string substitution performed on it within JJB and therefore should not use !include-raw-escape.<br></div><div> </div>This is because it is not expanded within the template before the substitution is performed, but rather afterwards. Think of macros as kind of like mini-job/mini-template-job definitions. If a macro does not take parameters, it is like a standard job definition and no substitution is performed. If it does take parameters then it's like job-template and substitution takes place.<br></div><br></div><div class="gmail_extra"><br>It would be nicer if it was possible to add something in that knew when to perform the escaping automatically, maybe it's possible to have the code that expands templates and does the substitution to be able to check if something takes parameters then automatically escape anything that is of a certain object type or responds to a specific method and then perform the full substitution, otherwise if it doesn't take params just pass it through.<br></div><div class="gmail_extra"><br></div><div class="gmail_extra">Needs a fair bit of thought and break down of the different scenarios to support.<br><br></div><div class="gmail_extra">In the mean time Thanh has offered to help update the documentation to help clarify<br><br></div><div class="gmail_extra">job defintion, macro without parameters -> use !include-raw for scripts to just include the script as is<br></div><div class="gmail_extra">template job defintion, macro needing parameters -> use !include-raw-escape to escape the script so that when substitution is performed the final result is contents of the file provided.<br><br></div><div class="gmail_extra">Finally:<br></div><div class="gmail_extra">template job defintion, macro needing parameters -> Where you have a script that you wish to embed a parameter to be substituted, you must pre-escape the rest of the script and use !include-raw<br></div><div class="gmail_extra"><br></div><div class="gmail_extra">Distinguishing between the last scenario and previous one for template jobs/macros with parameters is the one that prevents the code from doing everything for you automatically.<br></div><div class="gmail_extra"><br>-- <br><div class="gmail_signature">Darragh Bailey<br>"Nothing is foolproof to a sufficiently talented fool"</div>
</div></div></div></div>