Open Stack

Hi Anil,


Sorry I didn't pick on this before.

On 23 February 2017 at 10:03, Anil SB <askb23 at gmail.com> wrote:

> Hi,
>
> We use JJB version (1.6.1) and Jenkins 2.32.1 for all our jobs on ODL
> infrastructure. While testing the below code, I am trying to understand
> the expected behavior / differences of custom yaml tag
> `!include-raw-escape` (and !include-raw). From docs in [1.] suggests
> that escaping of any shell variables (${{VAR}} in the below script) is
> not required when custom tags !include-raw-escape used in builder.
>
>
> For instance, below code is taken from the example in docs from [1.]:
>
> // yaml file
> - job:
>     name: raw-test
>     builders:
>       - shell:
>           !include-raw-escape: include-raw001-vars.sh
>

The short answer is that "!include-raw-escape" is for use inside template
definitions or macros that require parameters, in which case variable
substitution is performed and the use of "!include-raw-escape" prevents
undesired substitution of anything inside curly braces.



> // include-raw001-vars.sh
> #!/bin/bash
> #
> # sample script to check that brackets aren't escaped
> # when using the include-raw application yaml tag
>
> VAR1="hello"
> VAR2="world"
> VAR3="${VAR1} ${VAR2}"
>
> [[ -n "${VAR3}" ]] && {
>     # this next section is executed as one
>     echo "${VAR3}"
>     exit 0
> }
>
>
> Once the above job is pushed into Jenkins, this gets translated to:
>
> #!/bin/bash
> #
> # sample script to check that brackets aren't escaped
> # when using the include-raw application yaml tag
>
> VAR1="hello"
> VAR2="world"
> VAR3="${{VAR1}} ${{VAR2}}"
>
> [[ -n "${{VAR3}}" ]] && {{
>     # this next section is executed as one
>     echo "${{VAR3}}"
>     exit 0
> }}
>
> Above script when run returns variable substitution errors when running
> the job. Is this a expected behavior or known issue ? We also noticed
> that the escaping of variables works only for some of the scripts and
> not all.
>
> Thanks,
> Anil
>
> [1.]
> https://docs.openstack.org/infra/jenkins-job-builder/
> definition.html?highlight=job%20definition
>
> "
>
> The tag !include-raw-escape: treats the given string or list of strings
> as filenames to be opened as one or more data blobs, which should be
> escaped before being read in as string data. This allows job-templates
> to use this tag to include scripts from files without needing to escape
> braces in the original file.
>
> "
>

Thanh mentioned in IRC that this needs clearing up to note the behaviour
around macros, since a macro that does not take parameters does not have
the string substitution performed on it within JJB and therefore should not
use !include-raw-escape.

This is because it is not expanded within the template before the
substitution is performed, but rather afterwards. Think of macros as kind
of like mini-job/mini-template-job definitions. If a macro does not take
parameters, it is like a standard job definition and no substitution is
performed. If it does take parameters then it's like job-template and
substitution takes place.


It would be nicer if it was possible to add something in that knew when to
perform the escaping automatically, maybe it's possible to have the code
that expands templates and does the substitution to be able to check if
something takes parameters then automatically escape anything that is of a
certain object type or responds to a specific method and then perform the
full substitution, otherwise if it doesn't take params just pass it through.

Needs a fair bit of thought and break down of the different scenarios to
support.

In the mean time Thanh has offered to help update the documentation to help
clarify

job defintion, macro without parameters -> use !include-raw for scripts to
just include the script as is
template job defintion, macro needing parameters -> use !include-raw-escape
to escape the script so that when substitution is performed the final
result is contents of the file provided.

Finally:
template job defintion, macro needing parameters -> Where you have a script
that you wish to embed a parameter to be substituted, you must pre-escape
the rest of the script and use !include-raw

Distinguishing between the last scenario and previous one for template
jobs/macros with parameters is the one that prevents the code from doing
everything for you automatically.

-- 
Darragh Bailey
"Nothing is foolproof to a sufficiently talented fool"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20170308/54696330/attachment.html>