<div dir="ltr"><div class="markdown-here-wrapper" style><p style="margin:1.2em 0px!important">Apologies for the long email. If this fancy email doesn’t render correctly for you, please read it here: <a href="https://gist.github.com/lakshmi-kannan/cf953f66a397b153254a">https://gist.github.com/lakshmi-kannan/cf953f66a397b153254a</a></p>
<p style="margin:1.2em 0px!important">I was looking into fixing bug: <a href="https://bugs.launchpad.net/mistral/+bug/1401039">https://bugs.launchpad.net/mistral/+bug/1401039</a>. My idea was to use shlex to parse the string. This actually would work for anything that is supplied in the linux shell syntax. Problem is this craps out when we want to support complex data structures such as arrays and dicts as arguments. I did not think we supported a syntax to take in complex data structures in a one line format. Consider for example:</p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);border-radius:3px;display:inline;background-color:rgb(248,248,248);white-space:pre;overflow:auto;border-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block!important">      task7:
        for-each:
          vm_info: $.vms
        workflow: wf2 is_true=true object_list=[1, null, "str"]
        on-complete:
          - task9
          - task10
</code></pre><p style="margin:1.2em 0px!important">Specifically </p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);border-radius:3px;display:inline;background-color:rgb(248,248,248);white-space:pre;overflow:auto;border-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block!important">wf2 is_true=true object_list=[1, null, "str"]
</code></pre><p style="margin:1.2em 0px!important">shlex will not handle this correctly because object_list is an array. Same problem with dict. </p>
<p style="margin:1.2em 0px!important">There are 3 potential options here:</p>
<h2 id="option-1" style="margin:1.3em 0px 1em;padding:0px;font-weight:bold;font-size:1.4em;border-bottom-width:1px;border-bottom-style:solid;border-bottom-color:rgb(238,238,238)">Option 1</h2>
<p style="margin:1.2em 0px!important">1) Provide a spec for specifying lists and dicts like so:<br>list_arg=1,2,3 and dict_arg=h1:h2,h3:h4,h5:h6</p>
<p style="margin:1.2em 0px!important">shlex will handle this fine but there needs to be a code that converts the argument values to appropriate data types based on schema. (ActionSpec should have a parameter schema probably in jsonschema). This is doable. </p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);border-radius:3px;display:inline;background-color:rgb(248,248,248);white-space:pre;overflow:auto;border-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block!important">wf2 is_true=true object_list="1,null,"str""
</code></pre><h2 id="option-2" style="margin:1.3em 0px 1em;padding:0px;font-weight:bold;font-size:1.4em;border-bottom-width:1px;border-bottom-style:solid;border-bottom-color:rgb(238,238,238)">Option 2</h2>
<p style="margin:1.2em 0px!important">2) Allow JSON strings to be used as arguments so we can json.loads them (if it fails, use them as simple string). For example, with this approach, the line becomes</p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);border-radius:3px;display:inline;background-color:rgb(248,248,248);white-space:pre;overflow:auto;border-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block!important">wf2 is_true=true object_list="[1, null, "str"]"
</code></pre><p style="margin:1.2em 0px!important">This would pretty much resemble <a href="http://stackoverflow.com/questions/7625786/type-dict-in-argparse-add-argument">http://stackoverflow.com/questions/7625786/type-dict-in-argparse-add-argument</a></p>
<h2 id="option-3" style="margin:1.3em 0px 1em;padding:0px;font-weight:bold;font-size:1.4em;border-bottom-width:1px;border-bottom-style:solid;border-bottom-color:rgb(238,238,238)">Option 3</h2>
<p style="margin:1.2em 0px!important">3) Keep the spec as such and try to parse it. I have no idea how we can do this reliably. We need a more rigorous lexer. This syntax doesn’t translate well when we want to build a CLI. Linux shells cannot support this syntax natively. This means people would have to use shlex syntax and a translation needs to happen in CLI layer. This will lead to inconsistency. CLI uses some syntax and the action input line in workflow definition will use another. We should try and avoid this. </p>
<h2 id="option-4" style="margin:1.3em 0px 1em;padding:0px;font-weight:bold;font-size:1.4em;border-bottom-width:1px;border-bottom-style:solid;border-bottom-color:rgb(238,238,238)">Option 4</h2>
<p style="margin:1.2em 0px!important">4) Completely drop support for this fancy one line syntax in workflow. This is probably the least desired option. </p>
<h2 id="my-preference" style="margin:1.3em 0px 1em;padding:0px;font-weight:bold;font-size:1.4em;border-bottom-width:1px;border-bottom-style:solid;border-bottom-color:rgb(238,238,238)">My preference</h2>
<p style="margin:1.2em 0px!important">Looking the options, I like option2/option 1/option 4/option 3 in the order of preference. </p>
<p style="margin:1.2em 0px!important">With some documentation, we can tell people why this is hard. People will also grok because they are already familiar with CLI limitations in linux.</p>
<p style="margin:1.2em 0px!important">Thoughts? </p>
<div title="MDH:QXBvbG9naWVzIGZvciB0aGUgbG9uZyBlbWFpbC4gSWYgdGhpcyBmYW5jeSBlbWFpbCBkb2Vzbid0
IHJlbmRlciBjb3JyZWN0bHkgZm9yIHlvdSwgcGxlYXNlIHJlYWQgaXQgaGVyZTrCoGh0dHBzOi8v
Z2lzdC5naXRodWIuY29tL2xha3NobWkta2FubmFuL2NmOTUzZjY2YTM5N2IxNTMyNTRhPGRpdj48
YnI+PC9kaXY+PGRpdj48ZGl2Pkkgd2FzIGxvb2tpbmcgaW50byBmaXhpbmcgYnVnOiBodHRwczov
L2J1Z3MubGF1bmNocGFkLm5ldC9taXN0cmFsLytidWcvMTQwMTAzOS4gTXkgaWRlYSB3YXMgdG8g
dXNlIHNobGV4IHRvIHBhcnNlIHRoZSBzdHJpbmcuIFRoaXMgYWN0dWFsbHkgd291bGQgd29yayBm
b3IgYW55dGhpbmcgdGhhdCBpcyBzdXBwbGllZCBpbiB0aGUgbGludXggc2hlbGwgc3ludGF4LiBQ
cm9ibGVtIGlzIHRoaXMgY3JhcHMgb3V0IHdoZW4gd2Ugd2FudCB0byBzdXBwb3J0IGNvbXBsZXgg
ZGF0YSBzdHJ1Y3R1cmVzIHN1Y2ggYXMgYXJyYXlzIGFuZCBkaWN0cyBhcyBhcmd1bWVudHMuIEkg
ZGlkIG5vdCB0aGluayB3ZSBzdXBwb3J0ZWQgYSBzeW50YXggdG8gdGFrZSBpbiBjb21wbGV4IGRh
dGEgc3RydWN0dXJlcyBpbiBhIG9uZSBsaW5lIGZvcm1hdC4gQ29uc2lkZXIgZm9yIGV4YW1wbGU6
PC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj5gYGA8L2Rpdj48ZGl2PiZuYnNwOyAmbmJzcDsgJm5i
c3A7IHRhc2s3OjwvZGl2PjxkaXY+Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IGZvci1lYWNo
OjwvZGl2PjxkaXY+Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7ICZuYnNwOyB2bV9pbmZvOiAk
LnZtczwvZGl2PjxkaXY+Jm5ic3A7ICZuYnNwOyAmbmJzcDsgJm5ic3A7IHdvcmtmbG93OiB3ZjIg
aXNfdHJ1ZT10cnVlIG9iamVjdF9saXN0PVsxLCBudWxsLCAic3RyIl08L2Rpdj48ZGl2PiZuYnNw
OyAmbmJzcDsgJm5ic3A7ICZuYnNwOyBvbi1jb21wbGV0ZTo8L2Rpdj48ZGl2PiZuYnNwOyAmbmJz
cDsgJm5ic3A7ICZuYnNwOyAmbmJzcDsgLSB0YXNrOTwvZGl2PjxkaXY+Jm5ic3A7ICZuYnNwOyAm
bmJzcDsgJm5ic3A7ICZuYnNwOyAtIHRhc2sxMDwvZGl2PjxkaXY+YGBgPC9kaXY+PGRpdj48YnI+
PC9kaXY+PGRpdj5TcGVjaWZpY2FsbHkmbmJzcDs8L2Rpdj48ZGl2PmBgYDwvZGl2PjxkaXY+d2Yy
IGlzX3RydWU9dHJ1ZSBvYmplY3RfbGlzdD1bMSwgbnVsbCwgInN0ciJdPC9kaXY+PGRpdj5gYGA8
L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PnNobGV4IHdpbGwgbm90IGhhbmRsZSB0aGlzIGNvcnJl
Y3RseSBiZWNhdXNlIG9iamVjdF9saXN0IGlzIGFuIGFycmF5LiBTYW1lIHByb2JsZW0gd2l0aCBk
aWN0LiZuYnNwOzwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+VGhlcmUgYXJlIDMgcG90ZW50aWFs
IG9wdGlvbnMgaGVyZTo8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PiMjIE9wdGlvbiAxPC9kaXY+
PGRpdj4xKSBQcm92aWRlIGEgc3BlYyBmb3Igc3BlY2lmeWluZyBsaXN0cyBhbmQgZGljdHMgbGlr
ZSBzbzo8L2Rpdj48ZGl2Pmxpc3RfYXJnPTEsMiwzIGFuZCBkaWN0X2FyZz1oMTpoMixoMzpoNCxo
NTpoNjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+c2hsZXggd2lsbCBoYW5kbGUgdGhpcyBmaW5l
IGJ1dCB0aGVyZSBuZWVkcyB0byBiZSBhIGNvZGUgdGhhdCBjb252ZXJ0cyB0aGUgYXJndW1lbnQg
dmFsdWVzIHRvIGFwcHJvcHJpYXRlIGRhdGEgdHlwZXMgYmFzZWQgb24gc2NoZW1hLiAoQWN0aW9u
U3BlYyBzaG91bGQgaGF2ZSBhIHBhcmFtZXRlciBzY2hlbWEgcHJvYmFibHkgaW4ganNvbnNjaGVt
YSkuIFRoaXMgaXMgZG9hYmxlLiZuYnNwOzwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+YGBgPC9k
aXY+PGRpdj53ZjIgaXNfdHJ1ZT10cnVlIG9iamVjdF9saXN0PSIxLG51bGwsInN0ciIiPC9kaXY+
PGRpdj5gYGA8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PiMjIE9wdGlvbiAyPC9kaXY+PGRpdj48
YnI+PC9kaXY+PGRpdj4yKSBBbGxvdyBKU09OIHN0cmluZ3MgdG8gYmUgdXNlZCBhcyBhcmd1bWVu
dHMgc28gd2UgY2FuIGpzb24ubG9hZHMgdGhlbSAoaWYgaXQgZmFpbHMsIHVzZSB0aGVtIGFzIHNp
bXBsZSBzdHJpbmcpLiBGb3IgZXhhbXBsZSwgd2l0aCB0aGlzIGFwcHJvYWNoLCB0aGUgbGluZSBi
ZWNvbWVzPC9kaXY+PGRpdj5gYGA8L2Rpdj48ZGl2PndmMiBpc190cnVlPXRydWUgb2JqZWN0X2xp
c3Q9IlsxLCBudWxsLCAic3RyIl0iPC9kaXY+PGRpdj5gYGA8L2Rpdj48ZGl2PlRoaXMgd291bGQg
cHJldHR5IG11Y2ggcmVzZW1ibGUgaHR0cDovL3N0YWNrb3ZlcmZsb3cuY29tL3F1ZXN0aW9ucy83
NjI1Nzg2L3R5cGUtZGljdC1pbi1hcmdwYXJzZS1hZGQtYXJndW1lbnQ8L2Rpdj48ZGl2Pjxicj48
L2Rpdj48ZGl2PiMjIE9wdGlvbiAzPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj4zKSBLZWVwIHRo
ZSBzcGVjIGFzIHN1Y2ggYW5kIHRyeSB0byBwYXJzZSBpdC4gSSBoYXZlIG5vIGlkZWEgaG93IHdl
IGNhbiBkbyB0aGlzIHJlbGlhYmx5LiBXZSBuZWVkIGEgbW9yZSByaWdvcm91cyBsZXhlci4gVGhp
cyBzeW50YXggZG9lc24ndCB0cmFuc2xhdGUgd2VsbCB3aGVuIHdlIHdhbnQgdG8gYnVpbGQgYSBD
TEkuIExpbnV4IHNoZWxscyBjYW5ub3Qgc3VwcG9ydCB0aGlzIHN5bnRheCBuYXRpdmVseS4gVGhp
cyBtZWFucyBwZW9wbGUgd291bGQgaGF2ZSB0byB1c2Ugc2hsZXggc3ludGF4IGFuZCBhIHRyYW5z
bGF0aW9uIG5lZWRzIHRvIGhhcHBlbiBpbiBDTEkgbGF5ZXIuIFRoaXMgd2lsbCBsZWFkIHRvIGlu
Y29uc2lzdGVuY3kuIENMSSB1c2VzIHNvbWUgc3ludGF4IGFuZCB0aGUgYWN0aW9uIGlucHV0IGxp
bmUgaW4gd29ya2Zsb3cgZGVmaW5pdGlvbiB3aWxsIHVzZSBhbm90aGVyLiBXZSBzaG91bGQgdHJ5
IGFuZCBhdm9pZCB0aGlzLiZuYnNwOzwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+IyMgT3B0aW9u
IDQ8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PjQpIENvbXBsZXRlbHkgZHJvcCBzdXBwb3J0IGZv
ciB0aGlzIGZhbmN5IG9uZSBsaW5lIHN5bnRheCBpbiB3b3JrZmxvdy4gVGhpcyBpcyBwcm9iYWJs
eSB0aGUgbGVhc3QgZGVzaXJlZCBvcHRpb24uJm5ic3A7PC9kaXY+PC9kaXY+PGRpdj48YnI+PC9k
aXY+PGRpdj4jIyBNeSBwcmVmZXJlbmNlPC9kaXY+PGRpdj48YnI+PC9kaXY+PGRpdj48c3BhbiBz
dHlsZT0iY29sb3I6IHJnYig1MSwgNTEsIDUxKTsgZm9udC1mYW1pbHk6ICdIZWx2ZXRpY2EgTmV1
ZScsIEhlbHZldGljYSwgJ1NlZ29lIFVJJywgQXJpYWwsIGZyZWVzYW5zLCBzYW5zLXNlcmlmOyBm
b250LXNpemU6IDE1cHg7IGxpbmUtaGVpZ2h0OiAyMi4zOTk5OTk2MTg1MzAzcHg7Ij5Mb29raW5n
IHRoZSBvcHRpb25zLCBJIGxpa2Ugb3B0aW9uMi9vcHRpb24gMS9vcHRpb24gNC9vcHRpb24gMyBp
biB0aGUgb3JkZXIgb2YgcHJlZmVyZW5jZS4mbmJzcDs8L3NwYW4+PC9kaXY+PGRpdj48c3BhbiBz
dHlsZT0iY29sb3I6IHJnYig1MSwgNTEsIDUxKTsgZm9udC1mYW1pbHk6ICdIZWx2ZXRpY2EgTmV1
ZScsIEhlbHZldGljYSwgJ1NlZ29lIFVJJywgQXJpYWwsIGZyZWVzYW5zLCBzYW5zLXNlcmlmOyBm
b250LXNpemU6IDE1cHg7IGxpbmUtaGVpZ2h0OiAyMi4zOTk5OTk2MTg1MzAzcHg7Ij48YnI+PC9z
cGFuPjwvZGl2PjxkaXY+PHNwYW4gc3R5bGU9ImNvbG9yOiByZ2IoNTEsIDUxLCA1MSk7IGZvbnQt
ZmFtaWx5OiAnSGVsdmV0aWNhIE5ldWUnLCBIZWx2ZXRpY2EsICdTZWdvZSBVSScsIEFyaWFsLCBm
cmVlc2Fucywgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNXB4OyBsaW5lLWhlaWdodDogMjIuMzk5
OTk5NjE4NTMwM3B4OyI+V2l0aCBzb21lIGRvY3VtZW50YXRpb24sIHdlIGNhbiB0ZWxsIHBlb3Bs
ZSB3aHkgdGhpcyBpcyBoYXJkLiBQZW9wbGUgd2lsbCBhbHNvIGdyb2sgYmVjYXVzZSB0aGV5IGFy
ZSBhbHJlYWR5IGZhbWlsaWFyIHdpdGggQ0xJIGxpbWl0YXRpb25zIGluIGxpbnV4Ljwvc3Bhbj48
YnI+PC9kaXY+PGRpdj48c3BhbiBzdHlsZT0iY29sb3I6IHJnYig1MSwgNTEsIDUxKTsgZm9udC1m
YW1pbHk6ICdIZWx2ZXRpY2EgTmV1ZScsIEhlbHZldGljYSwgJ1NlZ29lIFVJJywgQXJpYWwsIGZy
ZWVzYW5zLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE1cHg7IGxpbmUtaGVpZ2h0OiAyMi4zOTk5
OTk2MTg1MzAzcHg7Ij48YnI+PC9zcGFuPjwvZGl2PjxkaXY+PHNwYW4gc3R5bGU9ImNvbG9yOiBy
Z2IoNTEsIDUxLCA1MSk7IGZvbnQtZmFtaWx5OiAnSGVsdmV0aWNhIE5ldWUnLCBIZWx2ZXRpY2Es
ICdTZWdvZSBVSScsIEFyaWFsLCBmcmVlc2Fucywgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNXB4
OyBsaW5lLWhlaWdodDogMjIuMzk5OTk5NjE4NTMwM3B4OyI+VGhvdWdodHM/Jm5ic3A7PC9zcGFu
PjwvZGl2Pg==" style="height:0;width:0;max-height:0;max-width:0;overflow:hidden;font-size:0em;padding:0;margin:0">​</div></div></div>