Open Stack

The problem with existing syntax is it is not defined: there is no docs on inlining complex variables [*], and we haven’t tested it for anything more than the simplest cases: https://github.com/stackforge/mistral/blob/master/mistral/tests/unit/workbook/v2/test_dsl_specs_v2.py#L114. I will be surprised if anyone figured how to provide a complex object as an inline parameter.

Do you think regex is the right approach for parsing arbitrary key-values where values is arbitrary json structures? Will it work with something like 
	workflow: wf2 object_list=[ {“url”: “http://{$hostname}.example.com:8080?x=a&y={$.b}}, 33, null, {{$.key}, [{$.value1}, {$.value2}]}
How much tests should we write to be confident we covered all cases? I share Lakshmi’s concern it is fragile and maintaining it reliably is difficult. 

But back to the original question, it’s about requirements, not implementation. 
My preference is “option 3”, “make it work as is now”. But if it’s too hard I am ok to compromise. 
Than option 2 as it resembles option 3 and YAML/JSON conversion makes complete sense. At the expense of quoting the objects. Slight change, not significant. 
Option 1 introduce a new syntax; although familiar to CLI users, I think it’s a bit out of place in YAML definition. 
Option 4 is no go :)

DZ. 

[*] “there is no docs to this” - I subscribe on fixing this.

On Dec 16, 2014, at 9:48 PM, Renat Akhmerov <rakhmerov at mirantis.com> wrote:

> Ok, I would prefer to spend some time and think how to improve the existing reg exp that we use to parse key-value pairs. We definitely can’t just drop support of this syntax and can’t even change it significantly since people already use it.
> 
> Renat Akhmerov
> @ Mirantis Inc.
> 
> 
> 
>> On 17 Dec 2014, at 07:28, Lakshmi Kannan <lakshmi at lakshmikannan.me> wrote:
>> 
>> Apologies for the long email. If this fancy email doesn’t render correctly for you, please read it here: https://gist.github.com/lakshmi-kannan/cf953f66a397b153254a
>> 
>> I was looking into fixing bug: https://bugs.launchpad.net/mistral/+bug/1401039. My idea was to use shlex to parse the string. This actually would work for anything that is supplied in the linux shell syntax. Problem is this craps out when we want to support complex data structures such as arrays and dicts as arguments. I did not think we supported a syntax to take in complex data structures in a one line format. Consider for example:
>> 
>>       task7:
>>         for-each:
>>           vm_info: $.vms
>>         workflow: wf2 is_true=true object_list=[1, null, "str"]
>>         on-complete:
>>           - task9
>>           - task10
>> Specifically
>> 
>> wf2 is_true=true object_list=[1, null, "str"]
>> shlex will not handle this correctly because object_list is an array. Same problem with dict.
>> 
>> There are 3 potential options here:
>> 
>> Option 1
>> 
>> 1) Provide a spec for specifying lists and dicts like so:
>> list_arg=1,2,3 and dict_arg=h1:h2,h3:h4,h5:h6
>> 
>> shlex will handle this fine but there needs to be a code that converts the argument values to appropriate data types based on schema. (ActionSpec should have a parameter schema probably in jsonschema). This is doable.
>> 
>> wf2 is_true=true object_list="1,null,"str""
>> Option 2
>> 
>> 2) Allow JSON strings to be used as arguments so we can json.loads them (if it fails, use them as simple string). For example, with this approach, the line becomes
>> 
>> wf2 is_true=true object_list="[1, null, "str"]"
>> This would pretty much resemble http://stackoverflow.com/questions/7625786/type-dict-in-argparse-add-argument
>> 
>> Option 3
>> 
>> 3) Keep the spec as such and try to parse it. I have no idea how we can do this reliably. We need a more rigorous lexer. This syntax doesn’t translate well when we want to build a CLI. Linux shells cannot support this syntax natively. This means people would have to use shlex syntax and a translation needs to happen in CLI layer. This will lead to inconsistency. CLI uses some syntax and the action input line in workflow definition will use another. We should try and avoid this.
>> 
>> Option 4
>> 
>> 4) Completely drop support for this fancy one line syntax in workflow. This is probably the least desired option.
>> 
>> My preference
>> 
>> Looking the options, I like option2/option 1/option 4/option 3 in the order of preference.
>> 
>> With some documentation, we can tell people why this is hard. People will also grok because they are already familiar with CLI limitations in linux.
>> 
>> Thoughts?
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141217/77eb665e/attachment.html>