Open Stack

On 17/02/18 16:40, Dan Prince wrote:
> Thanks for the update Emilien. A couple of things to add:
> 
> 1) This was really difficult to pin-point via the Heat stack error
> message ('list index out of range'). I actually had to go and add
> LOG.debug statements to Heat to get to the bottom of it. I aim to sync
> with a few of the Heat folks next week on this to see if we can do
> better here.

The message itself is pretty much all we get from yaql, even in its own 
interpreter:

(py27) cat yaql_data.json
{"data": [{"foo": "bar"}]}
(py27) yaql -d yaql_data.json
Yet Another Query Language - command-line query tool
Version 1.1.3
Copyright (c) 2013-2017 Mirantis, Inc

yaql> dict($.data.where($ != 
null).flatten().selectMany($.items()).groupBy($[0], $[1], $.flatten()))
{
     "foo": [
         "bar"
     ]
}
yaql> dict($.data.where($ != 
null).flatten().selectMany($.items()).groupBy($[0], $[1], [$[0], 
$[1].flatten()]))
Execution exception: list index out of range
yaql>

(Note that different lengths of data will give you different errors though.)

The big issue here though is that for failures in validation we report 
the path in the template to the function that failed, but we don't do 
the same for failures in actually resolving the function at runtime. A 
comprehensive fix is challenging without breaking what is supposed to be 
a stable third-party plugin API, but it might be possible. Was that the 
information you needed to debug this?

We do report which resource failed, but for something with a huge 
definition like allNodesConfig I can see why that might not help as much 
as you'd hope.

> 2) I had initially thought it would have been much better to revert
> the (breaking) change to python-yaql. That said it was from 2016! So I
> think our window of opportunity for the revert is probably way too
> large to consider that. Sounds like we need to publish the yaql
> package more often in RDO, etc. So your patch to update our queries is
> probably our only option.

I _think_ this should be OK for upgrades, as long as you never do a 
stack update using the existing (Pike) templates after upgrading the 
undercloud to Queens, but... sadface.

I think we need to either merge Thomas's patch that gets rid of this 
function altogether (https://review.openstack.org/#/c/545856/) and 
backport it to older versions of t-h-t, or make yaql itself 
backward-compatible by doing something like 
https://review.openstack.org/#/c/545996/

cheers,
Zane.

> On Fri, Feb 16, 2018 at 8:36 PM, Emilien Macchi <emilien at redhat.com> wrote:
>> Upgrading YAQL from 1.1.0 to 1.1.3 breaks advanced queries with groupBy
>> aggregation.
>>
>> The commit that broke it is
>> https://github.com/openstack/yaql/commit/3fb91784018de335440b01b3b069fe45dc53e025
>>
>> It broke TripleO: https://bugs.launchpad.net/tripleo/+bug/1750032
>> But Alex and I figured (after a strong headache) that we needed to update
>> the query like this: https://review.openstack.org/545498
>>
>> It would be great to avoid this kind of change within minor versions, please
>> please.
>>
>> Happy weekend,
>>
>> PS: I'm adding YAQL to my linkedin profile right now.
> 
> Be careful here. Do you really want to write YAQL queries all day!
> 
> Dan
> 
>> --
>> Emilien Macchi
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>