Open Stack

On 27/03/19 5:56 PM, Oleg Ovcharuk wrote:
> I’ve checked this solution. It doesn’t work as well in Python 2.7 - 
> dumped yaml includes this ugly parts 
> "!!python/object/apply:collections.OrderedDict».

It looks like you accidentally used yaml.dump() instead of 
yaml.safe_dump(), after adding the representer to only the SafeDumper class.

> Yes, it could be fixed by including another hundred lines of code, but 
> it still smells like we just copy and paste existing solution.

I've implemented this myself a couple of times and it is exactly 5 lines 
of code:

https://github.com/openstack/tripleo-heat-templates/blob/master/network/endpoints/build_endpoint_map.py#L223-L229
https://github.com/openstack/heat/blob/master/heat/common/template_format.py#L43-L58


Loading is more complicated though because you have to reimplement 
construct_mapping for maximum efficiency. The simplest version I can 
come up with is 9 lines:

class OrderedLoader(yaml.SafeLoader):
     def construct_yaml_map_ordered(self, node):
         data = collections.OrderedDict()
         yield data
         value = self.construct_mapping(node)
         keys = (self.construct_object(kn) for kn, vn in node.value)
         data.update((k, value[k]) for k in keys)

OrderedLoader.add_constructor(u'tag:yaml.org,2002:map',
                               OrderedLoader.construct_yaml_map_ordered)

but it obviously has the disadvantage of having to call 
construct_object() twice on each key.

cheers,
Zane.

>> 15 февр. 2019 г., в 8:28, Renat Akhmerov <renat.akhmerov at gmail.com 
>> <mailto:renat.akhmerov at gmail.com>> написал(а):
>>
>> This looks like a solution, yes. Thanks.
>>
>> @Oleg, please take a look. Maybe we really decided to add a new yaml 
>> lib too early.
>>
>>
>>
>> Renat
>> On 15 Feb 2019, 00:16 +0700, Ben Nemec <openstack at nemebean.com 
>> <mailto:openstack at nemebean.com>>, wrote:
>>>
>>>
>>> On 2/14/19 10:46 AM, Oleg Ovcharuk wrote:
>>>> Matthew, we use not only load, but also dump. We can't use custom
>>>> constructor and default representer - the output will be terrible.
>>>> This custom constructor contains about 50 lines of code, representer
>>>> would have a similar count. Also, we should think about compatibility
>>>> with Python 2.7, 3.x and about it's performance.
>>>> Summary, we would have about 150 lines of code, which is just copy-paste
>>>> from `yamlloader` library.
>>>> IMHO, it is better to use existing solutions.
>>>
>>> You don't need a complex representer to dump OrderedDicts. It can be
>>> done in about three lines:
>>> https://github.com/cybertron/tripleo-scripts/blob/105381d4f080394e68a40327c398d32eb9f4f580/net_processing.py#L302
>>>
>>> That's the code I used when I wanted to dump dicts in a particular
>>> order. Once you add the representer OrderedDicts are handled as you
>>> would expect.
>>>
>>>>
>>>> чт, 14 февр. 2019 г. в 19:14, Matthew Thode <mthode at mthode.org 
>>>> <mailto:mthode at mthode.org>
>>>> <mailto:mthode at mthode.org>>:
>>>>
>>>> On 19-02-14 16:58:49, Oleg Ovcharuk wrote:
>>>>> Hi! Can you please add yamlloader library to global requirements?
>>>>> https://pypi.org/project/yamlloader/
>>>>>
>>>>> It provides ability to preserve key order in dicts, it supports
>>>> either
>>>>> python 2.7 and python 3.x, it provides better performance than
>>>> built-in
>>>>> functions.
>>>>> Thank you.
>>>>
>>>> I'd like to know a little more about why we need this, yaml as a spec
>>>> itself doesn't guarantee order so order should be stored somewhere else.
>>>>
>>>> If all you need is ordereddict support something like this may be better
>>>> then adding yet another lib.
>>>>
>>>> https://gist.github.com/enaeseth/844388
>>>>
>>>> --
>>>> Matthew Thode
>>>>
>>>
>