<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.gmail-im
{mso-style-name:gmail-im;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>From:</span></b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'> Morgan Fainberg [mailto:morgan.fainberg@gmail.com] <br><b>Sent:</b> Thursday, November 3, 2016 4:31 PM<br><b>To:</b> OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org><br><b>Subject:</b> Re: [openstack-dev] [all][dev][python] constructing a deterministic representation of a python data structure<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On Thu, Nov 3, 2016 at 1:04 PM, Amrith Kumar <<a href="mailto:amrith@tesora.com" target="_blank">amrith@tesora.com</a>> wrote:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><p class=MsoNormal style='margin-bottom:12.0pt'>Gordon,<br><br>You can see a very quick-and-dirty prototype of the kind of thing I'm<br>looking to do in Trove at<br><a href="https://gist.github.com/amrith/6a89ff478f81c2910e84325923eddebe" target="_blank">https://gist.github.com/amrith/6a89ff478f81c2910e84325923eddebe</a><br><br>Uncommenting line 51 would simulate a bad hash.<br><br>I'd be happy to propose something similar in oslo.messaging if you think<br>that would pass muster there.<br><br><span class=gmail-im>-amrith</span><br><br><span class=gmail-im>-----Original Message-----</span><br><span class=gmail-im>From: gordon chung [mailto:<a href="mailto:gord@live.ca">gord@live.ca</a>]</span><br><span class=gmail-im>Sent: Thursday, November 3, 2016 3:09 PM</span><br><span class=gmail-im>To: <a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a></span><br><span class=gmail-im>Subject: Re: [openstack-dev] [all][dev][python] constructing a deterministic</span><br><span class=gmail-im>representation of a python data structure</span><br><br><br><o:p></o:p></p><div><div><p class=MsoNormal>On 03/11/16 02:24 PM, Amrith Kumar wrote:<br><br>><br>> So, just before calling call() or cast(), I could compute the hash and<br>> stuff it into the dictionary that is being sent over, and I can do the<br>> same on the receiving side. But since I cannot guarantee that the<br>> representation on the receiving side is necessarily identical to the<br>> representation on the sending side, I have issues computing the hash.<br>><br>><br><br>based on description, you're trying to sign the messages? there was some<br>effort done in oslo.messaging[1]<br><br>we do something similar in Ceilometer to sign IPC messages[2]. it does add<br>overhead though.<br><br>[1] <a href="https://review.openstack.org/#/c/205330/" target="_blank">https://review.openstack.org/#/c/205330/</a><br>[2]<br><a href="https://github.com/openstack/ceilometer/blob/ffc9ee99c10ede988769907fdb0594a512c890cd/ceilometer/publisher/utils.py#L43-L58" target="_blank">https://github.com/openstack/ceilometer/blob/ffc9ee99c10ede988769907fdb0594a<br>512c890cd/ceilometer/publisher/utils.py#L43-L58</a><br><br>cheers,<br>--<br>gord<br><br>__________________________________________________________________________<br>OpenStack Development Mailing List (not for usage questions)<br>Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p></div></div><p class=MsoNormal style='margin-bottom:12.0pt'><br>__________________________________________________________________________<br>OpenStack Development Mailing List (not for usage questions)<br>Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p></blockquote></div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I had to solve a similar issue for deterministic key generation in dogpile (key for memcache/etc) when memoizing methods/functions with kwargs. There are a couple issues you run into, default args are not represented in **kwargs, and non-positional args can come in any order. <o:p></o:p></p></div><div><p class=MsoNormal><span style='color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'>[Amrith Kumar] This, as it turns out, is a good thing. In the specific use case that I have, the sender of the message may be on one version and the recipient may be on a later version which defaults some parameters. Therefore intercepting and handing the arguments in a decorator is perfect for me because I don’t get to see the impact of the default arguments that are added by the new version of the receiver. Also, since this is used in the context of cast() and call() in oslo.messaging, all parameters are passed as kwargs and there are no positional arguments.<o:p></o:p></span></i></b></p><p class=MsoNormal><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'><o:p> </o:p></span></i></b></p><p class=MsoNormal><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'>Here, for example, is a call to _cast() for the resize_flavor() RPC call in the Trove Taskmanager.<o:p></o:p></span></i></b></p><p class=MsoNormal><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'><o:p> </o:p></span></i></b></p><p class=MsoNormal style='margin-left:5.25pt'><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'> def resize_flavor(self, instance_id, old_flavor, new_flavor):<o:p></o:p></span></i></b></p><p class=MsoNormal style='margin-left:5.25pt'><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'> …<o:p></o:p></span></i></b></p><p class=MsoNormal style='margin-left:5.25pt'><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'><o:p> </o:p></span></i></b></p><p class=MsoNormal style='margin-left:5.25pt'><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'> self._cast("resize_flavor", self.version_cap,<o:p></o:p></span></i></b></p><p class=MsoNormal style='margin-left:5.25pt'><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'> instance_id=instance_id,<o:p></o:p></span></i></b></p><p class=MsoNormal style='margin-left:5.25pt'><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'> old_flavor=self._transform_obj(old_flavor),<o:p></o:p></span></i></b></p><p class=MsoNormal><b><i><span style='font-size:11.0pt;font-family:Consolas;color:#1F497D'> new_flavor=self._transform_obj(new_flavor))<o:p></o:p></span></i></b></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'><o:p> </o:p></span></p></div><div><p class=MsoNormal>If you want an example of what we did to generate the cache-key programatically you can look here:<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal><a href="https://bitbucket.org/zzzeek/dogpile.cache/src/669582c2e5bf12b1303f50c4b7ba3dad308eb1cc/dogpile/cache/util.py?at=master&fileviewer=file-view-default#util.py-67:118">https://bitbucket.org/zzzeek/dogpile.cache/src/669582c2e5bf12b1303f50c4b7ba3dad308eb1cc/dogpile/cache/util.py?at=master&fileviewer=file-view-default#util.py-67:118</a><o:p></o:p></p><p class=MsoNormal><b><i><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'><o:p> </o:p></span></i></b></p><p class=MsoNormal><b><i><span style='font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D'>[Amrith Kumar] Thanks, this is awesome! But, maybe a little more than I need. I’ll snag the guts of generate_key() and that should give me what I need (I think).<o:p></o:p></span></i></b></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>You don't need all the namespace and probably not fn/module info, but this can look at the call and handle / ensure defaults also match (or be used to extract default kwargs if needed) for passing down to RPC.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>--Morgan<o:p></o:p></p></div></div></div></div></body></html>