[openstack-dev] [all][python3] use of six.iteritems()

gordon chung gord at live.ca
Thu Jun 11 14:31:22 UTC 2015


> it is just me, or are these differences pretty negligible considering 
> this is the "1 million item dictionary", which in itself is a unicorn 
> in openstack code or really most code anywhere? 
> 
> as was stated before, if we have million-item dictionaries floating 
> around, that code has problems. I already have to wait full seconds 
> for responses to come back when I play around with Neutron + Horizon in 
> a devstack VM, and that's with no data at all. 100ms extra for a 
> hypothetical million item structure would be long after the whole app 
> has fallen over from having just ten thousand of anything, much less a 
> million. 

my concern isn't the 1million item dictionary -- i think we're all just using that as a simple test -- it's the 100 concurrent actions against a 10,000 item dictionary or the  1000 concurrent actions against at 1000 item dictionary... when tracking htop to see memory consumption, items() consistently doubled the memory consumption of iteritems().

again, i just want to reiterate, i'm not saying don't use items(), i just think we should not blindly use items() just as we shouldn't blindly use iteritems()/viewitems()

> 
> My only concern with items() is that it is semantically different in 
> Py2k / Py3k. Code that would otherwise have a "dictionary changed 
> size" issue under iteritems() / py3k items() would succeed under py2k 
> items(). If such a coding mistake is not covered by tests (as this is 
> a data-dependent error condition), it would manifest as a sudden error 
> condition on Py3k only. 
> 
 		 	   		  


More information about the OpenStack-dev mailing list