[openstack-dev] OpenStack Developer Mailing List Digest May 7-13
mike at openstack.org
Mon May 16 15:39:39 UTC 2016
HTML render: http://www.openstack.org/blog/2016/05/openstack-developer-mailing-list-digest-20160513/
* Pabelanger: bare-precise has been replaced by ubuntu-precise. Long live DIB
* bknudson: The Keystone CLI is finally gone. Long live openstack CLI.
* Jrichli: swift just merged a large effort that started over a year ago that
will facilitate new capabilities - like encryption
* All: https://wiki.openstack.org/wiki/Successes
Release Count Down for Week R-20, May 16-20
- Teams should have published summaries from summit sessions to the
openstack-dev mailing list.
- Spec writing
- Review priority features
* General notes
- Release announcement emails will be tagged with 'new' instead of 'release'.
- Release cycle model tags now say explicitly that the release team manages
* Release actions
- Release liaisons should add their name and contact information to this list
- New liaisons should understand release instructions .
- Project teams that want to change their release model should do so before
the first milestone in R-18.
* Important dates
- Newton 1 milestone: R-18 June 2
- Newton release schedule 
Collecting Our Wiki Use Cases
* At the beginning, the community has been using a wiki  as a default
community information publication platform.
* There's a struggle with:
- Keeping things up-to-date.
- Prevent from being vandalized.
- Old processes.
- Projects that no longer exist.
* This outdated information can make it confusing to use, especially newcomers,
that search engines will provides references to.
* Various efforts have happened to push information out of the wiki to proper
documentation guides like:
- Infrastructure guide 
- Project team guide 
* Peer reviewed reference websites:
- Releases 
- Security 
* There are a lot of use cases that a wiki is a good solution, and we'll likely
need a lightweight publication platform like the wiki to cover those use
* If you use the wiki as part of your OpenStack work, make sure it's captured
in this etherpad .
* Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-May/094681.html
Supporting Go (continued)
* Continuing from previous Dev Digest .
* Before Go 1.5 (without the -buildmode=shared) it didn't support the concept
of shared libraries. As a consequence, when a library upgrades, the release
team has to trigger rebuild for each and every reverse dependency.
* In Swift's case for looking at Go, it's hard to write a network service in
Python that shuffles data between the network and a block device and
effectively use all the hardware available.
- Fork()'ing child processes using cooperative concurrency via eventlet has
worked well, but managing all async operations across many cores and many
drives is really hard. There's not an efficient interface in Python. We're
talking about efficient tools for the job at hand.
- Eventlet, asyncio or anything else single threaded will have the same
problem of the filesystem syscalls taking a long time and the call thread
can be blocked. For example:
+ Call select()/epoll() to wait for something to happen with many file
+ For each ready file descriptor, if the file descriptor socket is
readable, read it, otherwise EWOULDBLOCK is returned by the kernel, and
move on to the next file descriptor.
* Designate team explains their reasons for Go:
- MiniDNS is a component that due to the way it works, it's difficult to make
- The component takes data and sends a zone transfer every time a record set
gets updated. That is a full (AXFR) zone transfer where every record in
a zone gets sent to each DNS server that end users can hit.
+ There is a DNS standard for incremental change, but it's complex to
implement, and can often end up reverting to a full zone transfer.
- Ns[1-6].example.com may be tens or hundreds of servers behind anycast Ips
and load balancers.
- Internal or external zones can be quite large. Think 200-300Mb.
- A zone can have high traffic where a record is added/removed for each
- The Designate team is small, and after looking at options, judging the
amount of developer hours available, a different language was decided.
* Looking at Designates implementation, there are some low-hanging fruit
improvements that can be made:
- Stop spawning a thread per request.
- Stop instantiating Oslo config object per request.
- Avoid 3 round trips to the database every request. Majority of the request
here is not spent in Python. This data should be trivial to cache since
Designate knows when to invalidate the cache data.
+ In a real world use case, there could be a cache miss due to the shuffle
order of multiple miniDNS servers.
* The Designate team saw 10x improvement for 2000 record AXFR (without
caching). Caching would probably speed up the Go implementation as well.
* Go historically has poor performance with multiple cores .
- Main advantages with the language could be CSP model.
- Twisted does this very well, but we as a community consistently support
eventlet. Eventlet has threaded programming model, which is poorly suited
for Swift's case.
- PyPy got a 40% performance improvement over Cpython for a brenchmark of
Twisted's DNS component 6 years ago .
* Right now our stack already has dependency C, Python, Erlang, Java, Shell,
* End users emphatically do not care about the language API servers were
written in. They want stability, performance and features.
* The Infrastructure related issues with Go for reliable builds, packaging, etc
is being figured out 
* Swift has tested running under PyPy with some conclusions:
- Assuming production-ready stability of PyPy and OpenStack, everyone should
use PyPy over CPython.
+ It's just simply faster.
+ There are some garbage collector related issues to still work out in
+ There are a few patches that do a better job of socket handling in Swift
that runs better under PyPy.
- PyPy only helps when you've got a CPU-constrained environment.
- The GoLang targets in Swift are related to effective thread management
syscalls, and IO.
- See a talk from the Austin Conference about this work .
* Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-May/thread.html#93795
 - https://wiki.openstack.org/wiki/CrossProjectLiaisons#Release_management
 - http://git.openstack.org/cgit/openstack/releases/tree/README.rst
 - http://releases.openstack.org/newton/schedule.html
 - https://wiki.openstack.org/wiki/Main_Page
 - http://docs.openstack.org/infra/manual/
 - http://docs.openstack.org/project-team-guide/
 - http://releases.openstack.org/
 – https://security.openstack.org
 - https://etherpad.openstack.org/p/wiki-use-cases
 - http://www.openstack.org/blog/2016/05/openstack-developer-mailing-list-digest-20160506/
 - https://golang.org/doc/faq#Why_GOMAXPROCS
 - http://morepypy.blogspot.co.nz/2010/03/hello.html
 - https://etherpad.openstack.org/p/golang-infra-issues-to-solve
 - https://www.youtube.com/watch?v=L5sCD7zEENg
More information about the OpenStack-dev