[openstack-dev] OpenStack Developer Mailing List Digest May 14 to June 17
mike at openstack.org
Fri Jun 17 21:50:38 UTC 2016
HTML render: http://www.openstack.org/blog/2016/06/openstack-developer-mailing-list-digest-20160617/
* Qiming: Senlin has completed API migration from WADL.
* Mugsie: Kiall Fixed the gate - development can now continue!!!
* notmyname: exactly 6 years ago today, Swift was put into production
* kiall: DNS API reference is live .
* sdague: Nova legacy v2 api code removed .
* HenryG: Last remnant of oslo incubator removed from Neutron .
* dstanek: I was able to perform a roundtrip between keystone and testshib.org
using my new SAML2 middleware!
* Sdague: Nova now defaults to Glance v2 for image operations .
* Ajaeger: First Project Specific Install Guide is published - congrats to the
* Jeblair: There is no Jenkins, only Zuul.
* All: https://wiki.openstack.org/wiki/Successes
Require A Level Playing Field for OpenStack Projects
* Thierry Carrez proposes a new requirement  for OpenStack “official”
* An important characteristic of open collaboration grounds is they need to be
a level playing field. No specific organization can be given an unfair
- Projects that are blessed as “official” project teams need to operate in
a fair manner. Otherwise they could be essentially a trojan horse for
a given organization.
- If in a given project, developers from one specific organization benefit
from access specific knowledge or hardware, then the project should be
rejected under the “open community” rule.
- Projects like Cinder provide an interesting grey area, but as long as all
drivers are in and there is a fully functional (and popular) open source
implementation there is likely no specific organization considered as
* Neutron plugin targeting a specific piece of networking hardware would likely
given an unfair advantage to developers of the hardware's manufacturer
(having access to hardware for testing and being able to see and make changes
to its proprietary source code).
* Open source projects that don't meet the open community requirement can still
exist in the ecosystem (developed using gerrit and openstack/* git
repository, gate testing, but as an unofficial project.
* Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-June/thread.html#97307
Add Option to Disable Some Strict Response Checking for Interop Testing
* Nova introduced their API micro version change 
* QA team adds strict API schema checking to Tempest to ensure no additional
Nova API responses .
- In the last year, three vendors participating in the OpenStack powered
trademark program were impacted by this .
* DefCore working group determines guidelines for the OpenStack powered
- Includes capabilities with associated functional tests from Tempest that
- There is a balance of future direction of development with lagging
indicators of deployments and user adoption.
* A member of the working group Chris Hoge would like to implement a temporary
waiver for strict API checking requirements.
- While this was discussed publicly in the developer community and took some
time to implement. It still landed quickly, and broke several existing
- It's not viable for downstream deployers to use older versions of Tempest
that don't have these strict response checking, due to the TC resolution
passed  to advise DefCore to use Tempest as the single source of
- Short term:
+ There will be a blueprint and patch to Tempest that allows configuration
of a grey-list of Nova APIs which strict response checking on additional
properties will be disabled.
+ Using this code will emit a deprecation warning.
+ This will be removed 2017.01.
+ Vendors are required to submit the grey-list of APIs with additional
response data that would be published to their marketplace entry.
- Long term:
+ Vendors will be expected to work with upstream to update the API
returning additional data.
+ The waiver would no longer be allowed after the release of 2017.01
* Former QA PTL Matthew Treinish feels this a big step backwards.
- Vendors who have implemented out of band extensions or injected additional
things in responses believe that by doing so they're interoperable. The API
is not a place for vendor differentation.
- Being a user of several clouds, random data in the response makes it more
difficult to write code against. Which are the vendor specific responses?
* Alternatives to not giving vendors more time in the market:
- Having some vendors leave the the Powered program unnecessarily weakening
- Force DefCore to adopt non-upstream testing, either as a fork or an
independent test suite.
* If the new enforcement policies had been applied by adding new tests to
Tempest, then DefCore could have added them using it's processes over
a period of time.
- Instead behavior to a bunch of existing tests changed.
* Tempest master today supports all currently supported stable branches.
- Tags are made in the git repository support for a release is added/dropped.
+ Branchless Tempest was originally started back in Icehouse release and
was implemented to enforce the API is the same across release boundaries.
* If DefCore wants the lowest common denominator for Kilo, Liberty, and Mitaka
there's a tag for that . For Juno, Kilo, Liberty the tag would be .
* Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-June/thread.html#97349
There Is No Jenkins, Only Zuul
* Since the inception of OpenStack, we have used Jenkins to perform testing and
- When we only had two git repositories, we have one Jenkins master and a few
slaves. This was easy to maintain.
- Things have grown significantly with 1,200 git repositories, 8,000 jobs
spread across 8 Jenkins masters and 800 dynamic slave nodes.
* Jenkins job builder  was created to create 8,000 jobs from a templated
* Zuul  was created to drive project automation in directing our testing,
running tens of thousands of jobs each day. Responding to:
- Code reviews
- Stacking potential changes to be tested together.
* Zuul version 3 has major changes:
- Easier to run jobs in multi-node environments
- Easy to manage large number of jobs
- Job variations
- Support in-tree job configuration
- Ability to define jobs using Ansible
* While version 3 is still in development, it's today capable of running our
* As of June 16th, we have turned off our last Jenkins master and all of our
automation is being run by Zuul.
- Jenkins job builder has contributors beyond OpenStack, and will be
continued to be maintained by them.
* Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-June/thread.html#97584
Languages vs. Scope of “OpenStack”
* Where does OpenStack stop, and where does the wider open source community
start? Two options:
- If OpenStack is purely an “integration engine” to lower-level technologies
(e.g. hypervisors, databases, block storage) the scope is limited and
Python should be plenty and we don't need to fragment our community.
- If OpenStack is “whatever it takes to reach our mission”, then yes we need
to add one language to cover lower-level/native optimization.
* Swift PTL John Dickinson mentions defining the scope of OpenStack projects
does not define the languages needed to implement them. The considerations
- OpenStack is defined as whatever is takes to fulfill the mission statement.
- Defining “lower level” is very hard. Since the Nova API is listening to
public network interfaces and coordinating with various services in
a cluster, it lower level enough to consider optimizations.
+ There are a large number of open source projects and libraries that
OpenStack needs to fulfill its mission that are not “OpenStack”: Python,
MySQL, KVM, Ceph, OpenvSwitch.
* Another approach is product-centric. “Lower-level pieces are OpenStack
dependencies, rather that OpenStack itself.”
- Not governed by the TC, and it can use any language and tool deemed
* Do we want to be in the business of building data plane services that will
run into Python limitations?
- Control plane services are very unlikely to ever hit a scaling concern
where rewriting in another language is needed.
* Swift hit it first because of the maturity of the project and they are now
focused on this kind of optimization.
- Glance (partially data plane) did hit this limit and mitigated by folks
using Ceph and exposing that directly to Nova. So now Glance only cares
about location and metadata. Dependencies like Ceph care about data plane.
* The resolution for the Go programming language was discussed in previous
Technical Committee meetings and was not passed . John Dickinson and
others do plan to carry another effort forward for Swift to have an exception
for usage of the language.
* Full thread: http://lists.openstack.org/pipermail/openstack-dev/2016-May/thread.html#95452
 - http://developer.openstack.org/api-ref/dns/
 - https://review.openstack.org/324466
 - https://review.openstack.org/290305
 - https://review.openstack.org/#/c/321551/
 - https://review.openstack.org/#/c/329448/
 - https://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/api-microversions.html
 - http://lists.openstack.org/pipermail/openstack-dev/2015-February/057613.html
 - https://review.openstack.org/#/c/156130
 - http://lists.openstack.org/pipermail/defcore-committee/2016-January/000986.html
 - http://git.openstack.org/cgit/openstack/governance/tree/resolutions/20160504-defcore-test-location.rst
 - http://git.openstack.org/cgit/openstack/tempest/tag/?h=12.0.0
 - http://git.openstack.org/cgit/openstack/tempest/tag/?h=8
 - http://docs.openstack.org/infra/jenkins-job-builder/
 - https://review.openstack.org/312267
More information about the OpenStack-dev