Open Stack

On Sep 2, 2014, at 3:17 PM, Clark Boylan <cboylan at sapwetik.org> wrote:

> On Tue, Sep 2, 2014, at 11:30 AM, Yuriy Taraday wrote:
>> Hello.
>> 
>> Currently for alpha releases of oslo libraries we generate either
>> universal
>> or Python 2.x-only wheels. This presents a problem: we can't adopt alpha
>> releases in projects where Python 3.x is supported and verified in the
>> gate. I've ran into this in change request [1] generated after
>> global-requirements change [2]. There we have oslotest library that can't
>> be built as a universal wheel because of different requirements (mox vs
>> mox3 as I understand is the main difference). Because of that py33 job in
>> [1] failed and we can't bump oslotest version in requirements.
>> 
>> I propose to change infra scripts that generate and upload wheels to
>> create
>> py3 wheels as well as py2 wheels for projects that support Python 3.x (we
>> can use setup.cfg classifiers to find that out) but don't support
>> universal
>> wheels. What do you think about that?
>> 
>> [1] https://review.openstack.org/117940
>> [2] https://review.openstack.org/115643
>> 
>> -- 
>> 
>> Kind regards, Yuriy.
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> We may find that we will need to have py3k wheels in addition to the
> existing wheels at some point, but I don't think this use case requires
> it. If oslo.test needs to support python2 and python3 it should use mox3
> in both cases which claims to support python2.6, 2.7 and 3.2. Then you
> can ship a universal wheel. This should solve the immediate problem.

That sounds like a good solution to that specific issue. It may also require changes in the application test suites, but those changes can be made as we move them to use oslotest.

> 
> It has been pointed out to me that one case where it won't be so easy is
> oslo.messaging and its use of eventlet under python2. Messaging will
> almost certainly need python 2 and python 3 wheels to be separate. I
> think we should continue to use universal wheels where possible and only
> build python2 and python3 wheels in the special cases where necessary.
> 
> The setup.cfg classifiers should be able to do that for us, though PBR
> may need updating? We will also need to learn to upload potentially >1

How do you see that working? We want all of the Oslo libraries to, eventually, support both python 2 and 3. How would we use the classifiers to tell when to build a universal wheel and when to build separate wheels?

> wheel in our wheel jobs. That bit is likely straight foward. The last
> thing that we need to make sure we do is that we have some testing in
> place for the special wheels. We currently have the requirements
> integration test which runs under python2 checking that we can actually
> install all the things together. This ends up exercising our wheels and
> checking that they actually work. We don't have a python3 equivalent for

We only know the wheels can be installed. We don’t actually have a test that installs our code and runs it any more (devstack uses “develop” mode which bypasses some of the installation steps, as we found while fixing the recent neutron/pbr issue with a missing config file in their packaging instructions).

> that job. It may be better to work out some explicit checking of the
> wheels we produce that applies to both versions of python. I am not
> quite sure how we should approach that yet.

To fix the asymmetric gating we have between pbr and everything else, Robert suggested setting up some sort of job to install pbr and then build and install a package for the project being tested. We already, as you point out, have a job that does this for all of the projects to test changes to pbr itself. Maybe we can run the same test under python 2 and 3 as part of the same job?

Doug

> 
> Clark
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev