Open Stack

Thanks for the suggestion, and having a CI bot running on oslo.privsep
would be a good thing once the basic code works on Windows - I would have
expected it to be already running on all oslo projects (that the hyper-v
hypervisor does/might depend on) tbh.

But that's a clumsy way to actually develop something.  I _know_ privsep
won't work currently on windows (imports pwd/grp for starters), and having
to add print statements + submit + push-to-gerrit + wait for a CI bot + dig
through CI bot logs + repeat is a pretty terrible workflow ;-)

(I already have very little empathy for anyone choosing to run Windows just
to provide yet-another-x86-hypervisor when far easier alternatives are
available, and I won't donate a disproportionate amount of my time to a
well-funded company who has historically actively undermined the
intellectual commons when _I_ have far more rewarding alternatives
available.  I'm afraid this has to be easy for me or I'm just not going to
be able to sustain the effort required :-/ )

 - Gus

On Thu, 26 Nov 2015 at 11:56 Alessandro Pilotti <
apilotti at cloudbasesolutions.com> wrote:

> Hi Angus,
>
> First thanks for your concern on code portability! It still happens that
> we have to ask to revert patches on Oslo projects due to some Linux
> specific code that we discover only when the actual Oslo modules are used
> by Nova, Neutron, Cinder or other projects. Typically a running Windows CI
> suddenly turns red on every patch and that’s how we get to find out. We’re
> working on adding many more projects under Windows CI tests, so at some
> point all the relevant Oslo ones will be covered and we’ll be able to
> prevent those situations before the code gets merged.
>
> What about preparing some basic integration tests for oslo.privsep that we
> could add to our Windows CI infrastructure? This would give you peace of
> mind during development on Linux without needing to test manually all your
> patches on Windows, knowing that the Windows CI will give an error if the
> patch contains non portable code (or code that doesn’t behave as expected).
>
> Alessandro
>
>
>
> From: Angus Lees <gus at inodes.org>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev at lists.openstack.org>
> Date: Thursday 26 November 2015 at 01:56
> To: Claudiu Belu <cbelu at cloudbasesolutions.com>, "OpenStack Development
> Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org
> >
> Subject: Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows
>
> So I spent a day yesterday trying to get to the point where I could just
> run a no-change "tox" successfully on windows.  Unfortunately I gave up
> when I realised I still had several days of downloading+building things
> ahead of me and clearly I was doing it the hard way :(
>
> Could you point me to the "dev environment how-to" doc for hyper-v work?
> I'm thinking of something like
> http://docs.openstack.org/developer/nova/development.environment.html#setup with
> simple cut+paste instructions for the totally-windows-clueless like me ;)
>  Or perhaps a pre-prepared disk image, if the Microsoft license allows such
> a thing.  In particular, the details on
> https://wiki.openstack.org/wiki/Hyper-V look out of date (Grizzly, and
> several docs.openstack.org links are broken), and the links to things
> like https://cloudbase.it/hyper-v-nova-compute-installer-unattended-setup/ look
> like deployment rather than development environments (?).
>
> In particular, I think(?) I should be careful *not* to install too much of
> a cygwin-style environment, since I think(?) that might no longer be
> representative of the environment in which hyper-v is expected to operate.
> If I'm wrong, and cygwin/msys is ok, then it looks like there are several
> free-software-on-windows "distributions" that would make life a whole lot
> simpler (by frankly being less like Windows).  Some guidance from people
> who understand the issues here would be appreciated.
>
>  - Gus
>
> On Tue, 24 Nov 2015 at 22:01 Claudiu Belu <cbelu at cloudbasesolutions.com>
> wrote:
>
>> Hello,
>>
>> Thanks Dims for raising the concern and Angus for reaching out. :)
>>
>> Most of the time, python development on Windows is not too far off from
>> Linux. But the two systems are quite different, which imply different
>> modules (fcntl, pwd, grp modules do not exist in Windows) or different
>> implementations of some modules (multiprocessing uses Popen instead of
>> os.fork, os module is quite different) or some socket options and signals
>> are different in Windows.
>>
>> 1.a. As I've said earlier, some modules do not exist in Windows. All, or
>> at least most standard modules document the fact that they are strictly for
>> Linux. [1][2][3]
>> b. At the very least, running the unit tests in a Windows environment can
>> at least detect simple problems (e.g. imports). Secondly, there is a
>> Hyper-V / Windows CI running on some of the OpenStack projects (nova,
>> neutron, networking_hyperv, cinder) that can be checked before merging.
>>
>> 2. This is a bit more complicated question. Well, for functions, you
>> could have separate modules for Linux specific functions and Windows
>> specific functions. This has been done before: [4] As for object-oriented
>> implementations, I'd suggest having the system-specific calls be done in
>> private methods, which will be overriden by Windows / Linux subclasses with
>> their specific implementations. We've done something like this before, but
>> solutions were pretty much straight-forward; it might not be as simple for
>> oslo_privsep, since it is very Linux-specific.
>>
>> 3. Typically, the OpenStack services on Hyper-V / Windows are run with
>> users that have enough privileges to do their job. For example, the
>> nova-compute service is run with a user that has Hyper-V Admin privileges
>> and is not necessarily in the "Administrator" user group. We haven't used
>> rootwrap in our usecases, it is disabled by default, plus, oslo.rootwrap
>> imports pwd, which doesn't exist in Windows.
>>
>> [1] https://docs.python.org/2/library/fcntl.html
>> [2] https://docs.python.org/2/library/pwd.html
>> [3] https://docs.python.org/2/library/grp.html
>> [4]
>> https://github.com/openstack/neutron/blob/master/neutron/agent/common/utils.py
>> [5]
>> https://github.com/openstack/oslo.rootwrap/blob/master/oslo_rootwrap/wrapper.py#L19
>>
>> If you have any further questions, feel free to ask. :)
>>
>> Best regards,
>> Claudiu Belu
>>
>>
>> ------------------------------
>> *From:* Angus Lees [gus at inodes.org]
>> *Sent:* Tuesday, November 24, 2015 6:18 AM
>> *To:* OpenStack Development Mailing List (not for usage questions);
>> Claudiu Belu
>> *Subject:* [hyper-v] oslo.privsep vs Windows
>>
>> Dims has just raised[1] the excellent concern that oslo.privsep will need
>> to at least survive on Windows, because hyper-v.  I have no real experience
>> coding on windows (I wrote a windows C program once, but I only ever ran it
>> under wine ;) and certainly none within an OpenStack/python context:
>>
>> 1) How can I test whatever I'm working on to see if I have mistakenly
>> introduced something Linux-specific?  Surely this is a challenge common
>> across every project in the nova/oslo/hyper-v stack.
>>
>> 2) What predicate should I use to guard the inevitable Linux-specific or
>> Windows-specific code branches?
>>
>> and I guess:
>> 3) What does a typical OpenStack/hyper-v install even look like? Do we
>> run rootwrap with some sudo-like-thing, or just run everything as the
>> superuser?
>> What _should_ oslo.privsep do for this environment?
>>
>> [1] https://review.openstack.org/#/c/244984
>> <https://review.openstack.org/#/c/244984/1>
>>
>>  - Gus
>>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151126/001b7eaf/attachment.html>