<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>
<div>
<div>Basic Python development does not really differ too much on Windows compared to Linux.</div>
</div>
</div>
<div><br>
</div>
<div>Let’s start with the Python environment. I’d recommend to install a 2.7 (x86) one and a 3.4 (x64) one:</div>
<div><br>
</div>
<div><a href="https://www.python.org/ftp/python/2.7.10/python-2.7.10.msi">https://www.python.org/ftp/python/2.7.10/python-2.7.10.msi</a></div>
<div><a href="https://www.python.org/ftp/python/3.4.3/python-3.4.3.amd64.msi">https://www.python.org/ftp/python/3.4.3/python-3.4.3.amd64.msi</a></div>
<div><br>
</div>
<div>Download also git for Windows: <a href="https://git-scm.com/download/win">https://git-scm.com/download/win</a></div>
<div>Git works in the same way, so nothing particular to add. Just take care of the line endings. I usually keep them all in LF, avoiding CRLF, others let git do the conversion.</div>
<div><br>
</div>
<div>When done, open a PowerShell or command prompt and set your PATH and PYTHONPATH e.g.:</div>
<div><br>
</div>
<div>$ENV:PATH += ";C:\Python27;C:\Python27\Scripts"</div>
<div>$ENV:PYTHONPATH = "."</div>
<div><br>
</div>
<div>From here is pretty standard Python, e.g.:</div>
<div><br>
</div>
<div> pip install “blah>=1.0.0"</div>
<div> pip install –r requirements.txt</div>
<div><br>
</div>
<div> python yourmodule.py</div>
<div><br>
</div>
<div>For running tests you can use for example nose since tox / testr don’t really work:</div>
<div><br>
</div>
<div> pip install mock</div>
<div> pip install nose</div>
<div> nosetests .</div>
<div><br>
</div>
<div>Note: you can also use virtualenvs, exactly like on Linux.</div>
<div><br>
</div>
<div>
<div>You might incur in occasional native modules that require a C development environment. That’s usually more complicated as it requires also dependencies and needs C/C++ skills and either Mingw / mingw64 or Visual Studio 2008 (Python 2.7) or 2010 (Python
3.4).</div>
</div>
<div>For those few modules there are typically binary wheels available, either on Pypi or provided by 3rd parties. If not, just ask, we compile them from source and have wheels in the CI.</div>
<div><br>
</div>
<div>What you can optionally do, to get a Python environment pre-loaded with all the OpenStack dependencies is to install the OpenStack Hyper-V Compute MSI (see http://cloudbase.it) and use the Python env that comes with it.</div>
<div><br>
</div>
<div>If you need a basic working OpenStack, just get a standard Devstack running in a Linux host or VM plus the above MSI for Hyper-V configured to point to the DevStack node. There’s plenty of examples on our blog for similar cases (http://cloudbase.it/blog).</div>
<div><br>
</div>
<div>It terms of editors it’s a matter of personal taste, but you can find e.g. Sublime, Visual Studio Code, Notepad++ or Atom as very good and lightweight options.</div>
<div><br>
</div>
<div>Let me know if this works for you and if you have any other questions.</div>
<div><br>
</div>
<div>Alessandro</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:12pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Angus Lees <<a href="mailto:gus@inodes.org">gus@inodes.org</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Thursday 26 November 2015 at 04:22<br>
<span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>>, Claudiu Belu <<a href="mailto:cbelu@cloudbasesolutions.com">cbelu@cloudbasesolutions.com</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows<br>
</div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">So seriously - how do you set up a dev environment for hyper-v?
<div>It doesn't have to be written up all nice and pretty, just a 30s outline in an email would be really useful :-/</div>
</div>
</div>
</div>
</span>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div>
<div>
<div dir="ltr">
<div><br>
</div>
<div> - Gus</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Thu, 26 Nov 2015 at 13:13 Alessandro Pilotti <<a href="mailto:apilotti@cloudbasesolutions.com">apilotti@cloudbasesolutions.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>
<div>
<div>Angus,</div>
<div><br>
</div>
</div>
</div>
</div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>"I'm afraid this has to be easy for me or I'm just not going to be able to sustain the effort required :-/ )"</div>
<div><br>
</div>
</div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>What a tragedy, I’m soooo sorry that life is so terribly bitter and requires some effort to support people with views different from yours! :-)</div>
<div><br>
</div>
<div>Just kidding, we’re here to help, but we obviously won’t do your work. Given that Oslo code needs to be portable, I was just trying to give you some alternative that doesn’t require you to do useless manual work during development.</div>
<div><br>
</div>
<div>1) Unit tests are not the answer, as hopefully they mock out most significant underlying OS features. Sure, you will catch the basic issues, like importing a module that doesn’t exist on Windows, but you cannot rely on them as a proof of portability.</div>
<div>2) Integration testing on the other side provide by definition a proof of reliability on the target system (in the limits of the tests coverage at least) and CI testing needs eventually to be there on Windows as well for oslo.<insert_any<span style="font-style:italic">_</span>name_here>.
This is obvious not a way to develop, but a “guard”, against issues. In other words, regardless of your opinion on Windows, the Windows CI needs to be there, why not adding it now for this project?</div>
<div><br>
</div>
<div>Getting to development practices, when designing features, hopefully portability gets evaluated _before_ starting coding, so you should not need too many manual runs on every platform before being confident enough to send a patch to Gerrit and wait for
CI results. And for that you don’t need tox or anything fancy, just a Python environment and some very basic OS skills.</div>
<div><br>
</div>
<div>When in doubt, an option is to drop by on #openstack-hyper-v and ask. But take care, you might find open minded people, make sure you're at ease with this. ;-)</div>
<div><br>
</div>
<div>Alessandro</div>
<div><br>
</div>
<div><br>
</div>
<span>
<div style="font-family:Calibri;font-size:12pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt">
</div>
</span></div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<span>
<div style="font-family:Calibri;font-size:12pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt">
<span style="font-weight:bold">From: </span>Angus Lees <<a href="mailto:gus@inodes.org" target="_blank">gus@inodes.org</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
</div>
</span></div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<span>
<div style="font-family:Calibri;font-size:12pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt">
<span style="font-weight:bold">Date: </span>Thursday 26 November 2015 at 03:23<br>
<span style="font-weight:bold">To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>>, Claudiu Belu <<a href="mailto:cbelu@cloudbasesolutions.com" target="_blank">cbelu@cloudbasesolutions.com</a>></div>
</span></div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<span>
<div style="font-family:Calibri;font-size:12pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt">
<br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows<br>
</div>
</span></div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<span>
<div><br>
</div>
<div>
<div>
<div dir="ltr">Thanks for the suggestion, and having a CI bot running on oslo.privsep would be a good thing once the basic code works on Windows - I would have expected it to be already running on all oslo projects (that the hyper-v hypervisor does/might depend
on) tbh.
<div><br>
</div>
<div>But that's a clumsy way to actually develop something. I _know_ privsep won't work currently on windows (imports pwd/grp for starters), and having to add print statements + submit + push-to-gerrit + wait for a CI bot + dig through CI bot logs + repeat
is a pretty terrible workflow ;-)</div>
<div><br>
</div>
<div>(I already have very little empathy for anyone choosing to run Windows just to provide yet-another-x86-hypervisor when far easier alternatives are available, and I won't donate a disproportionate amount of my time to a well-funded company who has historically
actively undermined the intellectual commons when _I_ have far more rewarding alternatives available. I'm afraid this has to be easy for me or I'm just not going to be able to sustain the effort required :-/ )</div>
</div>
</div>
</div>
</span></div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div><br>
</div>
<span>
<div>
<div>
<div dir="ltr">
<div><br>
</div>
<div> - Gus</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Thu, 26 Nov 2015 at 11:56 Alessandro Pilotti <<a href="mailto:apilotti@cloudbasesolutions.com" target="_blank">apilotti@cloudbasesolutions.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<div>
<div>
<div>Hi Angus,</div>
<div><br>
</div>
<div>First thanks for your concern on code portability! It still happens that we have to ask to revert patches on Oslo projects due to some Linux specific code that we discover only when the actual Oslo modules are used by Nova, Neutron, Cinder or other projects.
Typically a running Windows CI suddenly turns red on every patch and that’s how we get to find out. We’re working on adding many more projects under Windows CI tests, so at some point all the relevant Oslo ones will be covered and we’ll be able to prevent
those situations before the code gets merged.</div>
<div><br>
</div>
<div>What about preparing some basic integration tests for oslo.privsep that we could add to our Windows CI infrastructure? This would give you peace of mind during development on Linux without needing to test manually all your patches on Windows, knowing that
the Windows CI will give an error if the patch contains non portable code (or code that doesn’t behave as expected).</div>
<div><br>
</div>
</div>
</div>
<div>Alessandro</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<span>
<div style="font-family:Calibri;font-size:12pt;text-align:left;color:black;BORDER-BOTTOM:medium none;BORDER-LEFT:medium none;PADDING-BOTTOM:0in;PADDING-LEFT:0in;PADDING-RIGHT:0in;BORDER-TOP:#b5c4df 1pt solid;BORDER-RIGHT:medium none;PADDING-TOP:3pt">
<span style="font-weight:bold">From: </span>Angus Lees <<a href="mailto:gus@inodes.org" target="_blank">gus@inodes.org</a>><br>
<span style="font-weight:bold">Reply-To: </span>"OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Date: </span>Thursday 26 November 2015 at 01:56<br>
<span style="font-weight:bold">To: </span>Claudiu Belu <<a href="mailto:cbelu@cloudbasesolutions.com" target="_blank">cbelu@cloudbasesolutions.com</a>>, "OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [openstack-dev] [hyper-v] oslo.privsep vs Windows<br>
</div>
</span></div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<span>
<div><br>
</div>
<div>
<div>
<div dir="ltr">So I spent a day yesterday trying to get to the point where I could just run a no-change "tox" successfully on windows. Unfortunately I gave up when I realised I still had several days of downloading+building things ahead of me and clearly I
was doing it the hard way :(
<div><br>
</div>
<div>Could you point me to the "dev environment how-to" doc for hyper-v work? I'm thinking of something like <a href="http://docs.openstack.org/developer/nova/development.environment.html#setup" target="_blank">http://docs.openstack.org/developer/nova/development.environment.html#setup</a> with
simple cut+paste instructions for the totally-windows-clueless like me ;) Or perhaps a pre-prepared disk image, if the Microsoft license allows such a thing. In particular, the details on <a href="https://wiki.openstack.org/wiki/Hyper-V" target="_blank">https://wiki.openstack.org/wiki/Hyper-V</a> look
out of date (Grizzly, and several <a href="http://docs.openstack.org" target="_blank">
docs.openstack.org</a> links are broken), and the links to things like <a href="https://cloudbase.it/hyper-v-nova-compute-installer-unattended-setup/" target="_blank">https://cloudbase.it/hyper-v-nova-compute-installer-unattended-setup/</a> look like deployment
rather than development environments (?).</div>
<div><br>
</div>
<div>In particular, I think(?) I should be careful *not* to install too much of a cygwin-style environment, since I think(?) that might no longer be representative of the environment in which hyper-v is expected to operate. If I'm wrong, and cygwin/msys is
ok, then it looks like there are several free-software-on-windows "distributions" that would make life a whole lot simpler (by frankly being less like Windows). Some guidance from people who understand the issues here would be appreciated.</div>
<div><br>
</div>
<div> - Gus</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Tue, 24 Nov 2015 at 22:01 Claudiu Belu <<a href="mailto:cbelu@cloudbasesolutions.com" target="_blank">cbelu@cloudbasesolutions.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">Hello,<br>
<br>
Thanks Dims for raising the concern and Angus for reaching out. :)<br>
<br>
Most of the time, python development on Windows is not too far off from Linux. But the two systems are quite different, which imply different modules (fcntl, pwd, grp modules do not exist in Windows) or different implementations of some modules (multiprocessing
uses Popen instead of os.fork, os module is quite different) or some socket options and signals are different in Windows.<br>
<br>
1.a. As I've said earlier, some modules do not exist in Windows. All, or at least most standard modules document the fact that they are strictly for Linux. [1][2][3]<br>
b. At the very least, running the unit tests in a Windows environment can at least detect simple problems (e.g. imports). Secondly, there is a Hyper-V / Windows CI running on some of the OpenStack projects (nova, neutron, networking_hyperv, cinder) that can
be checked before merging.<br>
<br>
2. This is a bit more complicated question. Well, for functions, you could have separate modules for Linux specific functions and Windows specific functions. This has been done before: [4] As for object-oriented implementations, I'd suggest having the system-specific
calls be done in private methods, which will be overriden by Windows / Linux subclasses with their specific implementations. We've done something like this before, but solutions were pretty much straight-forward; it might not be as simple for oslo_privsep,
since it is very Linux-specific.<br>
<br>
3. Typically, the OpenStack services on Hyper-V / Windows are run with users that have enough privileges to do their job. For example, the nova-compute service is run with a user that has Hyper-V Admin privileges and is not necessarily in the "Administrator"
user group. We haven't used rootwrap in our usecases, it is disabled by default, plus, oslo.rootwrap imports pwd, which doesn't exist in Windows.<br>
<br>
[1] <a href="https://docs.python.org/2/library/fcntl.htm" target="_blank">https://docs.python.org/2/library/fcntl.htm</a>l<br>
[2] <a href="https://docs.python.org/2/library/pwd.html" target="_blank">https://docs.python.org/2/library/pwd.html</a><br>
[3] <a href="https://docs.python.org/2/library/grp.html" target="_blank">https://docs.python.org/2/library/grp.html</a><br>
[4] <a href="https://github.com/openstack/neutron/blob/master/neutron/agent/common/utils.py" target="_blank">
https://github.com/openstack/neutron/blob/master/neutron/agent/common/utils.py</a><br>
[5] <a href="https://github.com/openstack/oslo.rootwrap/blob/master/oslo_rootwrap/wrapper.py#L19" target="_blank">
https://github.com/openstack/oslo.rootwrap/blob/master/oslo_rootwrap/wrapper.py#L19</a><br>
<br>
If you have any further questions, feel free to ask. :)<br>
<br>
Best regards,<br>
Claudiu Belu<br>
<br>
<br>
<div style="font-family:Times New Roman;color:#000000;font-size:16px">
<hr>
<div style="direction:ltr"><font color="#000000" size="2" face="Tahoma"><b>From:</b> Angus Lees [<a href="mailto:gus@inodes.org" target="_blank">gus@inodes.org</a>]<br>
<b>Sent:</b> Tuesday, November 24, 2015 6:18 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions); Claudiu Belu<br>
<b>Subject:</b> [hyper-v] oslo.privsep vs Windows<br>
</font><br>
</div>
</div>
</div>
</div>
<div>
<div style="direction:ltr;font-family:Tahoma;color:#000000;font-size:10pt">
<div style="font-family:Times New Roman;color:#000000;font-size:16px">
<div></div>
<div>
<div dir="ltr">Dims has just raised[1] the excellent concern that oslo.privsep will need to at least survive on Windows, because hyper-v. I have no real experience coding on windows (I wrote a windows C program once, but I only ever ran it under wine ;) and
certainly none within an OpenStack/python context:
<div><br>
</div>
<div>1) How can I test whatever I'm working on to see if I have mistakenly introduced something Linux-specific? Surely this is a challenge common across every project in the nova/oslo/hyper-v stack.</div>
<div><br>
</div>
<div>2) What predicate should I use to guard the inevitable Linux-specific or Windows-specific code branches?</div>
<div><br>
</div>
<div>and I guess:</div>
<div>3) What does a typical OpenStack/hyper-v install even look like? Do we run rootwrap with some sudo-like-thing, or just run everything as the superuser?</div>
<div>What _should_ oslo.privsep do for this environment?</div>
<div><br>
</div>
<div>[1] <a href="https://review.openstack.org/#/c/244984/1" target="_blank">https://review.openstack.org/#/c/244984</a></div>
<div><br>
</div>
<div> - Gus</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</span></div>
<div style="word-wrap:break-word;color:rgb(0,0,0);font-size:14px;font-family:Calibri,sans-serif">
<span></span></div>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote>
</div>
</div>
</div>
</span></div>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote>
</div>
</div>
</div>
</span>
</body>
</html>