Open Stack

On Tue, Nov 21, 2017 at 04:41:13PM +0000, Mooney, Sean K wrote:
> 
> 
> > -----Original Message-----
> > From: Jeremy Stanley [mailto:fungi at yuggoth.org]
> > Sent: Tuesday, November 21, 2017 3:05 PM
> > To: OpenStack Development Mailing List (not for usage questions)
> > <openstack-dev at lists.openstack.org>
> > Subject: Re: [openstack-dev] Removing internet access from unit test
> > gates
> > 
> > On 2017-11-21 09:28:20 +0100 (+0100), Thomas Goirand wrote:
> > [...]
> > > The only way that I see going forward, is having internet access
> > > removed from unit tests in the gate, or probably just the above
> > > variables set.
> > [...]
> > 
> > Historically, our projects hadn't done a great job of relegating their
> > "unit test" jobs to only run unit tests, and had a number of what would
> > be commonly considered functional tests mixed in. This has improved in
> > recent years as many of those projects have created separate functional
> > test jobs and are able to simplify their unit test jobs accordingly, so
> > this may be more feasible now than it was in the past.
> > 
> > Removing network access from the machines running these jobs won't
> > work, of course, because our job scheduling and execution service needs
> > to reach them over the Internet to start jobs, monitor progress and
> > collect results. As you noted, faking Python out with envvars pointing
> > it at nonexistent HTTP proxies might help at least where tests attempt
> > to make HTTP(S) connections to remote systems.
> > The Web is not all there is to the Internet however, so this wouldn't
> > do much to prevent use of remote DNS, NTP, SMTP or other
> > non-HTTP(S) protocols.
> > 
> > The biggest wrinkle I see in your "proxy" idea is that most of our
> > Python-based projects run their unit tests with tox, and it will use
> > pip to install project and job dependencies via HTTPS prior to starting
> > the test runner. As such, any proxy envvar setting would need to happen
> > within the scope of tox itself so that it will be able to set up the
> > virtualenv prior to configuring the proxy vars for the ensuing tests.
> > It might be easiest for you to work out the tox.ini modification on one
> > project (it'll be self-testing at least) and then once the pilot can be
> > shown working the conversation with the community becomes a little
> > easier.
> [Mooney, Sean K] I may be over simplifying here but our unit tests are still executed by
> Zuul in vms provided by nodepool. Could we simply take advantage of openstack and
> use security groups to to block egress traffic from the vm except that required to upload the logs?
> e.g. don't mess with tox or proxyies within the vms and insteand do this externally via neutron.
> This would require the cloud provider to expose neutron however which may be an issue for Rackspace but
> It its only for unit test which are relatively short lived vs tempest jobs perhaps the other providers would
> Still have enough capacity?
> > --
> > Jeremy Stanley
> 
I don't think we'd need to use security groups, we could just setup a local
firewall ruleset to do this on the node if we wanted.

-Paul