[openstack-dev] [Security] [Bandit] Using multiprocessing/threading to speed up analysis

Ian Cordasco ian.cordasco at RACKSPACE.COM
Mon Jun 8 17:07:07 UTC 2015



On 6/8/15, 11:38, "Clark, Robert Graham" <robert.clark at hp.com> wrote:

>Interesting work,
>
>I guess my initial thought would be - does it need to be faster?

That depends on how we expect people to use Bandit. Keystone is using it
at their gate. I expect some people will want to run it locally before
sending a patch set. They'll probably be less likely to bother if it takes
too long.

That said, it's totally a user experience thing. Did Flake8 need to be
faster? Probably not. Have we received any complaints about it being
faster? No. The only problem we had has been output ordering, which I
alluded to below.

>Will this work make maintenance and the addition of features more
>difficult?

It hasn't made maintenance or new feature addition for Flake8 harder.

Everything is written as you would expect. There wasn't any change in
Flake8 to any of the checks or how they work. Flake8 did have to make a
few options mutually exclusive. For example, if you use pep8's diff
capabilities then multiprocessing is turned off by default. It's unlikely
that you'll need it either.

>-Rob
>
>
>On 08/06/2015 08:26, "Ian Cordasco" <ian.cordasco at RACKSPACE.COM> wrote:
>
>>Hey everyone,
>>
>>I drew up a blueprint
>>(https://blueprints.launchpad.net/bandit/+spec/use-threading-when-running
>>-
>>c
>>hecks) to add the ability to use multiprocessing (or threading) to
>>Bandit.
>>This essentially means that each "thread" will be fed a file and analyze
>>it and return the results. (A file will only ever be analyzed by one
>>thread.)
>>
>>This has lead to significant speed improvements in Flake8 when running
>>against a project like Nova and I think the same improvements could be
>>made to Bandit.
>>
>>I'd love some feedback on the following points:
>>
>>1. Should this be on by default?
>>
>>   Technically, this is backwards incompatible (unless we decide to order
>>the output before printing results) but since we're still in the 0.x
>>release series of Bandit, SemVer allows backwards incompatible releases.
>>I
>>don't know if we want to take advantage of that or not though.
>>
>>2. Is output ordering significant/important to people?
>>
>>3. If this is off by default, should the flag accept a special value,
>>e.g., 'auto', to tell Bandit to always use the number of CPUs present on
>>the machine?
>>
>>Cheers,
>>Ian
>>
>>_________________________________________________________________________
>>_
>>OpenStack Development Mailing List (not for usage questions)
>>Unsubscribe: 
>>OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



More information about the OpenStack-dev mailing list