removing use of pkg_resources to improve command line app performance

Radosław Piliszek radoslaw.piliszek at gmail.com
Mon Jul 6 19:06:05 UTC 2020


On Mon, Jul 6, 2020 at 9:00 PM Sean Mooney <smooney at redhat.com> wrote:
>
> On Mon, 2020-07-06 at 14:37 -0400, Doug Hellmann wrote:
> > We have had a long-standing issue with the performance of the openstack command line tool. At least part of the
> > startup cost is the time taken in scanning for all of the plugins that are installed, which is a side-effect of
> > importing pkg_resources. To fix that, we need to eliminate all use of pkg_resources in code that would be used by a
> > command line application (long-running services are candidates, too, but the benefit is bigger in short-lived command
> > line apps).
> >
> > Python 3.8 added a new library importlib.metadata, which also has an entry points API. It is more efficient, and
> > produces data in a format that can be cached to make it even faster. I have started adding support for that caching to
> > stevedore [0], which is the Oslo library for managing application plugins. For version of python earlier than 3.8, the
> > same library is available on PyPI as “importlib_metadata”.
> based on https://opendev.org/openstack/governance/src/branch/master/reference/runtimes/victoria.rst we still need
> to support 3.6 for victoria. is there a backport lib like mock for this on older python releases?

Is [1] that Doug mentioned not what you mean?
It seems to support 3.5+

As a general remark, I've already seen the WIP. Very excited to see
this performance bottleneck eliminated.

[1] https://pypi.org/project/importlib-metadata/

-yoctozepto



More information about the openstack-discuss mailing list