Open Stack

Hi,

On Wed, Apr 15, 2020 at 9:55 PM Ben Nemec <openstack at nemebean.com> wrote:

>
>
> On 4/7/20 5:15 AM, Dmitry Tantsur wrote:
> > Hi,
> >
> > On Mon, Apr 6, 2020 at 11:06 PM Ben Nemec <openstack at nemebean.com
> > <mailto:openstack at nemebean.com>> wrote:
> >
> >     At the risk of getting defensive, I do want to make some points
> >     relating
> >     to Oslo specifically here.
>
<snip>

> >
> >      > 1) oslo.config is a fantastic piece of software that the whole
> >     python
> >      > world could benefit from. Same for oslo.service, probably.
> >
> >     oslo.config is an interesting case because there is definitely
> interest
> >     outside OpenStack thanks to things like the Castellan config backend,
> >     but as Doug mentioned oslo.config is heavily opinionated (global
> config
> >     object, anyone?) and that's an issue for a lot of people. I will also
> >     point out that the only oslo.* dependency that oslo.config has is
> >     oslo.i18n, which itself has no other oslo dependencies, so there's
> not
> >     much barrier to entry to using it standalone today.
> >
> >
> > oslo.config currently pulls in 16 dependencies to a clean venv,
> > including oslo.18n, Babel (why?) and requests (WHY?).
>
> Well, it's got translated strings and thus i18n, Babel is used for lazy
> translation, and requests is used for the remote file driver. It's
> posssible the latter two could/should be optional deps, but without them
> you do lose functionality.
>

Please pardon my ignorance, it's the first time I hear about this driver?
How many people are using it in the wild? Will it hurt much if we treat
this dependencies similar to database/tooz backend, i.e. as optional?

More on Babel below.


>
> >
> >
> >     I would not inflict oslo.service on anyone I liked. :-P
> >
> >
> > So, you don't like us? :-P
>
> I don't _think_ I'm responsible for anyone using oslo.service. If I am,
> it happened years ago before I had dug into oslo.service much. ;-)
>
> >
> >
> >     Seriously though, I would advocate for using cotyledon if you're
> >     looking
> >     for a general purpose service library. It's not eventlet-based and
> >     provides a lot of the functionality of oslo.service, at least as I
> >     understand it. It was also written by an ex-Oslo contributor outside
> of
> >     OpenStack so maybe it's an interesting case study for this
> >     discussion. I
> >     don't know how much contribution it gets from other sources, but that
> >     would be good to look into.
> >
> >
> > Good to know, thanks!
> >
> >
> >      > 2) oslo.utils as a catch-all repository of utilities should IMO be
> >      > either moved to existing python projects or decomposed into small
> >      > generally reusable libraries (essentially, each sub-module could
> >     be a
> >      > library of its own). Same for oslo.concurrency.
> >
> >     oslo.concurrency has already been decomposed into general purpose
> code
> >     (fasteners) and OpenStack-specific, at least for the most part. I'm
> >     sure
> >     the split isn't perfect, but it's not like we haven't recognized the
> >     need for better concurrency libraries in Python as a whole. Note that
> >     fasteners also lives outside Oslo in another ex-Osloers personal
> repo.
> >     Again, I'm not sure whether that was beneficial or not but it might
> be
> >     worth reaching out to Mehdi and Josh to see how they feel about it.
> >
> >
> > I *think* we mostly use oslo.concurrency for its execute() wrapper,
> > which seems like it could be a very useful library on its own (can I
> > suggest better-execute as a name?).
>
> Certainly something we could explore. Note that quite a few projects
> also use interprocess locks from o.c, in which case they're implicitly
> picking up some standard config opts for setting things like lock_path.
> I don't know if that's the case for Ironic though.
>

We only use in-process locks.


>
> >
> >
> >     I would probably -2 any attempt to split oslo.utils. You'd end up
> >     with a
> >     bunch of single file libraries and a metric ****-ton of
> administrative
> >     overhead. It's bad enough managing 40 some repos as it is. Also,
> that's
> >     another library with minimal cross-dependencies with the rest of Oslo
> >     (just oslo.i18n again), which was an intentional design decision to
> >     make
> >     it relatively painless to pull in.
> >
> >
> > I guess I have a personal dislike of libraries without a simple goal
> > (I've been also fighting with various "utils" modules inside Ironic -
> > with only limited luck).
>
> I guess I should have noted that I agree "utils" and "misc" modules tend
> to be a code smell, but in this case we had two bad options available
> and had to pick one. IMHO we've done a reasonable job of keeping
> oslo.utils pretty minimal, but obviously I'm a bit biased. :-)
>

I guess I'm fine with that if we stop pulling in Babel :)

By the way, only oslo.i18n imports Babel directly. I'm proposing patches to
various projects that move it out of requirements.txt. If you see one -
please review.

E.g. a shameless ping for osc-lib folks: https://review.opendev.org/717737


>
> >
> > I do suspect that at least some of oslo.utils contents could find a new
> > home, some maybe even in the stdlib. I also don't want us to end up in
> > the leafpad situation, on the other hand.
>
> That's entirely possible. We kind of lost our stdlib guy when Victor
> left the Oslo team so there may be things we've overlooked that could
> have been upstreamed.
>
> >
> >
> >      > 3) I'm genuinely not sure why oslo.log and oslo.i18n exist and
> which
> >      > parts of their functionality cannot be moved upstream.
> >
> >     oslo.log basically provides convenience functions for OpenStack
> >     services, which is kind of what the oslo.* libraries should do. It
> >     provides built-in support for things like context objects, which are
> >     fairly domain-specific and would be difficult to generalize. It also
> >     provides a common set of configuration options so each project
> doesn't
> >     have to write their own. We don't need 20 different ways to enable
> >     debug
> >     logging. :-)
> >
> >     oslo.i18n is mostly for lazy translation, which apparently is a thing
> >     people still use even though the company pushing for it in the first
> >     place no longer cares. We've also had calls to remove it completely
> >     because it's finicky and tends to break things if the consuming
> >     projects
> >     don't follow the best practices with translated strings. So it's in a
> >     weird limbo state.
> >
> >
> > May I join these calls please? :) Will I break Zanata (assuming anybody
> > still translates ironic projects) if I switch to the upstream gettext?
>
> I don't think you'll break Zanata, I think you'll break anyone who was
> using lazy translation. I know there are users of it out there, but
> maybe we should look into getting a user survey question for Oslo that
> would give us an idea of how widely its used.
>

I meant changing ironic, which honestly doesn't have real translations any
more..


>
> >
> >
> >     I did talk with JP in Shanghai about possibly making it optional
> >     because
> >     it pulls in a fair amount of translation files which can bloat
> minimal
> >     containers. I looked into it briefly and I think it would be
> possible,
> >     although I don't remember a lot of details because nobody was really
> >     pushing for it anymore so it's hard to justify spending a bunch of
> time
> >     on it.
> >
> >
> > I'd be curious to hear more.
>
> Upon reflection, I think the thing we were looking to make optional was
> the Babel dep of oslo.i18n. It pulls in something like 6 MB of
> translation files


Oh, yeah. When building our ramdisk, we have to manually delete these files
to make it smaller.


> and is only used to look up the available languages
> for lazy translation. I think code changes were needed though, it wasn't
> as simple as moving Babel to optional. There's probably an argument that
> we shouldn't be making that call when lazy translation is disabled
> anyway though.
>

Is it something I can help with?

It seems that Babel is only used in get_available_languages. Can we make it
an optional dependency for projects that do use this call (ironic does not)?

Dmitry


>
> >
> > Dmitry
> >
> >
> >     TLDR: I think Oslo provides value both in and out of OpenStack
> >     (shocking, I know!). I'm sure the separation isn't perfect, but what
> is?
> >
> >     There are some projects that have been split out of it that might be
> >     interesting case studies for this discussion if anyone wants to
> follow
> >     up. Not it. ;-)
> >
> >     -Ben
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200416/fa9075f1/attachment-0001.html>