Open Stack

On Thu, Apr 16, 2020 at 10:50 AM Dmitry Tantsur <dtantsur at redhat.com> wrote:

> Hi,
>
> On Wed, Apr 15, 2020 at 9:55 PM Ben Nemec <openstack at nemebean.com> wrote:
>
>>
>>
>> On 4/7/20 5:15 AM, Dmitry Tantsur wrote:
>> > Hi,
>> >
>> > On Mon, Apr 6, 2020 at 11:06 PM Ben Nemec <openstack at nemebean.com
>> > <mailto:openstack at nemebean.com>> wrote:
>> >
>> >     At the risk of getting defensive, I do want to make some points
>> >     relating
>> >     to Oslo specifically here.
>>
> <snip>
>
>> >
>> >      > 1) oslo.config is a fantastic piece of software that the whole
>> >     python
>> >      > world could benefit from. Same for oslo.service, probably.
>> >
>> >     oslo.config is an interesting case because there is definitely
>> interest
>> >     outside OpenStack thanks to things like the Castellan config
>> backend,
>> >     but as Doug mentioned oslo.config is heavily opinionated (global
>> config
>> >     object, anyone?) and that's an issue for a lot of people. I will
>> also
>> >     point out that the only oslo.* dependency that oslo.config has is
>> >     oslo.i18n, which itself has no other oslo dependencies, so there's
>> not
>> >     much barrier to entry to using it standalone today.
>> >
>> >
>> > oslo.config currently pulls in 16 dependencies to a clean venv,
>> > including oslo.18n, Babel (why?) and requests (WHY?).
>>
>> Well, it's got translated strings and thus i18n, Babel is used for lazy
>> translation, and requests is used for the remote file driver. It's
>> posssible the latter two could/should be optional deps, but without them
>> you do lose functionality.
>>
>
> Please pardon my ignorance, it's the first time I hear about this driver?
> How many people are using it in the wild? Will it hurt much if we treat
> this dependencies similar to database/tooz backend, i.e. as optional?
>
> More on Babel below.
>
>
>>
>> >
>> >
>> >     I would not inflict oslo.service on anyone I liked. :-P
>> >
>> >
>> > So, you don't like us? :-P
>>
>> I don't _think_ I'm responsible for anyone using oslo.service. If I am,
>> it happened years ago before I had dug into oslo.service much. ;-)
>>
>> >
>> >
>> >     Seriously though, I would advocate for using cotyledon if you're
>> >     looking
>> >     for a general purpose service library. It's not eventlet-based and
>> >     provides a lot of the functionality of oslo.service, at least as I
>> >     understand it. It was also written by an ex-Oslo contributor
>> outside of
>> >     OpenStack so maybe it's an interesting case study for this
>> >     discussion. I
>> >     don't know how much contribution it gets from other sources, but
>> that
>> >     would be good to look into.
>> >
>> >
>> > Good to know, thanks!
>> >
>> >
>> >      > 2) oslo.utils as a catch-all repository of utilities should IMO
>> be
>> >      > either moved to existing python projects or decomposed into small
>> >      > generally reusable libraries (essentially, each sub-module could
>> >     be a
>> >      > library of its own). Same for oslo.concurrency.
>> >
>> >     oslo.concurrency has already been decomposed into general purpose
>> code
>> >     (fasteners) and OpenStack-specific, at least for the most part. I'm
>> >     sure
>> >     the split isn't perfect, but it's not like we haven't recognized the
>> >     need for better concurrency libraries in Python as a whole. Note
>> that
>> >     fasteners also lives outside Oslo in another ex-Osloers personal
>> repo.
>> >     Again, I'm not sure whether that was beneficial or not but it might
>> be
>> >     worth reaching out to Mehdi and Josh to see how they feel about it.
>> >
>> >
>> > I *think* we mostly use oslo.concurrency for its execute() wrapper,
>> > which seems like it could be a very useful library on its own (can I
>> > suggest better-execute as a name?).
>>
>> Certainly something we could explore. Note that quite a few projects
>> also use interprocess locks from o.c, in which case they're implicitly
>> picking up some standard config opts for setting things like lock_path.
>> I don't know if that's the case for Ironic though.
>>
>
> We only use in-process locks.
>
>
>>
>> >
>> >
>> >     I would probably -2 any attempt to split oslo.utils. You'd end up
>> >     with a
>> >     bunch of single file libraries and a metric ****-ton of
>> administrative
>> >     overhead. It's bad enough managing 40 some repos as it is. Also,
>> that's
>> >     another library with minimal cross-dependencies with the rest of
>> Oslo
>> >     (just oslo.i18n again), which was an intentional design decision to
>> >     make
>> >     it relatively painless to pull in.
>> >
>> >
>> > I guess I have a personal dislike of libraries without a simple goal
>> > (I've been also fighting with various "utils" modules inside Ironic -
>> > with only limited luck).
>>
>> I guess I should have noted that I agree "utils" and "misc" modules tend
>> to be a code smell, but in this case we had two bad options available
>> and had to pick one. IMHO we've done a reasonable job of keeping
>> oslo.utils pretty minimal, but obviously I'm a bit biased. :-)
>>
>
> I guess I'm fine with that if we stop pulling in Babel :)
>
> By the way, only oslo.i18n imports Babel directly. I'm proposing patches
> to various projects that move it out of requirements.txt. If you see one -
> please review.
>
> E.g. a shameless ping for osc-lib folks: https://review.opendev.org/717737
>
>
>>
>> >
>> > I do suspect that at least some of oslo.utils contents could find a new
>> > home, some maybe even in the stdlib. I also don't want us to end up in
>> > the leafpad situation, on the other hand.
>>
>> That's entirely possible. We kind of lost our stdlib guy when Victor
>> left the Oslo team so there may be things we've overlooked that could
>> have been upstreamed.
>>
>> >
>> >
>> >      > 3) I'm genuinely not sure why oslo.log and oslo.i18n exist and
>> which
>> >      > parts of their functionality cannot be moved upstream.
>> >
>> >     oslo.log basically provides convenience functions for OpenStack
>> >     services, which is kind of what the oslo.* libraries should do. It
>> >     provides built-in support for things like context objects, which are
>> >     fairly domain-specific and would be difficult to generalize. It also
>> >     provides a common set of configuration options so each project
>> doesn't
>> >     have to write their own. We don't need 20 different ways to enable
>> >     debug
>> >     logging. :-)
>> >
>> >     oslo.i18n is mostly for lazy translation, which apparently is a
>> thing
>> >     people still use even though the company pushing for it in the first
>> >     place no longer cares. We've also had calls to remove it completely
>> >     because it's finicky and tends to break things if the consuming
>> >     projects
>> >     don't follow the best practices with translated strings. So it's in
>> a
>> >     weird limbo state.
>> >
>> >
>> > May I join these calls please? :) Will I break Zanata (assuming anybody
>> > still translates ironic projects) if I switch to the upstream gettext?
>>
>> I don't think you'll break Zanata, I think you'll break anyone who was
>> using lazy translation. I know there are users of it out there, but
>> maybe we should look into getting a user survey question for Oslo that
>> would give us an idea of how widely its used.
>>
>
> I meant changing ironic, which honestly doesn't have real translations any
> more..
>
>
>>
>> >
>> >
>> >     I did talk with JP in Shanghai about possibly making it optional
>> >     because
>> >     it pulls in a fair amount of translation files which can bloat
>> minimal
>> >     containers. I looked into it briefly and I think it would be
>> possible,
>> >     although I don't remember a lot of details because nobody was really
>> >     pushing for it anymore so it's hard to justify spending a bunch of
>> time
>> >     on it.
>> >
>> >
>> > I'd be curious to hear more.
>>
>> Upon reflection, I think the thing we were looking to make optional was
>> the Babel dep of oslo.i18n. It pulls in something like 6 MB of
>> translation files
>
>
> Oh, yeah. When building our ramdisk, we have to manually delete these
> files to make it smaller.
>
>
>> and is only used to look up the available languages
>> for lazy translation. I think code changes were needed though, it wasn't
>> as simple as moving Babel to optional. There's probably an argument that
>> we shouldn't be making that call when lazy translation is disabled
>> anyway though.
>>
>
> Is it something I can help with?
>
> It seems that Babel is only used in get_available_languages. Can we make
> it an optional dependency for projects that do use this call (ironic does
> not)?
>

I've prepared a patch: https://review.opendev.org/#/c/720404/ comments are
welcome.
It seems that we need to update all projects using get_available_languages
(mostly "core" projects) to depend on Babel explicity.

Dmitry


>
> Dmitry
>
>
>>
>> >
>> > Dmitry
>> >
>> >
>> >     TLDR: I think Oslo provides value both in and out of OpenStack
>> >     (shocking, I know!). I'm sure the separation isn't perfect, but
>> what is?
>> >
>> >     There are some projects that have been split out of it that might be
>> >     interesting case studies for this discussion if anyone wants to
>> follow
>> >     up. Not it. ;-)
>> >
>> >     -Ben
>> >
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200416/413e0aed/attachment-0001.html>