Open Stack

On Tuesday, 11 December 2018 23:42:40 CET Thomas Goirand wrote:
> Hi Luigi,
> 
> First thanks for communicating about this.
> 
> I'm afraid you probably will not like much what I'm going to have to
> write here. Sorry in advance if I'm not enthusiastic about this.
> 
> On 12/11/18 9:54 PM, Luigi Toscano wrote:
> > == Packaging
> > 
> > It would be nice if users could get all the plugins packages by installing
> > the same packages as before the split, but I understand that it may be
> > tricky, because each plugin depends on the core of sahara. This means
> > that a binary subpackage like openstack-sahara (RDO) or sahara (Debian)
> > can't depend on the binary packages of the plugins, or that would
> > introduce a circular dependency.
> > 
> > I raised the issue on the RDO list to get some initial feedback and the
> > solution that we are going to implement in RDO requires flipping the value
> > of a variable to handle the bootstrap case and the normal case, please
> > check:
> > https://lists.rdoproject.org/pipermail/dev/2018-November/008972.html> 
> > and the rest of the thread, like:
> >  https://lists.rdoproject.org/pipermail/dev/2018-December/008977.html
> > 
> > Any additional idea from other packagers would be really appreciated, also
> > because it has a direct impact on at least the puppet modules.
> 
> We aren't "packagers", we *maintain* packages, and therefore are package
> maintainers. And from that perspective, such decision from upstream is
> always disruptive, both from the package maintainer view, and for our users.

About the terminology, I'd like to point out that I've seen "packagers" used 
without any particular weird meaning attached in all the development 
communities I followed or I contributed to, even used by the people in charge 
of the packages, and I knew quite a lot of them.

Going to the proper point:

> While a plugin architecture looks sexy from the outside, it can bite
> hard in many ways.
> 
> In argumentation in the spec, the one and only thing which is annoying
> your users (according to the argumentation) is that plugins are tight to
> a milestone, and that it's not possible to upgrade a plug-in separately.
> I seriously doubt that Sahara users are going to do that, for many
> reasons. Here's 2 of them.
> 
> First, because if they consume packages, I don't see downstream
> distribution making update to packages so often. Especially if there's
> so many to update, and so many combination to test, it's going to be
> very difficult for downstream distribution to know what to do, and do
> regular plugin updates.

That's about people in charge of the packages, so me (partially for RDO), you, 
and other people.

> 
> Second, because I very much doubt that users will want to update their
> plugins that often. Most OpenStack users are struggling with updates of
> the OpenStack core already.

We didn't come out with this requirement outselves. We had various operators 
over the year who could not provide newer plugins to their users because they 
could not upgrade the whole of OpenStack and asked for this.
This feature is there exactly to not upgrade the core of OpenStack. Upgrading 
leaves components like the plugin is a totally different story.



> Another thing is experience we had with Cinder and Neutron. As you know,
> Cinder carries the drivers in the main repository, while Neutron decided
> to move to a plugin system. So we do trust the Cinder team to do a good
> job to maintain the plugins, and have them in sync with the release, for
> some Neutron plugins, it has been a complete disaster, with some plugins
> always lagging behind the needed API changes and so on. As for
> downstream distribution, it is very hard to know which plugins are worth
> packaging or not, and even harder to know which are well maintained
> upstream or not. What's your plan for addressing this issue?

By keeping a proper set of constraints in the plugins. As I mentioned, the 
core of Sahara is more than stable and most of the changes are in the plugins.
tl;dr we don't plan API changes in the interface between the core and the 
plugins. And we probably don't want them. At most we may add some of them, but 
not remove.


> As for myself, I'm really not sure I'm going to invest so much time on
> Sahara packaging plugin. Maybe I'll decide to package a few, but
> probably not all plugins, as it's very time consuming. Maybe I'll find
> spare cycles. It wont be important anyway, because Debian Buster will be
> frozen, so I may just skip that work for the Stein release, as I'll
> probably be busy on other stuff. So, not best timing for me as a Debian
> package maintainer. :/
> 
> So, maybe you could reconsider? Or is it already too late?

There is no going back.

> 
> > == Preview
> > 
> > The work-in-progress repositories are available here:
> > 
> > Sahara: https://github.com/tellesnobrega/sahara/tree/split-plugins
> > 
> > Plugins:
> > * Ambari: https://github.com/tellesnobrega/sahara-plugin-ambari
> > * CDH: https://github.com/tellesnobrega/sahara-plugin-cdh
> > * MapR: https://github.com/tellesnobrega/sahara-plugin-mapr
> > * Spark: https://github.com/tellesnobrega/sahara-plugin-spark
> > * Storm: https://github.com/tellesnobrega/sahara-plugin-storm
> > * Vanilla: https://github.com/tellesnobrega/sahara-plugin-vanilla
> 
> Oh gosh... This becomes very embarrassing. I have absolutely no clue
> what these plugins are for, and there's absolutely no clue in the
> repositories. Appart from "Spark", which I vaguely heard about, the
> above names give zero clue on what they do. There's no hyperlink to the
> projects these plugins are supporting. I have no clue which of the above
> plugins is for Hadoop, which was historically the first target of Sahara.

That's the meaning of "testing repository"; they are not final (they will need 
at least one or two more rebases from the current master). 
The description in setup.cfg can certainly be extended, and the related 
content of README.rst too. This is a valuable suggestion.

Ciao
-- 
Luigi