On 2021-10-18 11:22:11 +0800 (+0800), 高瀚翔 wrote:

Skyline-apiserver is a pure Python code project, following the Python wheel packaging standard, using pip for installation, and the dependency management of the project using poetry[1]

Skyline-console uses npm for dependency management, development and testing. During the packaging and distribution process, webpack will be used to process the source code and dependent library code first, and output the packaged static resource files. These static resource files will be stored in an empty Python module[2].

[...]

GNU/Linux distributions like Debian are going to want to separately package the original source code for all of these Web components and their dependencies, and recreate them at the time the distro's binary packages are built. I believe the concerns are making it easy for them to find the source for all of it, and to attempt to use dependencies which these distributions already package in order to reduce their workload. Further, it helps to make sure the software is capable of using multiple versions of its dependencies when possible, because it's going to be installed into shared environments with other software which may have some of the same dependencies, so may need to be able to agree on common versions they all support.

On Wed, Oct 20, 2021 at 5:22 PM Thomas Goirand zigo@debian.org wrote:

On 10/18/21 2:18 PM, Jeremy Stanley wrote:

...

On 2021-10-18 11:22:11 +0800 (+0800), 高瀚翔 wrote:

...

Skyline-apiserver is a pure Python code project, following the Python wheel packaging standard, using pip for installation, and the dependency management of the project using poetry[1]

Skyline-console uses npm for dependency management, development and testing. During the packaging and distribution process, webpack will be used to process the source code and dependent library code first, and output the packaged static resource files. These static resource files will be stored in an empty Python module[2].

[...]

GNU/Linux distributions like Debian are going to want to separately package the original source code for all of these Web components and their dependencies, and recreate them at the time the distro's binary packages are built. I believe the concerns are making it easy for them to find the source for all of it, and to attempt to use dependencies which these distributions already package in order to reduce their workload. Further, it helps to make sure the software is capable of using multiple versions of its dependencies when possible, because it's going to be installed into shared environments with other software which may have some of the same dependencies, so may need to be able to agree on common versions they all support.

Hi,

Thanks Jeremy for summing-up things in a better way that I ever would.

Also, using pip is *not* an option for distros. I'm not sure what you mean by "following the Python wheel packaging standard", but to me, we're not there yet. I'd like to have a normal setup.py and setup.cfg file in each Python module so it's easy to call "python3 setup.py install --root $(pwd)/debian/skyline-apiserver --install-layout=deb".

Side note: calling setup.py is essentially deprecated: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

I would also expect to see a "normal" requirements.txt and test-requirements.txt like in every other OpenStack project, the use of stestr to run unit tests, and so on.

Right now, when looking at the skyline-apiserver as the Debian OpenStack package maintainer, I'd need a lot of manual work to use pyproject.toml instead of my standard tooling.

PyProject is the universal way forward, you'll (and we'll) need to adopt sooner or later.

Dmitry

As I understand, dependencies are expressed in the pyproject.toml, but then how do I get the Python code installed under debian/skyline-apiserver?

BTW, what made you choose something completely different than the rest of the OpenStack project?

Cheers,

Thomas Goirand (zigo)

On Fri, Oct 22, 2021, at 3:06 PM, Thomas Goirand wrote:

On 10/21/21 1:18 PM, Dmitry Tantsur wrote:

...

Side note: calling setup.py is essentially deprecated: https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

This is essentially bullshit: the replacement method proposed in this page is supposed to be "pip install", which does dependency resolving and download from the internet, which is not useful (and even forbidden at package build time) for downstream distributions.

If using pip is the only thing that upstream Python people are proposing to distributions, without any distro-specific options (like the --install-layout=deb option of setuptools, for example), then something really wrong is going on.

Unless I missed something and upstream Python likes us from now on?!?

I think https://pypa-build.readthedocs.io/en/latest/ may be the new tool that is a bit more usable for distros. But I've not used it at all so couldn't comment on its applicability.

---- On Fri, 22 Oct 2021 17:40:29 -0500 Jeremy Stanley fungi@yuggoth.org wrote ----

On 2021-10-23 00:06:02 +0200 (+0200), Thomas Goirand wrote: [...]

...

This is essentially bullshit: the replacement method proposed in this page is supposed to be "pip install", which does dependency resolving and download from the internet, which is not useful (and even forbidden at package build time) for downstream distributions.

If using pip is the only thing that upstream Python people are proposing to distributions, without any distro-specific options (like the --install-layout=deb option of setuptools, for example), then something really wrong is going on.

Unless I missed something and upstream Python likes us from now on?!?

[...]

Their new answer to "how should we be building binary packages from Python source distributions" is this (already packaged in Debian):

https://packages.debian.org/python3-build

The idea is that a source distribution may have a variety of different possible build backends per PEP 517, but as long as you run `python3 -m build` (and the relevant build dependencies are made available somehow) then it shouldn't matter. Calling directly into setup.py is only going to work for projects which use Setuptools as their build backend and provide a setup.py, of course, probably not for any other build backend.

...

IMO skyline needs to conform with everything that's made elsewhere in OpenStack, and adopt the same standards.

[...]

I agree, and I attempted to leave some detailed notes on https://review.opendev.org/814037 highlighting the differences I spotted, which I think they're going to want to address for consistency with the rest of OpenStack.

Hello Skyline team,

We have not heard from you on the open questions (mentioned in https://review.opendev.org/c/openstack/governance/+/814037 too). It's been ~ 21 days TC has voted -1 in Gerrit with required things to fix/or tell plan so that TC can merge the new project application.

Please respond on the Gerrit to proceed next.

-gmann

-- Jeremy Stanley

On Thu, 2021-12-09 at 16:57 +0100, Jean-Philippe Evrard wrote:

Hello folks,

The current governance requires the projects to follow certain guidelines [1], some of which are already covered by skyline. One of such guidelines is the PTI [2], which is detailed in more details for python in [3].

I am interested by the case of Skyline. Skyline is a truely new project for OpenStack, bringing value to the ecosystem. It is forward looking. Yes, it is different than existing projects. Yes, it doesn't pass the pti for python [3].

If you check the PTI for golang [4], many of those commands could be easily wired on Skyline project using a Makefile (of course it wouldn't pass the dependency management part of that document, for which we require glide [5]... unmaintained since 2019, ahem... [6]).

So the question is: Is OpenStack ready to move forward and adapt to new kinds of packaging/software? If yes, let's modify the PTIs (or python PTI) to cover the case for skyline, and allow that team to move forward. It's not like it's doing something very different than some new python projects.

using a make file to me woudl be alarge regression in packaging and it does not align with the upstream recomendation for package form the python packaging athority. i support using projects.toml and the new standardised build system and i woudl be open to changeing the pti to allow that but  i dont think tha tallow make files would be a good thing. the use of poetry is also an interesting choice that has some advandtges but it will break much of our tooling with regards to upper-constraints.txt.

i woudl personally find it rather suprising as a downstream packageer or end user to consume skyline as currently wtieen as either a standar python proejct or an openstack one if i had to use make to build or install it.

i am happy to see that tox is used but since its callign dirctly into make i really dont think it being used the way it is intended to be.

Yes, OpenStack is mature... I (simply) hope it's also a place for casual contributions, and where developers can have fun with tech. And yes, it won't please everyone from day one (especially distros), but I hope we can work this out together, rather than discard the project because it doesn't fit the mold.

is there a reaosbon bveyond its fun to do it this way.  to me this add barries to contibutors using the proejct and to packagees consuming it.

Regards, Jean-Philippe Evrard (evrardjp)

Helllo,

On Thu, Dec 9, 2021, at 19:03, Jeremy Stanley wrote:

The reason I raised those points on the review was in order to highlight inconsistencies with OpenStack as a whole. If OpenStack is defined by anything, it's consistent development workflows and automation, which allow small groups of people to take advantage of economies of scale in order to minimize maintenance over time across the whole of the set of projects which comprise it. Inconsistency has an inherent cost which the entire community bears, from tracking integration failures in development to keeping concise recipes for packaging in distributions to operators being able to configure all services the same way and expect the same logging formats.

I agree that consistency is important, and helps us in many ways. I don't think it's the "defining" factor of OpenStack, however. Inconsistency in openstack is in many places, from the users' to the developer's perspective. It doesn't mean I am _looking for_ inconsistency. I am merely embracing the differences (and I am eager to remove them, if we can).

Most of the benefits a project gets from being "part of OpenStack" are derived directly from the established standards and norms which enable OpenStack to be treated as a cohesive whole rather than a mere "bucket of parts" with tooling choices made independently by their respective development teams. Deviation from these expectations should be deliberate and come with a clear rationale which benefits OpenStack as a whole rather than one project alone.

Correct again! :)

For the case of skyline, one could made a case to start building packages with a new way, using poetry, rather than setuptools. It could mean we need to ask the skyline team to help on improving the infrastructure and testing. Instead of arriving to a standard testing, we embrace the difference and provide a possible new way of working. We could have two kinds of python projects. Instead of accepting that difference when a single project asking for it, maybe we can just tally who wanted this, and the TC decides at the end "when" it is worth doing.

My whole point is that we need to document better and agree which parts benefits OpenStack as a whole. We should also have indicators for checking how the "ecosystem" is behaving. If we are considering the amount of new teams in OpenStack, it's not very encouraging. (It might not be a metric, I don't know, I rather leave Thierry judge there).

In other words: Is it really better to change the skyline team to adapt our mold, or should we have multiple molds? We agreed to go the way that skyline should adapt. I am simply raising the fact that some other people/projects won't do the effort to adapt to existing mold, because they won't necessarily see the value. (I won't prevent my teams to work on non-openstack projects ;))

Let's take the current example point by point:

• Skyline implements its own routines for things like configuration and logging rather than using Oslo libraries. Should all OpenStack projects "move forward" to this model, choose their own configuration and logging formats, implement their own solutions for documenting these, and so on? Does inconsistency here benefit operators in some way?

Is Oslo a requirement to be an OpenStack project now? AFAIK it wasn't (see swift).

It doesn't mean you can't be consistent for logs/configuration from the user perspective by not using oslo. Side note: Bringing oslo projects for configuration and logging isn't enough to bring consistency from the user perspective.

• Skyline defines the interdependencies between its components with a mix of "monorepo" (multiple packages from a single repository) and Git submodule references (pinning specific commit ids as virtual subtrees of another repository), rather than establishing a loose coupling with explicit version numbers and package relationships. This choice prevents them from taking advantage of features in Zuul like cross-project change dependencies. Is this a pattern the other OpenStack projects should follow?

Which is why I am highlighting that _as of today_, the governance requires certain expectations, which needs clarifications (or exceptions) if we want to integrate Skyline.

I know I am a bit pedantic in the following question: Is (nowadays) the expectation for projects to be testable by Zuul (+release-able by zuul, ... list to be defined) or is the expectation for projects to be following the PTI guidelines (tested by zuul)?

In other words: Do we need to enforce the tooling or the results (a frequently released and well tested software)? I would like the TC to think about this.

Does it benefit distribution package maintainers to need to track fundamentally different repository layouts and build processes for each of them?

Isn't that exactly what the distros are doing right now with the packages they are maintaining? Not every software is like openstack. Yes, almost all openstack projects are similar to each other, so they can be packaged the same way.

Yet, some official openstack projects aren't, as of today. So, some distros decide to not package them? Is that a problem for us? I didn't think so. Projects/Teams need to understand that if they don't follow "x", they will not have "y". Which sounds fair to me.

Keep in mind that we are _already doing exceptions_. Not all projects are similar to nova, glance, neutron, keystone, cinder. (Sorry for the many I missed).

• Skyline performs its release versioning by committing the version strings to files in its repositories rather than assigning them at build time from Git tag metadata. This leads to ambiguity as to what exact state of the repository represents that version, as well as opening up the possibility of forgetting to merge the version update before tagging (a very common problem which drove OpenStack to rely on its current model). Wholly different tools are also needed in order to track versions for these projects as a result. Should the rest of OpenStack's projects follow suit there? Has something changed to make the idea of using Git to signal releases a bad one?

You're raising again a valid point, for which the skyline team will have to find a solution.

For me, it boils down to defining that projects should follow a "project release interface", which explains the expected behaviour in regards of releasing for projects. We don't have such. I am not claiming we should, I am merely asking us to think about what we are doing.

Projects could indeed move to use pbr/setuptools. However, I am not sure many developers understand why. Should we simply explain what is expected in terms of building the version from git? People might even come with interesting solutions. Promoting creativity will create tech debt, but might also reduce it over time. What if we don't need pbr anymore? Will we even notice?

• Skyline uses makefiles to script its build and test workflows rather than tox, adding a tox wrapper around the make targets in order to nominally comply with the OpenStack PTI. As a result, it's unable to rely on most of tox's features for environment isolation, package management, path overrides, et cetera. OpenStack projects have solved a lot of cross-project development workflow challenges through applying consistent changes across the tox.ini files of repositories, should those solutions be abandoned and recreated for make instead?

I didn't say so. I am only partially agreeing with you there, for once :)

For me, tox is a test runner that provides many nice features thanks to its code (and integration with setuptools) I would prefer if python projects using tox for testing interface indeed. Should that be the standard? I don't know. Does it make sense to rely on tox features for isolations/package management/other? I don't know. You might be aware that it's possible to integrate tox with poetry. Does it provide all we _need_? I don't know: We aren't explicit in the requirements of what we need, we just talk about tools.

If I were to rephrase: Is it easier to fit the existing mold or clarify what we need, why, and how we can deal with the tech debt? Probably the former. Should we do it? I don't know. The TC should however evaluate this seriously.

• Skyline does not put its external Python package dependencies in central lists nor use tools which would allow it to rely on OpenStack's global constraints implementation, making it very hard for them to guarantee that their software can run with the same versions of shared dependencies as the rest of OpenStack. Is this unnecessary? Have we given up on the idea that two pieces of OpenStack software can be coinstalled into one environment, or that Linux distributions won't be stuck having to package many different versions of the same software because OpenStack projects don't agree on which versions work for them any longer?

Now, we are hitting the nail, IMO :)

This is a bit of the issue's core, isn't it? I think that's what we need to document. While I (personally) don't believe it's important nowadays (cf. containers), I am sure distros will disagree with me. It's important to know who we are pleasing, as we can't please everyone ;) Let's be clear about the audience.

If we intend to please the distros, did we ask about RedHat/Canonical committments on skyline? Are they even remotely interested? If they want to use it, wouldn't it make sense they do the work to make sure skyline is packageable for them, by improving upstream? If they don't, why do we care?

To be perfectly clear, I don't oppose accepting Skyline as a project in OpenStack, even in its current state, but I do feel like it's not going to end up released as part of OpenStack unless it can address the challenges I outlined above. Most of these are fairly superficial changes to implement, however, and can be done by simply copying the solutions already used in other OpenStack projects or retroactively applying our cookie-cutters.

I agree with you. I am merely asking for clarity. However, I want to think as a whole.

"Moving OpenStack forward" to be more like Skyline, on the other hand, is bound to be far more work than the Skyline team can handle on its own and take far longer, though I'm not opposed to the idea of discussing which of these differences may make sense for OpenStack to adopt in the future.

Depends on the definition of "Moving OpenStack forward". Do you mean all modifying all projects to be like skyline + integrate the testing? That was never my point. Do you mean "ensure a single project can be behaving like a majority of other big openstack projects are"? That's indeed more work for a team, compared to if they had their work done outside OpenStack governance.

Should we use this energy for thinking(/doing something) about OpenStack future? IMO, absolutely.

And for those arriving here: Thanks for reading my rant until the end ;) Jean-Philippe Evrard (evrardjp)

On 2021-12-10 10:43:10 +0100 (+0100), Jean-Philippe Evrard wrote:

On Thu, Dec 9, 2021, at 19:03, Jeremy Stanley wrote:

[...]

...

• Skyline performs its release versioning by committing the version strings to files in its repositories rather than assigning them at build time from Git tag metadata. This leads to ambiguity as to what exact state of the repository represents that version, as well as opening up the possibility of forgetting to merge the version update before tagging (a very common problem which drove OpenStack to rely on its current model). Wholly different tools are also needed in order to track versions for these projects as a result. Should the rest of OpenStack's projects follow suit there? Has something changed to make the idea of using Git to signal releases a bad one?

You're raising again a valid point, for which the skyline team will have to find a solution.

For me, it boils down to defining that projects should follow a "project release interface", which explains the expected behaviour in regards of releasing for projects. We don't have such. I am not claiming we should, I am merely asking us to think about what we are doing.

Projects could indeed move to use pbr/setuptools. However, I am not sure many developers understand why. Should we simply explain what is expected in terms of building the version from git? People might even come with interesting solutions. Promoting creativity will create tech debt, but might also reduce it over time. What if we don't need pbr anymore? Will we even notice?

Isn't that exactly what I did? You'll notice here and in my review I didn't say "use PBR," I outlined the properties desired instead. It's the case that using PBR is likely the easiest way to do that since we already have a lot of examples, but like you I don't think that the implementation is important as long as we can find a suitable solution to the problems we face.

That said, using a consistent implementation across as many projects as possible increases our familiarity and ability to address new issues with the fewest set of solutions.

...

• Skyline uses makefiles to script its build and test workflows rather than tox, adding a tox wrapper around the make targets in order to nominally comply with the OpenStack PTI. As a result, it's unable to rely on most of tox's features for environment isolation, package management, path overrides, et cetera. OpenStack projects have solved a lot of cross-project development workflow challenges through applying consistent changes across the tox.ini files of repositories, should those solutions be abandoned and recreated for make instead?

I didn't say so. I am only partially agreeing with you there, for once :)

For me, tox is a test runner that provides many nice features thanks to its code (and integration with setuptools) I would prefer if python projects using tox for testing interface indeed. Should that be the standard? I don't know. Does it make sense to rely on tox features for isolations/package management/other? I don't know. You might be aware that it's possible to integrate tox with poetry. Does it provide all we _need_? I don't know: We aren't explicit in the requirements of what we need, we just talk about tools.

If I were to rephrase: Is it easier to fit the existing mold or clarify what we need, why, and how we can deal with the tech debt? Probably the former. Should we do it? I don't know. The TC should however evaluate this seriously.

Let's be clear, I have nothing against make. In fact, I somewhat prefer it, since it's useful for projects written in all sorts of different programming languages beyond Python. At one point we even came very close to recommending OpenStack switch from tox to make during one of the more unfortunate backward-incompatible tox releases some years ago. Clark had a proof of concept recreating the fundamentals of tox itself in Makefiles, and it wasn't half bad.

But having one project use make while the rest use tox isn't really great for anyone, it turns the make-based project into a second-class citizen within the whole. Its maintainers have to do a lot more work to support the same workflows with a different tool, and the community as a whole incurs some additional burden in working around that difference (additional CI jobs, separate functionality, different developer documentation). If we think make is a superior solution we should use it across all of OpenStack and not just in a few random projects.

...

• Skyline does not put its external Python package dependencies in central lists nor use tools which would allow it to rely on OpenStack's global constraints implementation, making it very hard for them to guarantee that their software can run with the same versions of shared dependencies as the rest of OpenStack. Is this unnecessary? Have we given up on the idea that two pieces of OpenStack software can be coinstalled into one environment, or that Linux distributions won't be stuck having to package many different versions of the same software because OpenStack projects don't agree on which versions work for them any longer?

Now, we are hitting the nail, IMO :)

This is a bit of the issue's core, isn't it? I think that's what we need to document. While I (personally) don't believe it's important nowadays (cf. containers), I am sure distros will disagree with me. It's important to know who we are pleasing, as we can't please everyone ;) Let's be clear about the audience.

[...]

Just my opinion, but I think buying into container hype to the point where we make OpenStack only deployable through containerization would be a massive step backwards.

On 12/10/21 5:29 PM, Jean-Philippe Evrard wrote:

Totally agree. I have so many points I want to raise. I realise asking what I have in mind will cause more trouble on this mailing list than the value it will bring. I just want the TC to think broader than the "we used to do it this way, and this is the only way".

Willing to change things within OpenStack is a completely different topic than "doing things my own way". I very welcome change if you have a cross-project attitude but I don't welcome "we don't do like the others, because we can" type of thinking.

Packaging Telemetry, I've hated many times the design decision that was just a little bit different. It wasn't better or worse, just different. And it caused a lot of troubles just because I had to learn "their way" of doing things. I suppose the experience was the same for contributors.

For example, aodh has it's build-dependency (well, "test requirements" if you prefer the Python wording than the distro ones) in setup.cfg instead of test-requirements.txt. Both works, but when I upgrade any component, I diff the test-requirements.txt between version, to see if there's new Build-Depends: to add to my package. But with Aodh, I have to remember to do it with setup.cfg. Its *VERY* annoying. Yes, I'm aware that this way, Aodh can have different test-requirements for mysql and postgresql for example. But this is still very annoying. I would have very much prefer if the Telemetry guys pushed so that *ALL* projects would move to push test-requirements.txt within setup.cfg, and *THEN* I would have accepted the change.

Do you see where I'm going? :) I have other examples in mind, but I believe what's above is enough.

I am sure the topic raised some questions, and will give enough food for thoughts. Hence, I hope it will result in the improvement of OpenStack :)

If you don't do things on your own different than the others, and push every projects to move one direction at the same time then it's going to be an improvement. If you are the only one doing your own way, it's going to be a regression, and a pain for everyone.

...

Just my opinion, but I think buying into container hype to the point where we make OpenStack only deployable through containerization would be a massive step backwards.

I didn't say it was a goal or if I consider this important. I just wanted to mention that they are other ways to consume software than distribution packages.

Probably, but it's not up to you to decide for the others how they want to consume your deliverables.

Side note: I have spent to much time on this, so I will just step out now. I hope the TC got enough details about what to make out of this in the future. If you need an extra $0.02, you know where to find me :)

Regards, Jean-Philippe Evrard (evrardjp)

I very much hope that Skyline will be in a super nice shape in 6 months from now, so I can include in in Debian Bookworm! :)

By the way, we discussed Python packaging a lot, but never JS. There's a huge discussing that sooner or later will have to get started too. NPM is a very dangerous thing that often leads to "npm install world" (ie: a JS orgy that is barely maintainable). I have to admit that I didn't look closely into how that's done in Skyline. But let's just take something else as the worst example: Grafana. I've heard someone from Red Hat attempted to package it, and discovered it included (directly or indirectly) 1600 JS packages, and then gave-up. How is Skyline in this regard?

Cheers,

Thomas Goirand (zigo)

Hello,

On Fri, Dec 10, 2021, at 01:18, Ghanshyam Mann wrote:

Yes, those are the inconsistency we currently have in the Skyline project, but that is not what they will stick to. In Yoga PTG[1], the skyline team iterates through all these points and plans to improve those and be consistent with the other OpenStack projects. They said they will work on packaging, PTI, using Oslo code etc.

Boxiang Zhu mentioned the same in the governance patch also[2].

The questions I have still stand: Do we need to be more lax in the governance to not _require_ to follow the PTI (and/or have an intention to follow through)? Can the TC make an exception? When is the deadline for projects not following our testing to become compliant? What is the risk if the newly accepted project is not doing it? Moving it to a non-official project? Is that worth the effort?

If they deny moving Skyline towards the OpenStack way of packing, testing, using Oslo, etc., we should consider whether to reject the OpenStack official project or modify our new project application criteria. But this is not the case, and they agree to improve all those things.

I agree. Let's not talk hypothetics.

Now we have more clear direction from them in the governance patch also, so I feel this is good to go as an official project, and while they will be in OpenStack, we all can help them to improve the things in OpenStack way.

Perfect, great news.

Also, TC discussed in PTG the new concept of 'Tech pre-review' where we will monitor such new projects' health, contribution, and direction going in the direction of OpenStack way. This 'Tech pre-review' is not up yet, so let's see how we define it.

I am sorry I have missed that from the notes I read.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2021-October/025541.h... [2] https://review.opendev.org/c/openstack/governance/+/814037 [3] https://etherpad.opendev.org/p/tc-yoga-ptg#L263

-gmann

...

Regards, Jean-Philippe Evrard (evrardjp)

[1]:

https://governance.openstack.org/tc/reference/new-projects-requirements.html

...

[2]:

https://governance.openstack.org/tc/reference/project-testing-interface.html

...

On Fri, 2021-12-10 at 10:50 +0100, Thomas Goirand wrote:

On 12/10/21 9:16 AM, Jean-Philippe Evrard wrote:

...

Hello,

On Fri, Dec 10, 2021, at 01:18, Ghanshyam Mann wrote:

...

Yes, those are the inconsistency we currently have in the Skyline project, but that is not what they will stick to. In Yoga PTG[1], the skyline team iterates through all these points and plans to improve those and be consistent with the other OpenStack projects. They said they will work on packaging, PTI, using Oslo code etc.

Boxiang Zhu mentioned the same in the governance patch also[2].

The questions I have still stand: Do we need to be more lax in the governance to not _require_ to follow the PTI (and/or have an intention to follow through)? Can the TC make an exception?

IMO, we shouldn't allow an exception. Otherwise, it's going to become super hard to contribute to OpenStack if every project is different. It is a pandora box that should not be opened.

i would agree with this we should not make an excption however we could work on evolving the PTI over time.

That being said, I don't think poetry is a problem for Debian anymore: it's been packaged already. :)

poetry i see as less of a porblem then using make honestly. as someone who has avoided using make for most of the last decade moderatly succesfully that is perhaps the most objectionable part of its build system.

one concern i would have with using poetry is just makeing sure that upper-constratints.txt is used correctly based on https://github.com/python-poetry/poetry/pull/4005 and https://github.com/python-poetry/poetry/issues/3225 i dont think that poetry will ever actuly support upper-constarits so we would need to have some tooling to generate the equivalent for there format. i still think coinstallablity is an important requirement for being an offical python porject and moving to poetry or other packaging may not support that.

i recently have been workign on a new python project which i was considering proposing to opendev/openstack at some point in the future i used https://pyscaffold.org/ to generate the skeloton of the project and do have a pyproject.toml https://github.com/SeanMooney/arbiterd/blob/master/pyproject.toml but have intentioally choose to continue to use setuptools so that it would be simple to support the pti in the future i think the main delta i have right now is i dont have my deps in requirement.txt but i can simply generate that in the future https://github.com/SeanMooney/arbiterd/blob/master/setup.cfg#L49-L77 that is just because pyscaffold put them in setup.cfg and i havent bothered to move them.

i likely would have to add pbr for ChangeLog and AUTHORS generation and some minor other tweaks but you can have a pretty moddern an painless setup and be pti compliant at the same time.

Cheers,

Thomas Goirand (zigo)

---- On Fri, 10 Dec 2021 02:16:59 -0600 Jean-Philippe Evrard openstack@a.spamming.party wrote ----

Hello,

On Fri, Dec 10, 2021, at 01:18, Ghanshyam Mann wrote:

...

Yes, those are the inconsistency we currently have in the Skyline project, but that is not what they will stick to. In Yoga PTG[1], the skyline team iterates through all these points and plans to improve those and be consistent with the other OpenStack projects. They said they will work on packaging, PTI, using Oslo code etc.

Boxiang Zhu mentioned the same in the governance patch also[2].

The questions I have still stand: Do we need to be more lax in the governance to not _require_ to follow the PTI (and/or have an intention to follow through)? Can the TC make an exception? When is the deadline for projects not following our testing to become compliant? What is the risk if the newly accepted project is not doing it? Moving it to a non-official project? Is that worth the effort?

Those are all good points. We discussed around similar one in PTG and even considering if we should introduce different level of the official project like CNCF (Sandbox, graduate) or our old model or integrated vs incubated so that we can make it clear what projects are in-progress of becoming the OpenStack way and which are already. But these add more complications than solve the problem.

We agreed to try a new concept with 'Tech Preview' to monitor the project health as well as the OpenStack consistency. Let's wait for that to be up and there we can review/improve the process. Or PTI change if needed. We will also define what is the deadline for projects to be in 'Tech Preview process.

Let's wait for the Skyline team to improve the things which hey already mentioned to do and I think that can be done in a better way if they are in OpenStack community. If they do not then it can be dropped from OpenStack governance is nothing different from any existing projects do that.

-gmann

...

If they deny moving Skyline towards the OpenStack way of packing, testing, using Oslo, etc., we should consider whether to reject the OpenStack official project or modify our new project application criteria. But this is not the case, and they agree to improve all those things.

I agree. Let's not talk hypothetics.

...

Now we have more clear direction from them in the governance patch also, so I feel this is good to go as an official project, and while they will be in OpenStack, we all can help them to improve the things in OpenStack way.

Perfect, great news.

...

Also, TC discussed in PTG the new concept of 'Tech pre-review' where we will monitor such new projects' health, contribution, and direction going in the direction of OpenStack way. This 'Tech pre-review' is not up yet, so let's see how we define it.

I am sorry I have missed that from the notes I read.

...

[1] http://lists.openstack.org/pipermail/openstack-discuss/2021-October/025541.h... [2] https://review.opendev.org/c/openstack/governance/+/814037 [3] https://etherpad.opendev.org/p/tc-yoga-ptg#L263

-gmann

...

Regards, Jean-Philippe Evrard (evrardjp)

[1]:

https://governance.openstack.org/tc/reference/new-projects-requirements.html

...

[2]:

https://governance.openstack.org/tc/reference/project-testing-interface.html

...

---- On Fri, 10 Dec 2021 11:32:32 -0600 Thierry Carrez thierry@openstack.org wrote ----

Ghanshyam Mann wrote:

...

[...] Yes, those are the inconsistency we currently have in the Skyline project, but that is not what they will stick to. In Yoga PTG[1], the skyline team iterates through all these points and plans to > improve those and be consistent with the other OpenStack projects.

They said they will> work on packaging, PTI, using Oslo code etc. Ultimately there is a chicken and egg situation there -- they need to work on making it more like OpenStack, but it's difficult to justify the time to work on this until it actually is accepted as an OpenStack project.

We should communicate expectations clearly, make sure that promises are made, and then there is a leap of faith. But IMHO it is a very small leap as the TC can still control if it should be included in the next release, and the TC can also remove the project team if it never gets where it needs to be.

...

Boxiang Zhu mentioned the same in the governance patch also[2].

If they deny moving Skyline towards the OpenStack way of packing, testing, using Oslo, etc., we should consider whether to reject the OpenStack official project or modify our new project application criteria. But this is not the case, and they agree to improve all those things.

Now we have more clear direction from them in the governance patch also, so I feel this is good to go as an official project, and while they will be in OpenStack, we all can help them to improve the things in OpenStack way.

I agree. Expectations have been clearly communicated. It's time for us to give them a good chance to meet those.

Indeed.

Also, I think we are making assumptions too early (I mentioned in this thread), if Skyline team deny doing OpenStack way then it is better to discuss all these points otherwise, we should just wait for them to improve the things and even better is that we (who need this project or think it is an interested one) should be part of their team and improve things.

-gmann

-- Thierry Carrez (ttx)