March 2019 - openstack-discuss - lists.openstack.org

Re: [openstack-dev] Are we ready to put stable/ocata into extended maintenance mode?
by Alex Schultz 03 Mar '21

03 Mar '21

On Tue, Sep 18, 2018 at 1:30 PM Alex Schultz <aschultz(a)redhat.com> wrote: > > On Tue, Sep 18, 2018 at 1:27 PM, Matt Riedemann <mriedemos(a)gmail.com> wrote: > > The release page says Ocata is planned to go into extended maintenance mode > > on Aug 27 [1]. There really isn't much to this except it means we don't do > > releases for Ocata anymore [2]. There is a caveat that project teams that do > > not wish to maintain stable/ocata after this point can immediately end of > > life the branch for their project [3]. We can still run CI using tags, e.g. > > if keystone goes ocata-eol, devstack on stable/ocata can still continue to > > install from stable/ocata for nova and the ocata-eol tag for keystone. > > Having said that, if there is no undue burden on the project team keeping > > the lights on for stable/ocata, I would recommend not tagging the > > stable/ocata branch end of life at this point. > > > > So, questions that need answering are: > > > > 1. Should we cut a final release for projects with stable/ocata branches > > before going into extended maintenance mode? I tend to think "yes" to flush > > the queue of backports. In fact, [3] doesn't mention it, but the resolution > > said we'd tag the branch [4] to indicate it has entered the EM phase. > > > > 2. Are there any projects that would want to skip EM and go directly to EOL > > (yes this feels like a Monopoly question)? > > > > I believe TripleO would like to EOL instead of EM for Ocata as > indicated by the thead > http://lists.openstack.org/pipermail/openstack-dev/2018-September/134671.ht… > Bringing this backup to see what we need to do to get the stable/ocata branches ended for the TripleO projects. I'm bringing this up because we have https://review.openstack.org/#/c/647009/ which is for the upcoming rename but CI is broken and we have no interest in continue to keep the stable/ocata branches alive (or fix ci for them). Thanks, -Alex > Thanks, > -Alex > > > [1] https://releases.openstack.org/ > > [2] > > https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… > > [3] > > https://docs.openstack.org/project-team-guide/stable-branches.html#extended… > > [4] > > https://governance.openstack.org/tc/resolutions/20180301-stable-branch-eol.… > > > > -- > > > > Thanks, > > > > Matt > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

5 11

[openstack-ansible] dropping suse support
by Mohammed Naser 06 May '20

06 May '20

Hi everyone, I've been trying to avoid writing this email for the longest time ever. It's really painful to have to reach this state, but I think we've hit a point where we can no longer maintain support for SUSE within OpenStack Ansible. Unfortunately, with the recent layoffs, the OSA team has taken a huge hit in terms of contributors which means that there is far less contributors within the project. This means that we have less resources to go around and it forces us to focus on a more functional, reliable and well tested set of scenarios. Over the past few cycles, there has been effort to add SUSE support to OpenStack Ansible, during the time that we had maintainers, it was great and SUSE issues were being fixed promptly. In addition, due to the larger team at the time, we found ourselves having some extra time where we can help unbreak other gates. Jesse used to call this a "labour of love", which I admired at the time and hoped we continue to do as much as we can of. However, the lack of a committed maintainer for OpenSUSE has resulted in constantly failing jobs[1][2] (which were moved to non-voting, wasting CI resources as no one fixed them). In addition, it's causing several gate blocks for a few times with no one really finding the time to clean them up. We are resource constrained at this point and we need the resource to go towards making the small subset of supported features functional (i.e. CentOS/Ubuntu). We struggle with that enough, and there seems to be no deployers that are running SUSE in real life at the moment based on bugs submitted. With that, I propose that we drop SUSE support this cycle. If anyone would like to volunteer to maintain it, we can review that option, but that would require a serious commitment as we've had maintainers step off and it hurts the velocity of the project as no one can merge code anymore. ..Really wish I didn't have to write this email Mohammed [1]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct… [2]: http://zuul.opendev.org/t/openstack/builds?job_name=openstack-ansible-funct…

6 9

[nova] critical bug around reload/upgrades
by Mohammed Naser 24 Aug '19

24 Aug '19

Hello: I've discussed this for quite sometime with Dan over IRC and a bit with Zane as well, but basically, Nova thinks that when it gets a reload (aka SIGHUP), nothing else has occurred. However, oslo.service actually calls stop(), reload() then start() again, which potentially kills all RPC. This has caused a pretty big issue in our gates and it also means that the whole idea behind 'reload nova-compute while upgrading to refresh info' concept is fundamentally broken. I tried to do some work on this here, however, I wasn't really able to get to the bottom of it. There seems to be a decision that needs to be taken in terms of .. do we change what reload() actually means in oslo_service (it actually is more like a restart, not a reload) or does Nova (and other projects) change their implementation in assuming what reload() does? https://review.openstack.org/#/c/641907/ This seems to have been floating around for a really long time, so I'd be happy to work with someone to find the fix (and we can totally test it inside OpenStack Ansible by reloading instead of restarting). Thanks! Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

4 4

Re: [nova][placement] Need for a nova-manage placement audit command
by Matt Riedemann 20 Jun '19

20 Jun '19

Re-posting this since it came up again in IRC today. On 9/20/2018 6:09 PM, Matt Riedemann wrote: > mnaser wrote a simple placement audit tool today and dumped his script > into a bug report [1]. It turns out several operators have already > written a variant of this same tool. The idea is to compare what's in > placement for allocations and (compute node) resource providers against > what's in nova for instances and compute nodes, look for orphans in > placement and then report them, or maybe also clean them up. > > I think this could go into a "nova-manage placement audit" command and > should be pretty easy to write for minimum functionality (start with > just the report). > > I'm advertising the need here in case someone wants to work on this. I'd > like to myself, but just can't justify the time right now. > > [1] https://bugs.launchpad.net/nova/+bug/1793569 > -- Thanks, Matt

2 1

[kolla][magnum] Cluster creation failed due to "Waiting for Kubernetes API..."
by Giuseppe Sannino 30 May '19

30 May '19

Hi all, need an help. I deployed an AIO via Kolla on a baremetal node. Here some information about the deployment: --------------- kolla-ansible: 7.0.1 openstack_release: Rocky kolla_base_distro: centos kolla_install_type: source TLS: disabled --------------- VMs spawn without issue but I can't make the "Kubernetes cluster creation" successfully. It fails due to "Time out" I managed to log into Kuber Master and from the cloud-init-output.log I can see: + echo 'Waiting for Kubernetes API...' Waiting for Kubernetes API... ++ curl --silent http://127.0.0.1:8080/healthz + '[' ok = '' ']' + sleep 5 Checking via systemctl and journalctl I see: [fedora@kube-clsuter-qamdealetlbi-master-0 log]$ systemctl status kube-apiserver ● kube-apiserver.service - kubernetes-apiserver Loaded: loaded (/etc/systemd/system/kube-apiserver.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Tue 2019-02-19 15:31:41 UTC; 45min ago Process: 3796 ExecStart=/usr/bin/runc --systemd-cgroup run kube-apiserver (code=exited, status=1/FAILURE) Main PID: 3796 (code=exited, status=1/FAILURE) Feb 19 15:31:40 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: kube-apiserver.service: Main process exited, code=exited, status=1/FAILURE Feb 19 15:31:40 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: kube-apiserver.service: Failed with result 'exit-code'. Feb 19 15:31:41 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: kube-apiserver.service: Service RestartSec=100ms expired, scheduling restart. Feb 19 15:31:41 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: kube-apiserver.service: Scheduled restart job, restart counter is at 6. Feb 19 15:31:41 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: Stopped kubernetes-apiserver. Feb 19 15:31:41 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: kube-apiserver.service: Start request repeated too quickly. Feb 19 15:31:41 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: kube-apiserver.service: Failed with result 'exit-code'. Feb 19 15:31:41 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: Failed to start kubernetes-apiserver. [fedora@kube-clsuter-qamdealetlbi-master-0 log]$ sudo journalctl -u kube-apiserver -- Logs begin at Tue 2019-02-19 15:21:36 UTC, end at Tue 2019-02-19 16:17:00 UTC. -- Feb 19 15:31:33 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: Started kubernetes-apiserver. Feb 19 15:31:34 kube-clsuter-qamdealetlbi-master-0.novalocal runc[2794]: Flag --insecure-bind-address has been deprecated, This flag will be removed in a future version. Feb 19 15:31:34 kube-clsuter-qamdealetlbi-master-0.novalocal runc[2794]: Flag --insecure-port has been deprecated, This flag will be removed in a future version. Feb 19 15:31:35 kube-clsuter-qamdealetlbi-master-0.novalocal runc[2794]: Error: error creating self-signed certificates: open /var/run/kubernetes/apiserver.crt: permission denied : : : Feb 19 15:31:35 kube-clsuter-qamdealetlbi-master-0.novalocal runc[2794]: error: error creating self-signed certificates: open /var/run/kubernetes/apiserver.crt: permission denied Feb 19 15:31:35 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: kube-apiserver.service: Main process exited, code=exited, status=1/FAILURE Feb 19 15:31:35 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: kube-apiserver.service: Failed with result 'exit-code'. Feb 19 15:31:35 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: kube-apiserver.service: Service RestartSec=100ms expired, scheduling restart. Feb 19 15:31:35 kube-clsuter-qamdealetlbi-master-0.novalocal systemd[1]: kube-apiserver.service: Scheduled restart job, restart counter is at 1. May I ask for an help on this ? Many thanks /Giuseppe

5 11

[all][docs] season of docs
by Mohammed Naser 24 May '19

24 May '19

Hi there: It seems like Google has come up with a new somewhat-GSoC-like idea but focused on documentation. I think it could be a good opportunity for the documentation team (or any specific team actually, coordinated with docs) to be part of this. https://opensource.googleblog.com/2019/03/introducing-season-of-docs.html I'm not sure if the team has the amount of resources, but it seems they should be able to apply to this. Does this seem like something that might help the team more (or perhaps a specific project, coordinating with the docs team) to apply for this? Thanks, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

7 10

[PTL][SIG][WG] PTG Team Photos
by Kendall Nelson 15 May '19

15 May '19

Hello! If your team is attending the PTG and is interested in having a team photo taken, here is the signup[1]! There are slots Thursday and Friday from 10:00 AM to 4:30 PM. The location is TBD but will likely be close to where registration will be. I'll send an email out the day before with a reminder of your time slot and an exact location. -Kendall (diablo_rojo) [1] https://docs.google.com/spreadsheets/d/1DgsRHVWW2YLv7ewfX0M21zWJRf4wUfPG4ff…

5 10

[nova][dev] Pike Extended Maintenance coming soon
by melanie witt 27 Apr '19

27 Apr '19

Hi all, As mentioned in the nova meeting today, Pike Extended Maintenance is coming soon. The date shown on the release page says 2019-03-03 but it's predicted to occur after the Stein release (2019-04-10): http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003745.ht… We are currently in the process of flushing the stable/pike review queue for a final release of stable/pike: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:… Please be sure you get your backports proposed that you'd like to land in the final Pike release. After Pike is put into extended maintenance, there will be no more releases, but patches will still be accepted to the branch: https://docs.openstack.org/project-team-guide/stable-branches.html#maintena… Cheers, -melanie

2 3

Migrating git repos to OpenDev
by corvus＠inaugust.com 25 Apr '19

25 Apr '19

Hi, As discussed in November[1], the OpenStack project infrastructure is being rebranded as "OpenDev" to better support a wider community of projects. We are nearly ready to perform the part of this transition with the largest impact: moving the authoritative git repositories for existing projects. In this email, I'd like to introduce the new hosting system we are preparing, discuss the transition, and invite projects to work with us on the logistics of the change. Gerrit ====== Gerrit is the core of our system and it will remain so in OpenDev. As part of this move, we will rename the gerrit server from review.openstack.org to review.opendev.org. As part of the transition, we will automatically merge appropriate changes to all branches of all repositories updating .gitreview and Zuul configuration files. Any further changes (README files, etc.) we expect to be made by individual project contributors. Repository Browsing =================== Currently our canonical *public* repository system is the cgit server at https://git.openstack.org/ (and git.airshipit.org, git.starlingx.io, and git.zuul-ci.org) This is a load balanced cluster of several servers which is designed to handle all the public git repository traffic, as it scales much better than Gerrit (and has a more friendly domain name). >From a technical standpoint, it's excellent, but its usability could be improved. Therefore, as part of this transition, we will replace the cgit servers with a new system based on Gitea. Gitea is a complete development collaboration system, but it's very flexible and will allow us to disable components which we aren't using. We will operate it in a read-only configuration where it will act as the public mirror for Gerrit. The advantages it has over the current system are: * Shorter domain name in project URLs: https://git.openstack.org/openstack/nova vs https://opendev.org/openstack/nova * Clone and browsing URLs are the same (with cgit, the browsing URL has an extra path component) * More visually pleasing code browsing * Integrated code searching * Ability to highlight multiple lines in links When we perform the transition we will install redirects from git.openstack.org (and the other git sites) to opendev.org, and will maintain those redirects for the foreseeable future. We will construct them so that even existing deep links to individual files in individual commits to cgit will redirect to the correct location on opendev.org. This system is up and running now with a live mirror of data from Gerrit, and you can start testing it out today at https://opendev.org/ Please let us know if you encounter any problems. If you would like to read more about the design of this system and the transition, see the infra-spec[2]. GitHub ====== Currently all OpenStack projects are replicated to GitHub. We do not plan on changing that during the transition, however, any projects outside of the openstack*/ namespaces will not automatically be replicated to GitHub, and we do not plan on adding that in the future. We do, however, support projects using Zuul to run post-merge jobs to push updates to GitHub or any other third-party mirrors with their own credentials. We will be happy to work with anyone interested in that to help set up jobs to do so. We are adopting this approach so that individual projects can have more control over how they are represented in social media, and to give us more flexibility in supporting our own organizational namespaces on OpenDev without assuming they map directly to GitHub. Eventually we plan on moving the OpenStack project to that system as well and retiring direct replication from Gerrit to GitHub completely. But we will defer that work until after this transition. Logistics ========= We can prepare much of the system in advance (as we have for the hosting system on opendev.org) but the actual transition and renaming of the Gerrit server will need to happen at once during an outage window. We need to schedule that outage and begin preparing for it. Since all of the project git URLs are going to change (to replace git.openstack.org with opendev.org and review.openstack.org with review.opendev.org) we can additionally take the opportunity to reorganize projects into different organizations. For example, during the transition we will rename Zuul, and it's associated projects, from the "openstack-infra" org to "zuul". So their new names will be "zuul/zuul", "zuul/nodepool", etc. This is an excellent time for the rest of the OpenStack Foundation pilot projects to do the same. If the OpenStack project desires this, it would also be a good time to move unofficial projects out of the openstack/ namespace. Therefore, we need your help: Action Items ============ We need each of the following projects: * OpenStack * Airship * StarlingX * Zuul To nominate a single point of contact to work with us on the transition. It would be helpful for that person to attend the next (and possibly next several) openstack infra team meetings in IRC [3]. We will work with those people on scheduling the transition, as well as finalizing the list of projects which should be renamed as part of the transition. If you manage an unofficial project and would like to take the opportunity to move or rename your project, please add it to this ethercalc[4]. [1] http://lists.openstack.org/pipermail/openstack-dev/2018-November/136403.html [2] http://specs.openstack.org/openstack-infra/infra-specs/specs/opendev-gerrit… [3] http://eavesdrop.openstack.org/#Project_Infrastructure_Team_Meeting [4] https://ethercalc.openstack.org/opendev-transition

19 39

[dev] oslosphinx Sphinx theme is EOL
by Stephen Finucane 24 Apr '19

24 Apr '19

Just a quick pointer that the oslosphinx theme does not work with Sphinx 2.0, which was released yesterday, failing with the following error: Exception occurred: File "/home/zuul/src/git.openstack.org/openstack/nova-specs/.tox/docs/lib/python3.6/site-packages/oslosphinx/__init__.py", line 76, in builder_inited app.info('Using openstack theme from %s' % theme_dir) AttributeError: 'Sphinx' object has no attribute 'info' The oslo team have decided not to fix these issues, given that oslosphinx is EOL since Pike, and we suggest that people instead migrate to openstackdocstheme [1] (for projects that are able to use the OpenStack brand [2]) or something else (for projects that are not). For the former, I suggest follow the openstackdocstheme [1] or look at what we've done in nova-specs [3] for an existing example. If you have issues, please reach out on #openstack-doc and one of us will try our best to help you migrate. I'll work with infra to get the repo shuttered and avoid any further confusion with this theme going forward. Stephen [1] https://docs.openstack.org/openstackdocstheme/latest/ [2] https://www.openstack.org/brand [3] https://review.openstack.org/#/c/648670/

1 1