November 2018 - openstack-discuss

Re: [openstack-dev] [vitrage] I have some problems with Prometheus alarms in vitrage.
by Won 30 Nov '18

30 Nov '18

Hi, I attached four log files. I collected the logs from about 17:14 to 17:42. I created an instance of 'deltesting3' at 17:17. 7minutes later, at 17:24, the entity graph showed the dentesting3 and vitrage colletor and graph logs are appeared. When creating an instance in ubuntu server, it appears immediately in the entity graph and logs, but when creating an instance in computer1 (multi node), it appears about 5~10 minutes later. I deleted an instance of 'deltesting3' around 17:26. > After ~20minutes, there was only Apigateway. Does it make sense? did you > delete the instances on ubuntu, in addition to deltesting? > I only deleted 'deltesting'. After that, only the logs from 'apigateway' and 'kube-master' were collected. But other instances were working well. I don't know why only two instances are collected in the log. NOV 19 In this log, 'agigateway' and 'kube-master' were continuously collected in a short period of time, but other instances were sometimes collected in long periods. In any case, I would expect to see the instances deleted from the graph at > this stage, since they were not returned by get_all. > Can you please send me the log of vitrage-graph at the same time (Nov 15, > 16:35-17:10)? > Information 'deldtesting3' that has already been deleted continues to be collected in vitrage-graph.service. Br, Won 2018년 11월 15일 (목) 오후 10:13, Ifat Afek <ifatafekn(a)gmail.com>님이 작성: > > On Thu, Nov 15, 2018 at 10:28 AM Won <wjstk16(a)gmail.com> wrote: > >> Looking at the logs, I see two issues: >>> 1. On ubuntu server, you get a notification about the vm deletion, while >>> on compute1 you don't get it. >>> Please make sure that Nova sends notifications to >>> 'vitrage_notifications' - it should be configured in /etc/nova/nova.conf. >>> 2. Once in 10 minutes (by default) nova.instance datasource queries all >>> instances. The deleted vm is supposed to be deleted in Vitrage at this >>> stage, even if the notification was lost. >>> Please check in your collector log for the a message of "novaclient.v2.client >>> [-] RESP BODY" before and after the deletion, and send me its content. >> >> >> I attached two log files. I created a VM in computer1 which is a >> computer node and deleted it a few minutes later. Log for 30 minutes from >> VM creation. >> The first is the log of the vitrage-collect that grep instance name. >> The second is the noovaclient.v2.clinet [-] RESP BODY log. >> After I deleted the VM, no log of the instance appeared in the collector >> log no matter how long I waited. >> >> I added the following to Nova.conf on the computer1 node.(attached file >> 'compute_node_local_conf.txt') >> notification_topics = notifications,vitrage_notifications >> notification_driver = messagingv2 >> vif_plugging_timeout = 300 >> notify_on_state_change = vm_and_task_state >> instance_usage_audit_period = hour >> instance_usage_audit = True >> > > Hi, > > From the collector log RESP BODY messages I understand that in the > beginning there were the following servers: > compute1: deltesting > ubuntu: Apigateway, KubeMaster and others > > After ~20minutes, there was only Apigateway. Does it make sense? did you > delete the instances on ubuntu, in addition to deltesting? > In any case, I would expect to see the instances deleted from the graph at > this stage, since they were not returned by get_all. > Can you please send me the log of vitrage-graph at the same time (Nov 15, > 16:35-17:10)? > > There is still the question of why we don't see a notification from Nova, > but let's try to solve the issues one by one. > > Thanks, > Ifat > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

2 2

[openlab] November Update
by Melvin Hillsman 29 Nov '18

29 Nov '18

Hi everyone, We should have been doing this - totally my fault here - and hope you find updates at least monthly re OpenLab useful. In order to allow all people participating in OpenLab to provide details on the updates we are using etherpad: https://etherpad.openlabtesting.org/p/november2018update Thanks to all the community members helping to make OpenLab successful! -- Kind regards, Melvin Hillsman mrhillsman(a)gmail.com mobile: (832) 264-2646

2 1

[ironic] Finding a time to discuss cleaning/deployment steps visibility
by Julia Kreger 29 Nov '18

29 Nov '18

Greetings everyone, As discussed previously, I'd like to find a time when we can get on a higher bandwidth medium to discuss making clean and deployment steps visible. Generally we've struggled to gain consensus and I'd like for us to focus on trying to reach a minimally viable feature. I've created a doodle at: https://doodle.com/poll/yan4wyvztf7mpq46 Most recent specification: https://review.openstack.org/#/c/606199/ -Julia

1 0

[all] [infra] StoryBoard Migration: The Remaining Blockers Summary
by Kendall Nelson 29 Nov '18

29 Nov '18

Hello! As with all of my other summaries, if you are looking for the long etherpad version, you can find it here[1]. Somewhere along the way I used the wrong etherpad for notes and so we ended up doing the StoryBoard things in the etherpad originally meant for another session so pardon the name. Anywho. Overall the session went really well, there weren’t any huge glaring issues that anyone brought up. The majority of things were all smaller usability things. Just about all of which are definitely doable. The only slightly larger feature request (that we hadn't previously discussed) was a separate interface for filing stories- similar to the one LP provides. Something simple so the user doesn’t feel like they need to fill in all this information, but can just provide the minimum. I haven’t quite gotten around to making stories for all of the things discussed in the etherpad, but should have them up soon! You will be able to see them here[2], once they’ve been created. If you have opinions about any of what we discussed or have something you wanted to bring up but could not attend the session please drop into #storyboard and chat with us, or join us at our weekly meeting on Wednesday at 19:00 UTC. -Kendall (diablo_rojo) [1]https://etherpad.openstack.org/p/BER-Contrib-Portal-Feedback (yes, I know the name doesn’t seem right..) [2] https://storyboard.openstack.org/#!/story/list?status=active&project_group_…

2 2

[tripleo] gate is backed up (again)
by Alex Schultz 29 Nov '18

29 Nov '18

Ahoy folks. We're running into a massive backlog and it's messing with everyone again. As stop gap, we're clearing the gate and will need to land the patches to switch them back to non-voting and remove the scenarios[0] from the gate in stable branches. We had switched them back to voting ~2 weeks ago because they were still in the gate (but non-voting). I thought we weren't going to run into as many issues in the stable branches but we've been backporting a bit more lately. We are proposing this because in the gate we've had a bunch of scenario failures[1] not related to code but just time outs. Additionally we're making progress on converting the master scenario jobs to standalone versions which may be backportable to stable/rocky. So in the mean time, we're asking that you not approve or recheck anything until the CI has stabilized. We'll send out a note when we're clear. Thanks, -Alex [0] https://review.openstack.org/#/q/topic:scenarios-nv+(status:open+OR+status:… [1] http://zuul.openstack.org/builds?pipeline=gate&result=failure&result=timed_…

1 0

Re: [Openstack-operators] Fwd: Nova hypervisor uuid
by Ignazio Cassano 29 Nov '18

29 Nov '18

Hi Matt, sorry but I lost your answer and Gianpiero forwarded it to me. I am sure kvm nodes names are note changed. Tables where uuid are duplicated are: dataresource_providers in nova_api db compute_nodes in nova db Regards Ignazio Il 28/Nov/2018 11:09 AM, "Gianpiero Ardissono" <gp.ardissono(a)gmail.com> ha scritto: > > ---------- Forwarded message --------- > From: Matt Riedemann <mriedemos(a)gmail.com> > Date: mar 27 nov 2018, 19:03 > Subject: Re: [Openstack-operators] Nova hypervisor uuid > To: <openstack-operators(a)lists.openstack.org> > > > On 11/27/2018 11:32 AM, Ignazio Cassano wrote: > > Hi All, > > Please anyone know where hypervisor uuid is retrived? > > Sometime updating kmv nodes with yum update it changes and in nova > > database 2 uuids are assigned to the same node. > > regards > > Ignazio > > > > > > > > > > _______________________________________________ > > OpenStack-operators mailing list > > OpenStack-operators(a)lists.openstack.org > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > > > > To be clear, do you mean the computes_nodes.uuid column value in the > cell database? Which is also used for the GET /os-hypervisors response > 'id' value if using microversion >= 2.53. If so, that is generated > randomly* when the compute_nodes table record is created: > > > https://github.com/openstack/nova/blob/8545ba2af7476e0884b5e7fb90965bef92d6… > > > https://github.com/openstack/nova/blob/8545ba2af7476e0884b5e7fb90965bef92d6… > > When you hit this problem, are you sure the hostname on the compute host > is not changing? Because when nova-compute starts up, it should look for > the existing compute node record by host name and node name, which for > the libvirt driver should be the same. That lookup code is here: > > > https://github.com/openstack/nova/blob/8545ba2af7476e0884b5e7fb90965bef92d6… > > So the only way nova-compute should create a new compute_nodes table > record for the same host is if the host/node name changes during the > upgrade. Is the deleted value in the database the same (0) for both of > those records? > > * The exception to this is for the ironic driver which re-uses the > ironic node uuid as of this change: > https://review.openstack.org/#/c/571535/ > > -- > > Thanks, > > Matt > > _______________________________________________ > OpenStack-operators mailing list > OpenStack-operators(a)lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators > _______________________________________________ OpenStack-operators mailing list OpenStack-operators(a)lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

2 6

[release][ptl] Holding all release activity
by Sean McGinnis 29 Nov '18

29 Nov '18

Hey everyone, There have been a rash of Pypi and other issues lately that have led to release job failures. We will be holding off on processing any more releases until we've seen some evidence that things have been resolved. I would guess this means that there will be no releases done until early next week. If you have a critical need for a release, please come talk to us in the #openstack-release channel and we can see if there is anything we can do in the interim. Thanks! Sean

1 1

[glance][ops][users]OpenStack Glance default visibility survey
by ekuvaja＠redhat.com 29 Nov '18

29 Nov '18

I've invited you to fill out the following form: OpenStack Glance default visibility survey To fill it out, visit: https://docs.google.com/forms/d/e/1FAIpQLScbbmeLQrx34P84U-Wr2RoLOpXdWMEZg-X… Hello all, Time for another very quick survey from Glance team. This time we would like to get your feedback on if we should set the default visibility of newly created images to "private" or not. Please note this survey closes in a week and we will take the action based on the results provided. Erno jokke Kuvaja I've invited you to fill out a form: Google Forms: Create and analyze surveys.

1 0

[release] Release countdown for week R-18, Dec 3-7
by Sean McGinnis 29 Nov '18

29 Nov '18

Welcome back from the Summit and holidays. Back to our regularly scheduled countdown emails. Development Focus ----------------- With the longer development cycle we are still a few weeks out until milestone 2. Hopefully folks got some good input and feedback and the Summit and Forum. Teams should be taking input and adjusting any earlier plans as needed based on this. General Information ------------------- Now may be a good time for teams to check on any stable releases that need to be done for your deliverables. If you have bugfixes that have been backported, but no stable release getting those. If you are unsure what is out there committed but not released, in the openstack/releases repo, running the command "tools/list_stable_unreleased_changes.sh <cycle_name>" gives a nice report. Upcoming Deadlines & Dates -------------------------- Start using openstack-discuss ML: November 19 (in case you missed it) Stein-2 Milestone: January 10 -- Sean McGinnis (smcginnis)

1 0

[dev] [tc][all] TC office hours is started now on #openstack-tc
by Ghanshyam Mann 29 Nov '18

29 Nov '18

Hi All, TC office hour is started on #openstack-tc channel. Feel free to reach to us for anything you want discuss/input/feedback/help from TC. - TC

1 0