April 2019 - openstack-discuss - lists.openstack.org

[trove] Use storyboard
by Lingxian Kong 02 Apr '19

02 Apr '19

Hi trove team, We are going to use storyboard[1] for task tracking from Train dev cycle, please make sure there is related story/task for every new patch from now on. I didn't migrate things from launchpad so the links in the existing patches are still valid. [1]: https://storyboard.openstack.org/#!/project_group/trove --- Cheers, Lingxian Kong Catalyst Cloud

1 0

[neutron] CI meeting cancel
by Slawomir Kaplonski 02 Apr '19

02 Apr '19

Hi, I can’t chair neutron CI meeting this Tuesday (2.04) so I cancel it. See You on next meeting on 9.04. — Slawek Kaplonski Senior software engineer Red Hat

1 0

[training-labs] DHCP IP conflict with Ubuntu 18.04.2
by Julian Meier 02 Apr '19

02 Apr '19

Hi, we are currently working on a new environment for our school to teach some basic knowledge about OpenStack. Our goal is to run OpenStack training-labs for each student on VMs running on ESXi. Therefor we've enabled nested virtualization (vhv.enable=TRUE) for the VMs in vSphere. Then we basically do following on Ubuntu 18.04.2: ``` echo 1 > /sys/module/kvm/parameters/ignore_msr # to avoid following error while building the base image: "This usually means the libguestfs appliance failed to start or crashed." sudo apt-get install -y qemu-kvm libvirt-bin libguestfs-tools bridge-utils virt-manager virsh pool-create-as --name default --type dir --target /var/lib/libvirt/images wget http://tarballs.openstack.org/training-labs/dist/labs-stable-rocky.tgz tar xzf labs-stable-rocky.tgz cd labs/ ``` We build the base image ubuntu-18.04-server-amd64 with `./st.py --build basedisk --provider kvm` and then the cluster `./st.py --build cluster --provider kvm`. Everything works fine till...: stacktrain.log: ``` 12256 13:08:19.992 stacktrain.core.autostart INFO Start autostart/04_setup_nova_compute.sh 12256 13:08:19.994 stacktrain.core.ssh DEBUG vm_ssh: ssh -q -i /opt/oslab/labs/lib/osbash-ssh-keys/osbash_key -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ConnectTimeout=20 -o ControlPath=none -p 22 osbash(a)192.168.122.208 bash autostart/04_setup_nova_compute.sh && rm -vf autostart/04_setup_nova_compute.sh 12256 13:08:19.994 stacktrain.core.ssh DEBUG Writing live log for ssh call at /opt/oslab/labs/log/021_04_setup_nova_compute.auto. 12256 13:08:56.551 stacktrain.core.ssh ERROR ssh returned status 255. 12256 13:08:56.552 stacktrain.core.autostart ERROR Script failure: 04_setup_nova_compute 11272 13:08:57.110 stacktrain.core.autostart ERROR Script failed. Exiting. ``` ssh.log: ``` ssh -q -i /opt/oslab/labs/lib/osbash-ssh-keys/osbash_key -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ConnectTimeout=20 -o ControlPath=none -p 22 osbash(a)192.168.122.208 mkdir -p autostart ssh -q -i /opt/oslab/labs/lib/osbash-ssh-keys/osbash_key -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ConnectTimeout=20 -o ControlPath=none -p 22 osbash(a)192.168.122.208 bash autostart/04_setup_nova_compute.sh && rm -vf autostart/04_setup_nova_compute.sh ``` For me it looks like both VMs (controller + compute1) get the same IP address (with `./st.py --build nodes_only --provider kvm`): ... Your cluster nodes: INFO VM name: compute1 INFO SSH login: ssh osbash(a)192.168.122.47 INFO (password: osbash) INFO VM name: controller INFO SSH login: ssh osbash(a)192.168.122.47 INFO (password: osbash) INFO Dashboard: Assuming horizon is on controller VM. INFO http://192.168.122.47/horizon/ INFO User : myuser (password: myuser_user_pass) INFO User : admin (password: admin_user_secret) INFO Network: mgmt INFO Network address: 10.0.0.0 INFO Network: provider INFO Network address: 203.0.113.0 ``` >From within the VMs: ``` # controller: osbash@osbash:~$ ifconfig ens3 ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.122.47 netmask 255.255.255.0 broadcast 192.168.122.255 inet6 fe80::5054:ff:fed9:b165 prefixlen 64 scopeid 0x20<link> ether 52:54:00:d9:b1:65 txqueuelen 1000 (Ethernet) ... osbash@osbash:~$ netplan ip leases ens3 # This is private data. Do not parse. ADDRESS=192.168.122.47 NETMASK=255.255.255.0 ROUTER=192.168.122.1 SERVER_ADDRESS=192.168.122.1 NEXT_SERVER=192.168.122.1 BROADCAST=192.168.122.255 T1=1800 T2=3150 LIFETIME=3600 DNS=192.168.122.1 HOSTNAME=osbash CLIENTID=ffb55e67ff00020000ab11bba91ddef33599c5 osbash@osbash:~$ cat /etc/machine-id b8c7057a4c084e69aa3743960d41d5ce # compute1: osbash@compute1:~$ ifconfig ens3 ens3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.122.47 netmask 255.255.255.0 broadcast 192.168.122.255 inet6 fe80::5054:ff:fe17:b640 prefixlen 64 scopeid 0x20<link> ether 52:54:00:17:b6:40 txqueuelen 1000 (Ethernet) ... osbash@compute1:~$ netplan ip leases ens3 # This is private data. Do not parse. ADDRESS=192.168.122.47 NETMASK=255.255.255.0 ROUTER=192.168.122.1 SERVER_ADDRESS=192.168.122.1 NEXT_SERVER=192.168.122.1 BROADCAST=192.168.122.255 T1=1800 T2=3150 LIFETIME=3600 DNS=192.168.122.1 HOSTNAME=compute1 CLIENTID=ffb55e67ff00020000ab11bba91ddef33599c5 osbash@compute1:~$ cat /etc/machine-id b8c7057a4c084e69aa3743960d41d5ce ``` I believe the VMs share the same IP address because of the same machine-id (which is used as DUID: http://manpages.ubuntu.com/manpages/bionic/man5/networkd.conf.5.html) Am I wrong? Thanks for your help, Julian

1 0

[placement] update 19-12
by Chris Dent 02 Apr '19

02 Apr '19

HTML: https://anticdent.org/placement-update-19-12.html Placement update 19-12. Nearing 1/4 of the way through the year. I won't be around on Monday, if someone else can chair the [meeting](http://eavesdrop.openstack.org/#Placement_Team_Meeting) that would be great. Or feel free to cancel it. # Most Important An RC2 was cut earlier this week, expecting it to be the last, but there are a [couple of patches](https://review.openstack.org/#/q/project:openstack/placement+branc… which could be put in an RC3 if we were inclined that way. Discuss. We merged a first suite of [contribution guidelines](https://docs.openstack.org/placement/latest/contributor/contrib…. These are worth reading as they explain how to manage bugs, start new features, and be a good reviewer. Because of the introduction of StoryBoard, processes are different from what you may have been used to in Nova. Because of limited time and space and conflicting responsibilities the Placement team will be doing a [Virtual Pre-PTG](http://lists.openstack.org/pipermail/openstack-discuss/2019-March/…. # What's Changed * The contribution guidelines linked above describe how to manage specs, which will now be in-tree. If you have a spec to propose or re-propose (from stein in nova), it now goes in ``doc/source/specs/train/approved/``. * Some [image type traits](https://review.openstack.org/648147) have merged (to be used in a nova-side request pre-filter), but the change has exposed an issue we'll need to resolve: os-traits and os-resource-classes are under the cycle-with-intermediary style release which means that at this time in the cycle it is difficult to make a release which can delay work. We could switch to independent. This would make sense for libraries that are basically lists of strings. It's hard to break that. We could also investigate making os-traits and os-resource-classes required-projects in job templates in zuul. This would allow them to be "tox siblings". Or we could wait. Please express an opinion if you have one. * In discussion in `#openstack-nova` about the patch to [delete placement from nova](https://review.openstack.org/#/c/618215/), it was decided that rather than merge that after the final RC, we would wait until the PTG. There is discussion on the patch which attempts to explain the reasons why. # Specs/Blueprint/Features * The spec for [forbidden aggregate membership](https://docs.openstack.org/placement/latest/specs/train/approve… has merged. * The two traits-related specs from stein will need to be re-proposed to placement for train: * [any traits](http://specs.openstack.org/openstack/nova-specs/specs/stein/approve… * [mixing required traits](http://specs.openstack.org/openstack/nova-specs/specs/stein/approve… * The spec for [request group mapping](https://review.openstack.org/597601) will need to be revisited. # Bugs * StoryBoard stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement): 4 * Placement related [bugs not yet in progress](https://goo.gl/TgiPXb): 14. * [In progress placement bugs](https://goo.gl/vzGGDQ) 7. +1. We should do a bug squash day at some point. Should we wait until after the PTG or no? Note that the [contribution guidelines](https://docs.openstack.org/placement/latest/contributor/contrib… has some information on how to evaluate new stories and what tags to add. # osc-placement osc-placement is currently behind by 13 microversions. Pending changes: * [support for 1.19](https://review.openstack.org/#/c/641094/) * [support for 1.21](https://review.openstack.org/#/c/641123/) * [aggregate allocation ratio tool](https://review.openstack.org/#/c/640898/) # Main Themes Be thinking about what you'd like the main themes to be. Put them on the [PTG etherpad](https://etherpad.openstack.org/p/placement-ptg-train). # Other Placement * <https://review.openstack.org/#/q/topic:2005297-negative-aggregate-membership> Negative member of aggregate filtering resource providers and allocation candidates. This is nearly ready. * <https://review.openstack.org/#/c/645255/> This is a start at unit tests for the PlacementFixture. It is proving a bit "fun" to get right, as there are many layers involved. Making sure seemingly unrelated changes in placement don't break the nova gate is important. Besides these unit tests, there's discussion on the PTG etherpad of running the nova functional tests, or a subset thereof, in placement's check run. On the one hand this is a pain and messy, but on the other consider what we're enabling: Functional tests that use the real functionality of an external service (real data, real web requests), not stubs or fakes. * <https://review.openstack.org/641404> Use ``code`` role in api-ref titles # Other Service Users There's a lot here, but it is certain this is not all of it. If I missed something you care about, followup mentioning it. * <https://review.openstack.org/552924> Nova: Spec: Proposes NUMA topology with RPs * <https://review.openstack.org/622893> Nova: Spec: Virtual persistent memory libvirt driver implementation * <https://review.openstack.org/641899> Nova: Check compute_node existence in when nova-compute reports info to placement * <https://review.openstack.org/601596> Nova: spec: support virtual persistent memory * <https://review.openstack.org/#/q/topic:bug/1790204> Workaround doubling allocations on resize * <https://review.openstack.org/555081> Nova: Spec: Standardize CPU resource tracking * <https://review.openstack.org/646029> Nova: Spec: Use in_tree getting allocation candidates * <https://review.openstack.org/645316> Nova: Pre-filter hosts based on multiattach volume support * <https://review.openstack.org/606199> Ironic: A fresh way of looking at step retrieval * <https://review.openstack.org/647396> Nova: Add flavor to requested_resources in RequestSpec * <https://review.openstack.org/633204> Blazar: Retry on inventory update conflict * <https://review.openstack.org/640080> Nova: Use aggregate_add_host in nova-manage * <https://review.openstack.org/#/q/topic:bp/count-quota-usage-from-placement> Nova: count quota usage from placement * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.openstack.org/648500> Tempest: Init placement client in tempest Manager object * <https://review.openstack.org/624335> puppet-tripleo: Initial extraction of the Placement service from Nova * <https://review.openstack.org/#/q/topic:bug/1821824> Nova: bug fix prevent forbidden traits from working as expected * <https://review.openstack.org/648665> Nova: Spec for a new nova virt driver to manage an RSD * <https://review.openstack.org/#/q/topic:bug/1819460> Nova: Handle placement error during re-schedule * <https://review.openstack.org/#/c/642067/> Helm: Allow more generic overrides for nova placement-api * <https://review.openstack.org/647578> Nova: add spec for image metadata prefiltering -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

3 3

[Magnum] Cluster Create failure
by Navdeep Uniyal 02 Apr '19

02 Apr '19

Hi All, I am trying to create a cluster using Magnum using : magnum cluster-create kubernetes-cluster --cluster-template kubernetes-cluster-template --master-count 1 --node-count 1 --keypair magnum_key My Template looks like this: +-----------------------+--------------------------------------+ | Property | Value | +-----------------------+--------------------------------------+ | insecure_registry | - | | labels | {} | | updated_at | - | | floating_ip_enabled | True | | fixed_subnet | - | | master_flavor_id | small | | user_id | b4727cb329c14c388d777d0ce38c8a6b | | uuid | 24d01aea-c968-42e3-bcaa-a2e756aac5c7 | | no_proxy | - | | https_proxy | - | | tls_disabled | False | | keypair_id | - | | hidden | False | | project_id | 4c6bc4445c764249921a0a6e40b192dd | | public | False | | http_proxy | - | | docker_volume_size | 4 | | server_type | vm | | external_network_id | 5guknet | | cluster_distro | fedora-atomic | | image_id | fedora-atomic-latest | | volume_driver | - | | registry_enabled | False | | docker_storage_driver | devicemapper | | apiserver_port | - | | name | kubernetes-cluster-template | | created_at | 2019-03-27T12:21:27+00:00 | | network_driver | flannel | | fixed_network | - | | coe | kubernetes | | flavor_id | small | | master_lb_enabled | False | | dns_nameserver | 8.8.8.8 | +-----------------------+--------------------------------------+ I am getting the following error: {"explanation": "The server could not comply with the request since it is either malformed or otherwise incorrect.", "code": 400, "error": {"message": "ResourceTypeUnavailable: : resources.kube_masters<nested_st ack>.resources.0<file:///var/lib/magnum/env/lib/python2.7/site-packages/magnum/drivers/k8s_fedora_atomic_v1/templates/kubemaster.yaml>: : HEAT-E99001 Service cinder is not available for resource type Magnum::Opt ional::Cinder::Volume, reason: cinder volumev3 endpoint is not in service catalog.", "traceback": null, "type": "StackValidationFailed"}, "title": "Bad Request"} log_http_response /var/lib/magnum/env/local/lib/python2.7/site-packages/heatclient/common/http.py:157 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server [req-b4037dc5-2fe1-4533-a6e3-433609c6b22d - - - - -] Exception during message handling: InvalidParameterValue: ERROR: ResourceTypeUnavailable: : res ources.kube_masters<nested_stack>.resources.0<file:///var/lib/magnum/env/lib/python2.7/site-packages/magnum/drivers/k8s_fedora_atomic_v1/templates/kubemaster.yaml>: : HEAT-E99001 Service cinder is not available for resource type Magnum::Optional::Cinder::Volume, reason: cinder volumev3 endpoint is not in service catalog. 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server File "/var/lib/magnum/env/local/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 166, in _process_incoming 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server File "/var/lib/magnum/env/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 265, in dispatch 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server File "/var/lib/magnum/env/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server File "/var/lib/magnum/env/local/lib/python2.7/site-packages/magnum/conductor/handlers/cluster_conductor.py", line 80, in cluster_create 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server raise e 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server InvalidParameterValue: ERROR: ResourceTypeUnavailable: : resources.kube_masters<nested_stack>.resources.0<file:///var/lib/magnum/env/lib/python2.7/s ite-packages/magnum/drivers/k8s_fedora_atomic_v1/templates/kubemaster.yaml>: : HEAT-E99001 Service cinder is not available for resource type Magnum::Optional::Cinder::Volume, reason: cinder volumev3 endpoint is not in service catalog. 2019-03-27 17:28:11.174 145885 ERROR oslo_messaging.rpc.server The magnum.conf file is as suggested in https://docs.openstack.org/magnum/pike/install/install-guide-from-source.ht… I DO NOT have cinder in my openstack. I believe, it is optional. Please suggest how can I resolve this issue. Kind Regards, Navdeep

4 13

[nova][CI] nova-live-migration failing
by Eric Fried 01 Apr '19

01 Apr '19

I've noticed the nova-live-migration job failing consistently over the weekend on clearly unrelated patches. Haven't had a chance to look into it at all. Please consider holding off blind rechecks for a bit. Thanks, Eric Fried Concept Brazilian Jiu Jitsu http://taylorbjj.com

2 1

[neutron] CI broken
by Slawomir Kaplonski 01 Apr '19

01 Apr '19

Hi, Just FYI, since few days we have broken neutron-tempest-plugin-designate-scenario job and it is failing 100% times. Bug is reported in [1]. If this job failed on Your patch, please don’t recheck as it will not solve the problem. [1] https://bugs.launchpad.net/neutron/+bug/1822453 — Slawek Kaplonski Senior software engineer Red Hat

2 2

Re: [nova][CI] nova-live-migration failing
by Eric Fried 01 Apr '19

01 Apr '19

> I've noticed the nova-live-migration job failing consistently over the > weekend on clearly unrelated patches. Haven't had a chance to look > into it at all. Please consider holding off blind rechecks for a bit. Matt identified bug [1], cause [2], and fix [3]. Please wait until the latter has merged (or rebase on top of it) before rechecking in nova. Thanks, efried [1] https://bugs.launchpad.net/nova/+bug/1822605 [2] https://review.openstack.org/#/c/601433/ [3] https://review.openstack.org/#/c/649036/

1 0

[neutron] bug deputy report for week of 2019-03-25
by Bence Romsics 01 Apr '19

01 Apr '19

Hi Neutrinos, I was the bug deputy this week. I'm going to be on a training on Monday, so I'm sending this report a bit early. If I have the chance on Monday I'll send an updated version still before the team meeting in reply to this. We have no critical bugs and no new RFEs this week. If I may, please let me call your attention to the two High bugs below. High: * https://bugs.launchpad.net/neutron/+bug/1821912 intermittent ssh failures in various scenario tests gate failure, no fix proposed yet * https://bugs.launchpad.net/neutron/+bug/1822256 Ip segments lost when restart ovs-agent with openvswitch firewall packet loss at agent restart, no fix proposed yet Medium: * https://bugs.launchpad.net/neutron/+bug/1822100 Network Update if current provider net attribure key/value pair in request fix proposed: https://review.openstack.org/648522 * https://bugs.launchpad.net/neutron/+bug/1822105 Policy rules related to "sub parameters" doesn't work properly fix proposed: https://review.openstack.org/648532 * https://bugs.launchpad.net/neutron/+bug/1821948 Unstable unit test uses subnet broadcast address low frequency gate failure, fix merged: https://review.openstack.org/648172 Low: * https://bugs.launchpad.net/neutron/+bug/1822155 neutron-keepalived-state-change can not start on some python3 distro problem specific to (unsupported) python3.4, fix proposed: https://review.openstack.org/648459 * https://bugs.launchpad.net/neutron/+bug/1822199 neutron-vpn-netns-wrapper not invoked with --rootwrap_config parameter low hanging fruit, no fix proposed yet Incomplete: * https://bugs.launchpad.net/neutron/+bug/1821567 network_segment_ranges could not load in tricirlce test likely not a neutron bug but a tricircle one * https://bugs.launchpad.net/neutron/+bug/1821925 Limit test coverage for Extended Maintenance stable branches please look at it if you're interested in stable / extended maintenance process * https://bugs.launchpad.net/neutron/+bug/1821963 Rally test delete-subnets fails at higher concurrency could not reproduce yet, waiting to hear back from reporter Duplicate: * https://bugs.launchpad.net/neutron/+bug/1821357 VRRP vip on VM not reachable from other network on DVR setup duplicate of https://bugs.launchpad.net/bugs/1774459 Cheers, Bence irc: rubasov

1 1

[kuryr] Changing upstream meeting time
by Daniel Mellado 01 Apr '19

01 Apr '19

Hi all! Since most of our regular meeting attendees are in CEST time zone, I'm shifting the time slot for our regular meetings to match DST, so the new meeting time would be Mondays at 14:00 UTC. I've already proposed a patch at [1]. Searching at http://eavesdrop.openstack.org/ shows no conflict, so we'll be meeting at that time at #openstack-meeting-4 (falling back to #openstack-kuryr should any issues arise). If you have any doubt or comment, feel free to ping me at the irc. Best! Daniel [1] https://review.openstack.org/#/c/648927/

1 0