April 2019 - openstack-discuss - lists.openstack.org

Re: [Release-job-failures] Release of openstack/solum failed
by Thierry Carrez 23 Apr '19

23 Apr '19

zuul(a)openstack.org wrote: > Build failed. > > - release-openstack-python http://logs.openstack.org/19/19a7574237f44807b16c37e0983223ff57340ba3/relea… : POST_FAILURE in 4m 40s > - announce-release announce-release : SKIPPED > - propose-update-constraints propose-update-constraints : SKIPPED Solum release post-job (triggered by stable/pike 5.4.1 tagging) failed with the following error: Ensure twine is installed: [ "python3", "-m", "pip", "install", "twine!=1.12.0", "readme_renderer[md]!=23.0", "requests-toolbelt!=0.9.0", "--user" ] Can not perform a '--user' install. User site-packages are not visible in this virtualenv. http://logs.openstack.org/19/19a7574237f44807b16c37e0983223ff57340ba3/relea… This seems to be a new error hitting us recently (also happened for networking-arista, which the release management team does not directly handle). It is weird since the job (exact same command) worked in subsequent runs: http://logs.openstack.org/d0/d0d2089455efb6d01a1158bad597f42cf2306c30/relea… -- Thierry Carrez (ttx)

1 1

[release-openstack-python][release pipeline] POST_FAILURE since last couple of hours, already known and being fixed?
by Yatin Karel 23 Apr '19

23 Apr '19

Hi all, release-openstack-python[1] job is failing since last couple of hours, is this already known and worked upon? If it's not being fixed yet good to hold other releases until it's fixed. The job failing at ansible task: ensure-twine : Ensure twine is installed with Error:- Can not perform a '--user' install. User site-packages are not visible in this virtualenv. Example log:- http://logs.openstack.org/c0/c09c5676b549b6705e7286ee6d1d7e000b9e756f/relea… Actually i tried to reach via IRC at #openstack-release but seems release team in other time zone, posting on mailing list so it can be followed up. Noticed the issue when found tarballs missing when building the released projects in RDO [2]. [1] http://zuul.openstack.org/builds?job_name=release-openstack-python [2] https://logs.rdoproject.org/97/20397/1/check/distgit-cbs-validate/7bd5d70/j… Thanks and Regards Yatin Karel

2 1

[Neutron][TripleO] Avoid dangling sidecars containers - add hook support
by Cédric Jeanneret 23 Apr '19

23 Apr '19

Dear List, While doing some validations on an "in-use" node, we detected a container being Exited with status 137. After some researches and discussions, it appears this container is launched from a Neutron container, as a sidecar, and then the service is killed: https://github.com/openstack/neutron/blob/master/neutron/agent/linux/extern… While the "kill" is fine outside of a container, it leads to some issues in a containerized world: - dangling container is here, with a failed state - validating container state is therefore complicated - might lead to false assumption if we don't know that "kill" process - might lead to disk space issues In order to sort that bad situation out, and prevent disk space issues and the like, I propose to discuss about some "hook" addition in that "external_process.py" (or wherever is more suited for that usage) There is apparently something like that for the service launch, since there are wrapper scripts in /var/lib/neutron. In my idea, that wrapper script could generate a new wrapper used in order to actually delete the container (instead of kill -9 <pid>). The neutron code could be modified in a simple way, something like: ############### if os.path.exists(<wrapper-file>): utils.execute(['bash', <wrapper-file>]) else: # current way to handle things ############### The <wrapper-file> might have "something" in its name in order to ensure we're actually killing the right process/container. self.pid, or maybe some specific string that neutron is aware of... you get the idea. Even outside of a container, that might be used in order to clean temporary configurations/files, and ensure we're actually facing a clean environment. It's really something more like "hooks" than "container-centric thingy" Would you consider that kind of new feature? Thank you for your time and consideration! Cheers, C. -- Cédric Jeanneret Software Engineer DFG:DF

2 2

vm missing psi-passthrough device if cpu pinning is in place
by Manuel Sopena Ballesteros 23 Apr '19

23 Apr '19

Dear Openstack community, I am trying to setup a vm with bith pci-passthrough and cpu pinning. This is my flavor: [root@openstack-deployment ~]# openstack flavor show kudu.test.dedicated.tiny +----------------------------+--------------------------------------------------------------+ | Field | Value | +----------------------------+--------------------------------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | disk | 10 | | id | 7e241e94-1c50-4a92-9ce6-8abb272d79d8 | | name | kudu.test.dedicated.tiny | | os-flavor-access:is_public | True | | properties | hw:cpu_policy='dedicated', hw:pci_passthrough:alias='nvme:1' | | ram | 4096 | | rxtx_factor | 1.0 | | swap | | | vcpus | 2 | +----------------------------+--------------------------------------------------------------+ And this is how I create the vm: [root@openstack-deployment ~]# openstack server create --flavor kudu.test.dedicated.tiny --image centos7.5-image --nic net-id=hpc --security-group admin --key-name mykey --availability-zone nova:zeus-53.localdomain kudu-test-3-1 The problem is that the vm does not have the nvme pci device attached to it. [root@zeus-53 test]# docker exec -it nova_libvirt virsh dumpxml instance-000000ad <domain type='kvm' id='9'> <name>instance-000000ad</name> <uuid>5d8f6cc1-e716-4f4d-ac55-df24d4cffd38</uuid> <metadata> <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> <nova:package version="18.2.0-1.el7"/> <nova:name>kudu-test-3-1</nova:name> <nova:creationTime>2019-04-17 05:17:02</nova:creationTime> <nova:flavor name="kudu.test.dedicated.tiny"> <nova:memory>4096</nova:memory> <nova:disk>10</nova:disk> <nova:swap>0</nova:swap> <nova:ephemeral>0</nova:ephemeral> <nova:vcpus>2</nova:vcpus> </nova:flavor> <nova:owner> <nova:user uuid="91e83343e9834c8ba0172ff369c8acac">admin</nova:user> <nova:project uuid="b91520cff5bd45c59a8de07c38641582">admin</nova:project> </nova:owner> <nova:root type="image" uuid="5ec18792-05f2-4bce-b5d8-cb3480cbb35a"/> </nova:instance> </metadata> <memory unit='KiB'>4194304</memory> <currentMemory unit='KiB'>4194304</currentMemory> <vcpu placement='static'>2</vcpu> <cputune> <shares>2048</shares> <vcpupin vcpu='0' cpuset='9'/> <vcpupin vcpu='1' cpuset='37'/> <emulatorpin cpuset='9,37'/> </cputune> <numatune> <memory mode='strict' nodeset='0'/> <memnode cellid='0' mode='strict' nodeset='0'/> </numatune> <resource> <partition>/machine</partition> </resource> <sysinfo type='smbios'> <system> <entry name='manufacturer'>RDO</entry> <entry name='product'>OpenStack Compute</entry> <entry name='version'>18.2.0-1.el7</entry> <entry name='serial'>00000000-0000-0000-0000-0cc47aa482cc</entry> <entry name='uuid'>5d8f6cc1-e716-4f4d-ac55-df24d4cffd38</entry> <entry name='family'>Virtual Machine</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-i440fx-rhel7.6.0'>hvm</type> <boot dev='hd'/> <smbios mode='sysinfo'/> </os> <features> <acpi/> <apic/> </features> <cpu mode='custom' match='exact' check='full'> <model fallback='forbid'>Skylake-Client-IBRS</model> <vendor>Intel</vendor> <topology sockets='1' cores='1' threads='2'/> <feature policy='require' name='ss'/> <feature policy='require' name='hypervisor'/> <feature policy='require' name='tsc_adjust'/> <feature policy='require' name='stibp'/> <feature policy='require' name='ssbd'/> <feature policy='require' name='pdpe1gb'/> <feature policy='disable' name='mpx'/> <feature policy='disable' name='xsavec'/> <feature policy='disable' name='xgetbv1'/> <numa> <cell id='0' cpus='0-1' memory='4194304' unit='KiB'/> </numa> </cpu> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> </clock> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/libexec/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2' cache='none'/> <source file='/var/lib/nova/instances/5d8f6cc1-e716-4f4d-ac55-df24d4cffd38/disk'/> <backingStore type='file' index='1'> <format type='raw'/> <source file='/var/lib/nova/instances/_base/4cc6eebe175e35178cb81853818a1eb103cea937'/> <backingStore/> </backingStore> <target dev='vda' bus='virtio'/> <alias name='virtio-disk0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </disk> <controller type='usb' index='0' model='piix3-uhci'> <alias name='usb'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/> </controller> <controller type='pci' index='0' model='pci-root'> <alias name='pci.0'/> </controller> <interface type='bridge'> <mac address='fa:16:3e:45:ba:a1'/> <source bridge='qbr2a0abb35-29'/> <target dev='tap2a0abb35-29'/> <model type='virtio'/> <mtu size='1500'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> <serial type='pty'> <source path='/dev/pts/14'/> <log file='/var/lib/nova/instances/5d8f6cc1-e716-4f4d-ac55-df24d4cffd38/console.log' append='off'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> <alias name='serial0'/> </serial> <console type='pty' tty='/dev/pts/14'> <source path='/dev/pts/14'/> <log file='/var/lib/nova/instances/5d8f6cc1-e716-4f4d-ac55-df24d4cffd38/console.log' append='off'/> <target type='serial' port='0'/> <alias name='serial0'/> </console> <input type='tablet' bus='usb'> <alias name='input0'/> <address type='usb' bus='0' port='1'/> </input> <input type='mouse' bus='ps2'> <alias name='input1'/> </input> <input type='keyboard' bus='ps2'> <alias name='input2'/> </input> <graphics type='vnc' port='5901' autoport='yes' listen='192.168.1.53'> <listen type='address' address='192.168.1.53'/> </graphics> <video> <model type='cirrus' vram='16384' heads='1' primary='yes'/> <alias name='video0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> </video> <memballoon model='virtio'> <stats period='10'/> <alias name='balloon0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> </memballoon> </devices> <seclabel type='dynamic' model='dac' relabel='yes'> <label>+42436:+42436</label> <imagelabel>+42436:+42436</imagelabel> </seclabel> </domain> However I do can see the nvme if I use a flavor without cpu pinning. Has anyone seen this before? Thank you NOTICE Please consider the environment before printing this email. This message and any attachments are intended for the addressee named and may contain legally privileged/confidential/copyright information. If you are not the intended recipient, you should not read, use, disclose, copy or distribute this communication. If you have received this message in error please notify us at once by return email and then delete both messages. We accept no liability for the distribution of viruses or similar in electronic communications. This notice should not be removed.

1 0

[Tripleo][Octavia][OVN] Overcloud deployment Issues with Octavia
by Rabi Mishra 23 Apr '19

23 Apr '19

Hi All, For some Octavia testing with heat, I tried deploying 3+1 node overcloud with Octavia environment and THT master. As expected, deployment finished successfully. However, when trying to do some Octavia API calls (ex. list laodbalancers etc), it was returning (503) Service Unavailable error. Checking the logs of octavia_api container (though the container was running), it was in a continous loop. I have captured some of these issues (as described below) in https://bugs.launchpad.net/tripleo/+bug/1825146 too. The intent of this mail is to bring it to the notice of the respective teams and understand the best possible way to fix these. 1. Incorrect config setting for provider_drivers. As of https://github.com/openstack/puppet-tripleo/commit/97c46ca76803fb238279d245…, we added OVN provider driver support for Octavia. However, the config option seems to be set incorrectly which resulted in octavia-api failing to start. 2. ovn_nb_connection not in octavia config It seems ovn provider driver (part of networking-ovn) connects to ovn nb database as part of initialization. However, we don't seem to configure connection string in octavia.conf and it tries to connect to 127.0.0.1 and fails. I've already proposed a few patches[1] to fix the above two issues. 3. Missing Octavia Driver Agent Octavia had added a driver-agent controller process in Stein (for provider drivers to communicate with the octavia for status and stats update). This process is expected to be collocated with octavia-api and creates a pair of unix domain sockets for communication that the drivers use. We don't seem to have this implemented in THT yet. Is this something planned and someone working on it? AFAICT, we can't enable/use the OVN provider driver without adding this service/container? Adding support for this in THT also seems little complex and probably can be done in few different ways. Is there a standard design pattern we use for these kind of requirements in Tripleo? If we already have an agreed plan on how to do it, please ignore below. I can only think of the few options below. Probably there are better ones too:) a. New OctivaDriverAgent Service. As the process is expected to be co-llocated with octavia-api to share the domain sockets (we can share /var/run/octavia from host on both containers), I'm not sure if this would be a good design though. b. As a sidecar of octavia_api Not a new service, but as a sidecar container for octavia-api (started when api starts, do we've something like this already there for other services?). Ideas/comments welcome:) [1] https://review.opendev.org/#/q/status:open+branch:master+topic:bug/1825146 -- Regards, Rabi Mishra

2 1

[Tacker] Weekly meeting and vPTG
by Dharmendra Kushwaha 23 Apr '19

23 Apr '19

Hi Team, Due to the summit, most of members busy in travel things, so canceling today's and next week IRC meeting. We will resume Tacker weekly meeting on 7th May. Due to U.S. visa issue, I am not able to travel this time, so as we agreed to cancel physical PTG, and will finalize vPTG schedule in weekly meeting of 7th May. I Will be available on Tacker channel, feel free to ping me for any discussion. Thanks & Regards Dharmendra Kushwaha

1 0

[oslo] No meeting next two weeks
by Ben Nemec 22 Apr '19

22 Apr '19

God dag, Norsk. As discussed in the meeting this morning, there will be no Oslo meeting for the next two weeks (Apr. 29 and May 6). The former is during Summit, and the latter will be the first work day after the PTG. As always, if anything comes up during that time you don't have to wait for a meeting to bring it to our attention. The mailing list is always open. -Ben

1 0

OpenStack Hound code search with OpenDev repos
by Boden Russell 22 Apr '19

22 Apr '19

Maybe I missed this, but are there some changes necessary for the OpenStack Hound (code search) service [1] to work with OpenDev git repos? As of right now Hound [1] doesn't seem to be finding any repos; click the "Advanced" option and see the empty list of repos. Thanks [1] http://codesearch.openstack.org/

3 2

[tripleo] Summit / PTG link
by Emilien Macchi 22 Apr '19

22 Apr '19

All (or almost) what you need to know about TripleO at Open Infra summit & PTG: I created a Google Calendar for TripleO related sessions (including PTG): web format: https://calendar.google.com/calendar/embed?src=es78vnjv5galctjgo1vsi3d7bo%4… ical format: https://calendar.google.com/calendar/ical/es78vnjv5galctjgo1vsi3d7bo%40grou… Our PTG etherpads are here (topics + agenda) https://etherpad.openstack.org/p/tripleo-train-topics https://etherpad.openstack.org/p/tripleo-ptg-train Don't miss the team picture on Thursday, 11am. Safe travels, and see you next week! -- Emilien Macchi

1 0

[placement][nova][ptg] Resourceless trait filters
by Chris Dent 22 Apr '19

22 Apr '19

>From the etherpad [1] (the cross project one): * This came up with the effort to add a multiattach capability filter, at https://review.openstack.org/#/c/645316/1/nova/compute/api.py@1098 * The problem: we want to be able to request allocation candidates filtered by a trait that exists on the root provider; but * (a) the root provider may provide no resources (eventually - e.g. CPU/mem in NUMA providers, shared disk); and/or * (My brain hurts from the concept for a provider that provides nothing. Perhaps it provides something we aren't remembering to count?) * (b) we are using only numbered groups and would have to guess where to stuff the `required` - again probably based on looking for the VCPU/mem resources, which, as above, may wind up not on the root provider. This, basically, is how/when do we deal with resource providers that have traits but have no classes of inventory of their own (all of it is on their descendants). The "My brain hurts" comment above is mine. The discussion on the review above (good backgrounder for this thread) had little to do with the immediate need of the proposed feature. It was a concern about how things will work in the future. So there are two salient questions (and presumably plenty of other sidebars): * How should it work? * What is the timeline for when it needs to work? [1] https://etherpad.openstack.org/p/ptg-train-xproj-nova-placement -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

6 19