August 2019 - openstack-discuss - lists.openstack.org

[placement] update 19-31
by Chris Dent 09 Aug '19

09 Aug '19

HTML: https://anticdent.org/placement-update-19-31.html Pupdate 19-31. No bromides today. # Most Important Same as last week: The main things on the Placement radar are implementing Consumer Types and cleanups, performance analysis, and documentation related to nested resource providers. We need to decide how much of a priority consumer types support is. I've taken the task of asking around with the various interested parties. # What's Changed * A more complex nested topology is now being used in the nested-perfload check job, and both that and the non-nest perfload run [apache benchmark](https://review.opendev.org/#/c/673540/) at the end. When you make changes you can have a look at the results of the `placement-perfload` and `placement-nested-perfload` gate jobs to see if there has been a performance impact. Keep in mind the numbers are only a guide. The performance characteristics of VMs from different CI providers varies _wildly_. * A stack of several performance related improvements has merged, with still more to come. I've written a separate [Placement Performance Analysis](https://anticdent.org/placement-performance-analysis.html) that summarizes some of the changes. Many of these may be useful for other services. Each iteration reveals another opportunity. * In some environments placement will receive a URL of '' when '/' is expected. Auth handling for version control needs to [handle this](https://review.opendev.org/674543). * osc-placmeent 1.6.0 is in the process of being [released](https://review.opendev.org/675311). # Stories/Bugs (Numbers in () are the change since the last pupdate.) There are 22 (-1) stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement). 0 (0) are [untagged](https://storyboard.openstack.org/#!/worklist/580). 3 (0) are [bugs](https://storyboard.openstack.org/#!/worklist/574). 4 (-1) are [cleanups](https://storyboard.openstack.org/#!/worklist/575). 11 (0) are [rfes](https://storyboard.openstack.org/#!/worklist/594). 4 (0) are [docs](https://storyboard.openstack.org/#!/worklist/637). If you're interested in helping out with placement, those stories are good places to look. * Placement related nova [bugs not yet in progress](https://goo.gl/TgiPXb) on launchpad: 17 (0). * Placement related nova [in progress bugs](https://goo.gl/vzGGDQ) on launchpad: 5 (1). # osc-placement osc-placement is currently behind by 12 microversions. * <https://review.opendev.org/666542> Add support for multiple member_of. There's been some useful discussion about how to achieve this, and a consensus has emerged on how to get the best results. * <https://review.opendev.org/640898> Adds a new '--amend' option which can update resource provider inventory without requiring the user to pass a full replacement for inventory. This has been broken up into three patches to help with review. # Main Themes ## Consumer Types Adding a type to consumers will allow them to be grouped for various purposes, including quota accounting. * <https://review.opendev.org/#/q/topic:bp/support-consumer-types> A WIP, as microversion 1.37, has started. ## Cleanup Cleanup is an overarching theme related to improving documentation, performance and the maintainability of the code. The changes we are making this cycle are fairly complex to use and are fairly complex to write, so it is good that we're going to have plenty of time to clean and clarify all these things. As said above, there's lots of performance work in progress. We'll need to make a similar effort with regard to docs. One outcome of this work will be something like a _Deployment Considerations_ document to help people choose how to tweak their placement deployment to match their needs. The simple answer is use more web servers and more database servers, but that's often very wasteful. # Other Placement Miscellaneous changes can be found in [the usual place](https://review.opendev.org/#/q/project:openstack/placement+status:op…. There are two [os-traits changes](https://review.opendev.org/#/q/project:openstack/os-traits+status:… being discussed. And zero [os-resource-classes changes](https://review.opendev.org/#/q/project:openstack/os-resource-class…. # Other Service Users New discoveries are added to the end. Merged stuff is removed. Anything that has had no activity in 4 weeks has been removed. * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.opendev.org/659233> Cyborg: Placement report * <https://review.opendev.org/662229> helm: add placement chart * <https://review.opendev.org/634551> libvirt: report pmem namespaces resources by provider tree * <https://review.opendev.org/660852> Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI * <https://review.opendev.org/670112> Nova: WIP: Add a placement audit command * <https://review.opendev.org/671312> blazar: Fix placement operations in multi-region deployments * <https://review.opendev.org/671793> Nova: libvirt: Start reporting PCPU inventory to placement A part of <https://review.opendev.org/#/q/topic:bp/cpu-resources * <https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports> Nova: support move ops with qos ports * <https://review.opendev.org/666202> Blazar: Create placement client for each request * <https://review.opendev.org/667952> nova: Support filtering of hosts by forbidden aggregates * <https://review.opendev.org/669079> blazar: Send global_request_id for tracing calls * <https://review.opendev.org/668252> Nova: Update HostState.\*\_allocation_ratio earlier * <https://review.opendev.org/670696> tempest: Add placement API methods for testing routed provider nets * <https://review.opendev.org/672678> openstack-helm: Build placement in OSH-images * <https://review.opendev.org/674129> Correct global_request_id sent to Placement * <https://review.opendev.org/#/q/topic:bp/cross-cell-resize> Nova: cross cell resize * <https://review.opendev.org/675506> Watcher: Remove resource used fields from ComputeNode * <https://review.opendev.org/674524> Nova: Scheduler translate properties to traits # End Somewhere in this performance work is a lesson for life: Every time I think we've reached the bottom of the "easy stuff", I find yet another bit of easy stuff. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

1 0

[monasca] Enable 'Review-Priority' voting in Gerrit
by Doug Szumski 09 Aug '19

09 Aug '19

A number of projects have added a 'Review-Priority' vote alongside the usual 'Code-Review' and 'Workflow' radio buttons [1]. The idea is to make it easier to direct reviewer attention towards high priority patches. As such, Gerrit dashboards can filter based on the 'Review-Priority' rating. Please vote on whether to enable this feature in Monasca here: https://review.opendev.org/#/c/675574 [1]: http://lists.openstack.org/pipermail/openstack-discuss/2018-December/001304…

2 1

inaccessibility of instances page in my openstack installation
by tim.j.culhane＠gmail.com 09 Aug '19

09 Aug '19

Hi, I'm a blind programmer and we use Openstack in our organisation to create and manage servers for development. In the past when I wanted to launch or delete an instance I'd go to the Instances page. Just above the table listing the current instances I have, would be a series of buttons which allowed you to launch, delete instances or carry out more actions. When using Firefox I can access these items via the keyboard by hitting enter on them. Up to very recently this was also the case in Chrome (which is my preferred browser). However, in the last week of so I've noticed that I can no longer use the keyboard to interact with these controls, you need to click them with the mouse. I'm not sure if this is an issue with Chrome or with Openstack. Has there been any recent changes to Openstack which might explain this? Many thanks, Tim Instance ID = Filter Launch Instance Delete Instances More Actions

2 5

[nova][entropy] what are your rate limits??
by Jordan Ansell 09 Aug '19

09 Aug '19

Hello Openstack Discuss, I am doing some investigation into instance entropy and was wondering what settings others are using with regard to rate limiting entropy supplied by the hypervisor. Specifically, we're adding the "hw_rng:allowed=True" nova flavor property to pass libvirt the relevant config, but need to decide the appropriate rate limiting settings to prevent instances from being greedy with entropy but still retain a comfortable level for themselves. I've done some experimenting (100 bytes/s is possibly a minimum, but still allows a comfortable value of ~1000 for free entropy in instances) I'm also curious to hear other's experiences when it comes to entropy in Openstack: * What sources of entropy did you use in the hypervisor? * Issues you've faced which was caused by insufficient entropy (instance or host) Note, this is for a public cloud scenario, should that impact any suggestions you have. Regards, Jordan

2 1

[keystone] [stein] user_enabled_emulation config problem
by Radosław Piliszek 09 Aug '19

09 Aug '19

Hello all, I have an issue using user_enabled_emulation with my LDAP solution. I set: user_tree_dn = ou=Users,o=UCO user_objectclass = inetOrgPerson user_id_attribute = uid user_name_attribute = uid user_enabled_emulation = true user_enabled_emulation_dn = cn=Users,ou=Groups,o=UCO user_enabled_emulation_use_group_config = true group_tree_dn = ou=Groups,o=UCO group_objectclass = posixGroup group_id_attribute = cn group_name_attribute = cn group_member_attribute = memberUid group_members_are_ids = true Keystone properly lists members of the Users group but they all remain disabled. Did I misinterpret something? Kind regards, Radek

2 3

[nova] Intermittent gate failures in functional tests
by Matt Riedemann 09 Aug '19

09 Aug '19

In case you're seeing a bunch of nova versioned notification functional tests failing this week, it's being tracked [1] and there is a skip patch approved [2] to hopefully resolve it while a long-term fix is worked. [1] http://status.openstack.org/elastic-recheck/#1839515 [2] https://review.opendev.org/#/c/675417 -- Thanks, Matt

1 0

[nova] API updates week 19-32
by Ghanshyam Mann 08 Aug '19

08 Aug '19

Hello Everyone, Please find the Nova API updates of this week. API Related BP : ============ COMPLETED: 1. Support adding description while locking an instance: - https://blueprints.launchpad.net/nova/+spec/add-locked-reason 2. Add host and hypervisor_hostname flag to create server - https://blueprints.launchpad.net/nova/+spec/add-host-and-hypervisor-hostnam… Code Ready for Review: ------------------------------ 1. Specifying az when restore shelved server - Topic: https://review.opendev.org/#/q/topic:bp/support-specifying-az-when-restore-… - Weekly Progress: It is ready for re-review. Patch is updated. 2. Nova API cleanup - Topic: https://review.opendev.org/#/q/topic:bp/api-consistency-cleanup+(status:ope… - Weekly Progress: This is on runway. stephenfin has +2 on nova patch. I will work on novalcient and osc patch tomorrow. 3. Show Server numa-topology - Topic: https://review.opendev.org/#/q/topic:bp/show-server-numa-topology+(status:o… - Weekly Progress: Under review. 4. Nova API policy improvement - Topic: https://review.openstack.org/#/q/topic:bp/policy-default-refresh+(status:op… - Weekly Progress: First set of os-service API policy series is ready to review - https://review.opendev.org/#/c/648480/7 Specs are merged and code in-progress: ------------------------------ ------------------ 5. Detach and attach boot volumes: - Topic: https://review.openstack.org/#/q/topic:bp/detach-boot-volume+(status:open+O… - Weekly Progress: No Progress. Patches are in merge conflict. Spec Ready for Review: ----------------------------- 1. Support for changing deleted_on_termination after boot -Spec: https://review.openstack.org/#/c/580336/ - Weekly Progress: No update this week. Pending on Lee Yarwood proposal after PTG discussion. 3. Support delete_on_termination in volume attach api -Spec: https://review.openstack.org/#/c/612949/ - Weekly Progress: No updates this week. matt recommend to merging this with 580336 which is pending on Lee Yarwood proposal. Previously approved Spec needs to be re-proposed for Train: --------------------------------------------------------------------------- 1. Servers Ips non-unique network names : - https://blueprints.launchpad.net/nova/+spec/servers-ips-non-unique-network-… - https://review.openstack.org/#/q/topic:bp/servers-ips-non-unique-network-na… - I remember I planned this to re-propose but could not get time. If anyone would like to help on this please repropose. otherwise I will start this in U cycle. 2. Volume multiattach enhancements: - https://blueprints.launchpad.net/nova/+spec/volume-multiattach-enhancements - https://review.openstack.org/#/q/topic:bp/volume-multiattach-enhancements+(… - This also need volutneer - http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007411.html Others: 1. Add API ref guideline for body text - 2 api-ref are left to fix. Bugs: ==== No progress report in this week. NOTE- There might be some bug which is not tagged as 'api' or 'api-ref', those are not in the above list. Tag such bugs so that we can keep our eyes.

1 0

[tripleo] Proposing Kevin Carter (cloudnull) for tripleo-ansible core
by Alex Schultz 08 Aug '19

08 Aug '19

Hey folks, I'd like to propose Kevin as a core for the tripleo-ansible repo (e.g. tripleo-ansible-core). He has made excellent progress centralizing our ansible roles and improving the testing around them. Please reply with your approval/objections. If there are no objections, we'll add him to tripleo-ansible-core next Friday Aug 2, 2019. Thanks, -Alex

11 11

[nova/neutron/openvswitch] "No such device" after failed live migrations
by Nicolas Ghirlanda 08 Aug '19

08 Aug '19

Hello all, after a misconfigured setup of a couple of VMs, the live migration of those VMs failed. Now we have some "No such device" listings in openvswitch for those VMs on some compute nodes. (openvswitch-vswitchd)[root@ewos1-com1-prod /]# ovs-vsctl show b5034213-9b15-45f5-8ce0-edcf32d16c57 Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure Port "qvo35487c68-23" tag: 5 Interface "qvo35487c68-23" * Port "qvo5b1aac7e-d4"** ** Interface "qvo5b1aac7e-d4"** ** error: "could not open network device qvo5b1aac7e-d4 (No such device)"** ** Port "qvod9cdff27-9c"** ** Interface "qvod9cdff27-9c"** ** error: "could not open network device qvod9cdff27-9c (No such device)"* Port "qvoed2da602-02" tag: 32 Interface "qvoed2da602-02" Port "qvoa63378f6-d9" for example qvo5b1aac7e-d4 (openvswitch-vswitchd)[root@ewos1-com1-prod /]# ovs-vsctl list Interface ca7e771b-88a0-4cf6-b9be-6be77816baff _uuid : ca7e771b-88a0-4cf6-b9be-6be77816baff admin_state : [] bfd : {} bfd_status : {} cfm_fault : [] cfm_fault_status : [] cfm_flap_count : [] cfm_health : [] cfm_mpid : [] cfm_remote_mpids : [] cfm_remote_opstate : [] duplex : [] error : "could not open network device qvo5b1aac7e-d4 (No such device)" external_ids : {attached-mac="fa:16:3e:4b:f0:4b", iface-id="5b1aac7e-d45b-4a40-8f30-1275b07ffc0b", iface-status=active, vm-uuid="c3fbcc6b-2dfe-4aa1-82bd-522b161a37a9"} ifindex : [] ingress_policing_burst: 0 ingress_policing_rate: 0 lacp_current : [] link_resets : [] link_speed : [] link_state : [] lldp : {} mac : [] mac_in_use : [] mtu : [] mtu_request : [] name : "qvo5b1aac7e-d4" ofport : -1 ofport_request : [] options : {} other_config : {} statistics : {} status : {} type : "" there are no tap devices or bridge devices on the compute node, but still the entry in openvswitch root@computenode5:~# brctl show | grep 5b1aac7e root@computenode5:~# Is that an issue? if yes, should we remove the ports manually or reboot the compute nodes? Could that lead to networking issues? kind regards Nicolas ... --

1 0

[neutron] OpenvSwitch firewall sctp getting dropped
by thuanlk＠viettel.com.vn 08 Aug '19

08 Aug '19

I have installed Openstack Queens on CentOs 7 with OvS and I recently used the native openvswitch firewall to implement SecusiryGroup. The native OvS firewall seems to work just fine with TCP/UDP traffic but it does not forward any SCTP traffic going to the VMs no matter how I change the security groups, But it run if i disable port security completely or use iptables_hybrid firewall driver. What do I have to do to allow SCTP packets to reach the VMs?

4 7