- openstack-discuss - lists.openstack.org

[Security SIG] Weekly Newsletter Aug 08 2019
by Gage Hugo 09 Aug '19

09 Aug '19

Of note, OSSA-2019-003 was released this week #Week of: 08 Aug 2019 - Security SIG Meeting Info: http://eavesdrop.openstack.org/#Security_SIG_meeting - Weekly on Thursday at 1500 UTC in #openstack-meeting - Agenda: https://etherpad.openstack.org/p/security-agenda - https://security.openstack.org/ - https://wiki.openstack.org/wiki/Security-SIG #Meeting Notes - Summary: http://eavesdrop.openstack.org/meetings/security/2019/security.2019-08-08-1… - Announced OSSA-2019-003 release on Tuesday August 06th 2019 - Image Encryption Spec - image encryption spec for nova unlikely to get a freeze exception - Will likely polish it up, target an early 'U' release # VMT Reports - A full list of publicly marked security issues can be found here: https://bugs.launchpad.net/ossa/ - OSSA-2019-003 was released this week: https://security.openstack.org/ossa/OSSA-2019-003.html

1 0

[keystone] Keystone Team Update - Week of 5 August 2019
by Colleen Murphy 09 Aug '19

09 Aug '19

# Keystone Team Update - Week of 5 August 2019 ## News ### CI instability update To follow up from this topic from last week, we came up with a solution[1][2] that at least reduces the size of the unit test log files to an acceptable, non-browser-crashing size. Unfortunately that didn't seem to be the root cause of the frequent timeouts, so it's unclear (to me, at least) whether the issue stems from a problem in our unit tests or if we're just getting unlucky with noisy neighbors. It could be as simple as needing to raise the timeout to account for all the additional protection tests we've added in the past few months. [1] https://review.opendev.org/673932 [2] https://review.opendev.org/673933 ### Call for help We need help completing the system-scope and default roles policy updates[3][4] before the end of the cycle, as operators cannot safely enable [oslo_policy]/enforce_scope until all of them are completed. For the most part, the task involves updating the scope_types option in the policy and adding a ton of unit tests. The already completed work[5][6] can serve as an example for what's needed. [3] https://bugs.launchpad.net/keystone/+bugs?field.tag=default-roles [4] https://bugs.launchpad.net/keystone/+bugs?field.tag=system-scope [5] https://bugs.launchpad.net/keystone/+bugs?field.status%3Alist=FIXRELEASED&f… [6] https://bugs.launchpad.net/keystone/+bugs?field.status%3Alist=FIXRELEASED&f… ### PTG attendance and Forum Planning Based on our poll[7] it's looking like there are not enough keystone-minded people planning to attend the Shanghai PTG to warrant requesting a room, so I will likely tell Kendall that we don't need a room unless something changes very soon. Even if you won't be attending, please use that etherpad to add topics you would like to see discussed at the Forum. We can use those discussions as a jumping off point for our pre- and post-PTG virtual gatherings. [7] https://etherpad.openstack.org/p/keystone-shanghai-ptg ## Office Hours When there are topics to cover, the keystone team holds office hours on Tuesdays at 17:00 UTC. We will skip next week's office hours since we don't have a topic planned. Add topics you would like to see covered during office hours to the etherpad: https://etherpad.openstack.org/p/keystone-office-hours-topics ## Open Specs Train specs: https://bit.ly/2uZ2tRl Ongoing specs: https://bit.ly/2OyDLTh ## Recently Merged Changes Search query: https://bit.ly/2pquOwT We merged 9 changes this week. ## Changes that need Attention Search query: https://bit.ly/2tymTje There are 43 changes that are passing CI, not in merge conflict, have no negative reviews and aren't proposed by bots. ### Priority Reviews * Train Roadmap Stories - https://review.opendev.org/#/q/topic:bug/1818734 (system scope and default roles) - https://review.opendev.org/#/q/topic:implement-default-roles+is:open (system scope and default roles) - https://review.opendev.org/#/q/topic:bp/whitelist-extension-for-app-creds+i… (application credential access rules) - https://review.opendev.org/672120 (caching guide) - https://review.opendev.org/#/q/project:openstack/oslo.limit+topic:rewrite+i… (oslo.limit) * Needs Discussion - https://review.opendev.org/618144 (Reparent Projects) - https://review.opendev.org/674940 (Make policy deprecation reasons less verbose) - https://review.opendev.org/675303 (Allows LDAP extra attributes to be exposed to the end user) ## Bugs This week we opened 4 new bugs and closed 4. Bugs opened (4) Bug #1839393 (keystone:Low) opened by Matthew Thode https://bugs.launchpad.net/keystone/+bug/1839393 Bug #1839133 (keystone:Undecided) opened by Radosław Piliszek https://bugs.launchpad.net/keystone/+bug/1839133 Bug #1839441 (keystone:Undecided) opened by Jose Castro Leon https://bugs.launchpad.net/keystone/+bug/1839441 Bug #1839577 (keystone:Undecided) opened by Adrian Turjak https://bugs.launchpad.net/keystone/+bug/1839577 Bugs fixed (4) Bug #1773967 (keystone:High) fixed by Jose Castro Leon https://bugs.launchpad.net/keystone/+bug/1773967 Bug #1838592 (keystone:High) fixed by Colleen Murphy https://bugs.launchpad.net/keystone/+bug/1838592 Bug #1709344 (keystone:Low) fixed by Adrian Turjak https://bugs.launchpad.net/keystone/+bug/1709344 Bug #1837741 (oslo.policy:High) fixed by no one https://bugs.launchpad.net/oslo.policy/+bug/1837741 ## Milestone Outlook https://releases.openstack.org/train/schedule.html Feature proposal freeze is NEXT WEEK (August 12-August 16). Spec implementations that are not submitted or still in a WIP state by the end of the week will need to be postponed until next cycle unless we agree on an exception. Code implementing system scope and default roles policy work will be accepted until feature freeze week (September 9-September 13). If you are able, please help by picking up some of these tasks[7][8] or helping to review them (thanks Vishakha for jumping on the endpoint groups policies!). Final release of non-client libraries is the week of September 2 which allows us about three weeks to both implement and review library changes needed for this cycle. [7] https://bugs.launchpad.net/keystone/+bugs?field.tag=default-roles [8] https://bugs.launchpad.net/keystone/+bugs?field.tag=system-scope ## Help with this newsletter Help contribute to this newsletter by editing the etherpad: https://etherpad.openstack.org/p/keystone-team-newsletter

1 0

[cyborg] [release] Process to discontinue os-acc
by Nadathur, Sundar 09 Aug '19

09 Aug '19

Hi, A project called os-acc [1] was created in Stein cycle based on an expectation that it will be used for Cyborg - Nova integration. It is not relevant anymore and we have no plans to support it in Train. It needs to be discontinued. What is the process for doing that? A part of that is presumably to update or delete the os-acc release yaml [2]. What else is needed? [1] https://opendev.org/openstack/os-acc/ [2] https://opendev.org/openstack/releases/src/branch/master/deliverables/train… Regards, Sundar

2 1

[ironic][ops] Taking ironic nodes out of production
by Arne Wiebalck 09 Aug '19

09 Aug '19

Dear all, One of the discussions at the PTG in Denver raised the need for a mechanism to take ironic nodes out of production (a task for which the currently available 'maintenance' flag does not seem appropriate [1]). The use case there is an unhealthy physical node in state 'active', i.e. associated with an instance. The request is then to enable an admin to mark such a node as 'faulty' or 'in quarantine' with the aim of not returning the node to the pool of available nodes once the hosted instance is deleted. A very similar use case which came up independently is node retirement: it should be possible to mark nodes ('active' or not) as being 'up for retirement' to prepare the eventual removal from ironic. As in the example above, ('active') nodes marked this way should not become eligible for instance scheduling again, but automatic cleaning, for instance, should still be possible. In an effort to cover these use cases by a more general "quarantine/retirement" feature: - are there additional use cases which could profit from such a "take a node out of service" mechanism? - would these use cases put additional constraints on how the feature should look like (e.g.: "should not prevent cleaning") - are there other characteristics such a feature should have (e.g.: "finding these nodes should be supported by the cli") Let me know if you have any thoughts on this. Cheers, Arne [1] https://etherpad.openstack.org/p/DEN-train-ironic-ptg, l. 360

5 7

[goal][python3] Train unit tests weekly update (goal-5)
by Corey Bryant 09 Aug '19

09 Aug '19

This is the goal-5 weekly update for the "Update Python 3 test runtimes for Train" goal [1]. There are 5 weeks remaining for completion of Train community goals [2]. == How can you help? == If your project has failing tests please take a look and help fix. Python 3.7 unit tests will be self-testing in Zuul. If your project has patches with successful tests please help get them merged. Failing patches: https://review.openstack.org/#/q/topic:python3-train +status:open+(+label:Verified-1+OR+label:Verified-2+) Open patches needing reviews: https://review.openstack.org/#/q/topic:python3 -train+is:open Patch automation scripts needing review: https://review.opendev.org/#/c/666934 == Ongoing Work == Today I reached out to all PTLs who have projects with failing patches, to ask for their help with getting tests to pass. == Completed Work == All patches have been submitted to all applicable projects for this goal. Merged patches: https://review.openstack.org/#/q/topic:python3-train +is:merged == What's the Goal? == To ensure (in the Train cycle) that all official OpenStack repositories with Python 3 unit tests are exclusively using the 'openstack-python3-train-jobs' Zuul template or one of its variants (e.g. 'openstack-python3-train-jobs-neutron') to run unit tests, and that tests are passing. This will ensure that all official projects are running py36 and py37 unit tests in Train. For complete details please see [1]. == Reference Material == [1] Goal description: https://governance.openstack.org/tc/goals/train/ python3-updates.html [2] Train release schedule: https://releases.openstack.org/train /schedule.html (see R-5 for "Train Community Goals Completed") Storyboard: https://storyboard.openstack.org/#!/story/2005924 Porting to Python 3.7: https://docs.python.org/3/whatsnew/3.7.html#porting-to-python-3-7 Python Update Process: https://opendev.org/openstack/governance/src/branch/master/resolutions/2018… Train runtimes: https://opendev.org/openstack/governance/src/branch/master/reference/runtim… train.rst Thanks, Corey

1 0

[sdk] Proposing Eric Fried for openstacksdk-core
by Monty Taylor 09 Aug '19

09 Aug '19

Hey all, I'd like to propose Eric Fried be made core on SDK. This is slightly different than a normal core proposal, so I'd like to say a few more words about it than normal- largely because I think it's a pattern we might want to explore in SDK land. Eric is obviously super smart and capable - he's PTL of Nova after all. He's one of the few people in OpenStack that has a handle on version discovery, having helped write the keystoneauth support for it. And he's core on os-service-types, which is another piece of client-side arcana. However, he's a busy human, what with being Nova core, and his interaction with SDK has been limited to the intersection of it needed for Nova integration. I don't expect that to change. Basically, as it stands now, Eric only votes on SDK patches that have impact on the use of SDK in Nova - but when he does they are thorough reviews and they indicate "this makes things better for Nova". So I'd like to start recognizing such a vote. As our overall numbers diminish, I think we need to be more efficient with the use of our human time - and along with that we need to find new ways to trust each other to act on behalf of the project. I'd like to give a stab at doing that here. Thoughts? Monty

9 9

[kolla][nova][cinder] Got Gateway-Timeout error on VM evacuation if it has volume attached.
by Eddie Yen 09 Aug '19

09 Aug '19

Hi everyone, I met an issue when try to evacuate host. The platform is stable/rocky and using kolla-ansible to deploy. And all storage backends are connected to Ceph. Before I try to evacuate host, the source host had about 24 VMs running. When I shutdown the node and execute evacuation, there're few VMs failed. The error code is 504. Strange is those VMs are all attach its own volume. Then I check nova-compute log, a detailed error has pasted at below link; https://pastebin.com/uaE7YrP1 Does anyone have any experience with this? I googled but no enough information about this. Thanks!

5 14

[placement] update 19-31
by Chris Dent 09 Aug '19

09 Aug '19

HTML: https://anticdent.org/placement-update-19-31.html Pupdate 19-31. No bromides today. # Most Important Same as last week: The main things on the Placement radar are implementing Consumer Types and cleanups, performance analysis, and documentation related to nested resource providers. We need to decide how much of a priority consumer types support is. I've taken the task of asking around with the various interested parties. # What's Changed * A more complex nested topology is now being used in the nested-perfload check job, and both that and the non-nest perfload run [apache benchmark](https://review.opendev.org/#/c/673540/) at the end. When you make changes you can have a look at the results of the `placement-perfload` and `placement-nested-perfload` gate jobs to see if there has been a performance impact. Keep in mind the numbers are only a guide. The performance characteristics of VMs from different CI providers varies _wildly_. * A stack of several performance related improvements has merged, with still more to come. I've written a separate [Placement Performance Analysis](https://anticdent.org/placement-performance-analysis.html) that summarizes some of the changes. Many of these may be useful for other services. Each iteration reveals another opportunity. * In some environments placement will receive a URL of '' when '/' is expected. Auth handling for version control needs to [handle this](https://review.opendev.org/674543). * osc-placmeent 1.6.0 is in the process of being [released](https://review.opendev.org/675311). # Stories/Bugs (Numbers in () are the change since the last pupdate.) There are 22 (-1) stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement). 0 (0) are [untagged](https://storyboard.openstack.org/#!/worklist/580). 3 (0) are [bugs](https://storyboard.openstack.org/#!/worklist/574). 4 (-1) are [cleanups](https://storyboard.openstack.org/#!/worklist/575). 11 (0) are [rfes](https://storyboard.openstack.org/#!/worklist/594). 4 (0) are [docs](https://storyboard.openstack.org/#!/worklist/637). If you're interested in helping out with placement, those stories are good places to look. * Placement related nova [bugs not yet in progress](https://goo.gl/TgiPXb) on launchpad: 17 (0). * Placement related nova [in progress bugs](https://goo.gl/vzGGDQ) on launchpad: 5 (1). # osc-placement osc-placement is currently behind by 12 microversions. * <https://review.opendev.org/666542> Add support for multiple member_of. There's been some useful discussion about how to achieve this, and a consensus has emerged on how to get the best results. * <https://review.opendev.org/640898> Adds a new '--amend' option which can update resource provider inventory without requiring the user to pass a full replacement for inventory. This has been broken up into three patches to help with review. # Main Themes ## Consumer Types Adding a type to consumers will allow them to be grouped for various purposes, including quota accounting. * <https://review.opendev.org/#/q/topic:bp/support-consumer-types> A WIP, as microversion 1.37, has started. ## Cleanup Cleanup is an overarching theme related to improving documentation, performance and the maintainability of the code. The changes we are making this cycle are fairly complex to use and are fairly complex to write, so it is good that we're going to have plenty of time to clean and clarify all these things. As said above, there's lots of performance work in progress. We'll need to make a similar effort with regard to docs. One outcome of this work will be something like a _Deployment Considerations_ document to help people choose how to tweak their placement deployment to match their needs. The simple answer is use more web servers and more database servers, but that's often very wasteful. # Other Placement Miscellaneous changes can be found in [the usual place](https://review.opendev.org/#/q/project:openstack/placement+status:op…. There are two [os-traits changes](https://review.opendev.org/#/q/project:openstack/os-traits+status:… being discussed. And zero [os-resource-classes changes](https://review.opendev.org/#/q/project:openstack/os-resource-class…. # Other Service Users New discoveries are added to the end. Merged stuff is removed. Anything that has had no activity in 4 weeks has been removed. * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.opendev.org/659233> Cyborg: Placement report * <https://review.opendev.org/662229> helm: add placement chart * <https://review.opendev.org/634551> libvirt: report pmem namespaces resources by provider tree * <https://review.opendev.org/660852> Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI * <https://review.opendev.org/670112> Nova: WIP: Add a placement audit command * <https://review.opendev.org/671312> blazar: Fix placement operations in multi-region deployments * <https://review.opendev.org/671793> Nova: libvirt: Start reporting PCPU inventory to placement A part of <https://review.opendev.org/#/q/topic:bp/cpu-resources * <https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports> Nova: support move ops with qos ports * <https://review.opendev.org/666202> Blazar: Create placement client for each request * <https://review.opendev.org/667952> nova: Support filtering of hosts by forbidden aggregates * <https://review.opendev.org/669079> blazar: Send global_request_id for tracing calls * <https://review.opendev.org/668252> Nova: Update HostState.\*\_allocation_ratio earlier * <https://review.opendev.org/670696> tempest: Add placement API methods for testing routed provider nets * <https://review.opendev.org/672678> openstack-helm: Build placement in OSH-images * <https://review.opendev.org/674129> Correct global_request_id sent to Placement * <https://review.opendev.org/#/q/topic:bp/cross-cell-resize> Nova: cross cell resize * <https://review.opendev.org/675506> Watcher: Remove resource used fields from ComputeNode * <https://review.opendev.org/674524> Nova: Scheduler translate properties to traits # End Somewhere in this performance work is a lesson for life: Every time I think we've reached the bottom of the "easy stuff", I find yet another bit of easy stuff. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

1 0

[monasca] Enable 'Review-Priority' voting in Gerrit
by Doug Szumski 09 Aug '19

09 Aug '19

A number of projects have added a 'Review-Priority' vote alongside the usual 'Code-Review' and 'Workflow' radio buttons [1]. The idea is to make it easier to direct reviewer attention towards high priority patches. As such, Gerrit dashboards can filter based on the 'Review-Priority' rating. Please vote on whether to enable this feature in Monasca here: https://review.opendev.org/#/c/675574 [1]: http://lists.openstack.org/pipermail/openstack-discuss/2018-December/001304…

2 1

inaccessibility of instances page in my openstack installation
by tim.j.culhane＠gmail.com 09 Aug '19

09 Aug '19

Hi, I'm a blind programmer and we use Openstack in our organisation to create and manage servers for development. In the past when I wanted to launch or delete an instance I'd go to the Instances page. Just above the table listing the current instances I have, would be a series of buttons which allowed you to launch, delete instances or carry out more actions. When using Firefox I can access these items via the keyboard by hitting enter on them. Up to very recently this was also the case in Chrome (which is my preferred browser). However, in the last week of so I've noticed that I can no longer use the keyboard to interact with these controls, you need to click them with the mouse. I'm not sure if this is an issue with Chrome or with Openstack. Has there been any recent changes to Openstack which might explain this? Many thanks, Tim Instance ID = Filter Launch Instance Delete Instances More Actions

2 5

[nova][entropy] what are your rate limits??
by Jordan Ansell 09 Aug '19

09 Aug '19

Hello Openstack Discuss, I am doing some investigation into instance entropy and was wondering what settings others are using with regard to rate limiting entropy supplied by the hypervisor. Specifically, we're adding the "hw_rng:allowed=True" nova flavor property to pass libvirt the relevant config, but need to decide the appropriate rate limiting settings to prevent instances from being greedy with entropy but still retain a comfortable level for themselves. I've done some experimenting (100 bytes/s is possibly a minimum, but still allows a comfortable value of ~1000 for free entropy in instances) I'm also curious to hear other's experiences when it comes to entropy in Openstack: * What sources of entropy did you use in the hypervisor? * Issues you've faced which was caused by insufficient entropy (instance or host) Note, this is for a public cloud scenario, should that impact any suggestions you have. Regards, Jordan

2 1

[keystone] [stein] user_enabled_emulation config problem
by Radosław Piliszek 09 Aug '19

09 Aug '19

Hello all, I have an issue using user_enabled_emulation with my LDAP solution. I set: user_tree_dn = ou=Users,o=UCO user_objectclass = inetOrgPerson user_id_attribute = uid user_name_attribute = uid user_enabled_emulation = true user_enabled_emulation_dn = cn=Users,ou=Groups,o=UCO user_enabled_emulation_use_group_config = true group_tree_dn = ou=Groups,o=UCO group_objectclass = posixGroup group_id_attribute = cn group_name_attribute = cn group_member_attribute = memberUid group_members_are_ids = true Keystone properly lists members of the Users group but they all remain disabled. Did I misinterpret something? Kind regards, Radek

2 3

[nova] Intermittent gate failures in functional tests
by Matt Riedemann 08 Aug '19

08 Aug '19

In case you're seeing a bunch of nova versioned notification functional tests failing this week, it's being tracked [1] and there is a skip patch approved [2] to hopefully resolve it while a long-term fix is worked. [1] http://status.openstack.org/elastic-recheck/#1839515 [2] https://review.opendev.org/#/c/675417 -- Thanks, Matt

1 0

[nova] API updates week 19-32
by Ghanshyam Mann 08 Aug '19

08 Aug '19

Hello Everyone, Please find the Nova API updates of this week. API Related BP : ============ COMPLETED: 1. Support adding description while locking an instance: - https://blueprints.launchpad.net/nova/+spec/add-locked-reason 2. Add host and hypervisor_hostname flag to create server - https://blueprints.launchpad.net/nova/+spec/add-host-and-hypervisor-hostnam… Code Ready for Review: ------------------------------ 1. Specifying az when restore shelved server - Topic: https://review.opendev.org/#/q/topic:bp/support-specifying-az-when-restore-… - Weekly Progress: It is ready for re-review. Patch is updated. 2. Nova API cleanup - Topic: https://review.opendev.org/#/q/topic:bp/api-consistency-cleanup+(status:ope… - Weekly Progress: This is on runway. stephenfin has +2 on nova patch. I will work on novalcient and osc patch tomorrow. 3. Show Server numa-topology - Topic: https://review.opendev.org/#/q/topic:bp/show-server-numa-topology+(status:o… - Weekly Progress: Under review. 4. Nova API policy improvement - Topic: https://review.openstack.org/#/q/topic:bp/policy-default-refresh+(status:op… - Weekly Progress: First set of os-service API policy series is ready to review - https://review.opendev.org/#/c/648480/7 Specs are merged and code in-progress: ------------------------------ ------------------ 5. Detach and attach boot volumes: - Topic: https://review.openstack.org/#/q/topic:bp/detach-boot-volume+(status:open+O… - Weekly Progress: No Progress. Patches are in merge conflict. Spec Ready for Review: ----------------------------- 1. Support for changing deleted_on_termination after boot -Spec: https://review.openstack.org/#/c/580336/ - Weekly Progress: No update this week. Pending on Lee Yarwood proposal after PTG discussion. 3. Support delete_on_termination in volume attach api -Spec: https://review.openstack.org/#/c/612949/ - Weekly Progress: No updates this week. matt recommend to merging this with 580336 which is pending on Lee Yarwood proposal. Previously approved Spec needs to be re-proposed for Train: --------------------------------------------------------------------------- 1. Servers Ips non-unique network names : - https://blueprints.launchpad.net/nova/+spec/servers-ips-non-unique-network-… - https://review.openstack.org/#/q/topic:bp/servers-ips-non-unique-network-na… - I remember I planned this to re-propose but could not get time. If anyone would like to help on this please repropose. otherwise I will start this in U cycle. 2. Volume multiattach enhancements: - https://blueprints.launchpad.net/nova/+spec/volume-multiattach-enhancements - https://review.openstack.org/#/q/topic:bp/volume-multiattach-enhancements+(… - This also need volutneer - http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007411.html Others: 1. Add API ref guideline for body text - 2 api-ref are left to fix. Bugs: ==== No progress report in this week. NOTE- There might be some bug which is not tagged as 'api' or 'api-ref', those are not in the above list. Tag such bugs so that we can keep our eyes.

1 0

[tripleo] Proposing Kevin Carter (cloudnull) for tripleo-ansible core
by Alex Schultz 08 Aug '19

08 Aug '19

Hey folks, I'd like to propose Kevin as a core for the tripleo-ansible repo (e.g. tripleo-ansible-core). He has made excellent progress centralizing our ansible roles and improving the testing around them. Please reply with your approval/objections. If there are no objections, we'll add him to tripleo-ansible-core next Friday Aug 2, 2019. Thanks, -Alex

11 11

[nova/neutron/openvswitch] "No such device" after failed live migrations
by Nicolas Ghirlanda 08 Aug '19

08 Aug '19

Hello all, after a misconfigured setup of a couple of VMs, the live migration of those VMs failed. Now we have some "No such device" listings in openvswitch for those VMs on some compute nodes. (openvswitch-vswitchd)[root@ewos1-com1-prod /]# ovs-vsctl show b5034213-9b15-45f5-8ce0-edcf32d16c57 Manager "ptcp:6640:127.0.0.1" is_connected: true Bridge br-int Controller "tcp:127.0.0.1:6633" is_connected: true fail_mode: secure Port "qvo35487c68-23" tag: 5 Interface "qvo35487c68-23" * Port "qvo5b1aac7e-d4"** ** Interface "qvo5b1aac7e-d4"** ** error: "could not open network device qvo5b1aac7e-d4 (No such device)"** ** Port "qvod9cdff27-9c"** ** Interface "qvod9cdff27-9c"** ** error: "could not open network device qvod9cdff27-9c (No such device)"* Port "qvoed2da602-02" tag: 32 Interface "qvoed2da602-02" Port "qvoa63378f6-d9" for example qvo5b1aac7e-d4 (openvswitch-vswitchd)[root@ewos1-com1-prod /]# ovs-vsctl list Interface ca7e771b-88a0-4cf6-b9be-6be77816baff _uuid : ca7e771b-88a0-4cf6-b9be-6be77816baff admin_state : [] bfd : {} bfd_status : {} cfm_fault : [] cfm_fault_status : [] cfm_flap_count : [] cfm_health : [] cfm_mpid : [] cfm_remote_mpids : [] cfm_remote_opstate : [] duplex : [] error : "could not open network device qvo5b1aac7e-d4 (No such device)" external_ids : {attached-mac="fa:16:3e:4b:f0:4b", iface-id="5b1aac7e-d45b-4a40-8f30-1275b07ffc0b", iface-status=active, vm-uuid="c3fbcc6b-2dfe-4aa1-82bd-522b161a37a9"} ifindex : [] ingress_policing_burst: 0 ingress_policing_rate: 0 lacp_current : [] link_resets : [] link_speed : [] link_state : [] lldp : {} mac : [] mac_in_use : [] mtu : [] mtu_request : [] name : "qvo5b1aac7e-d4" ofport : -1 ofport_request : [] options : {} other_config : {} statistics : {} status : {} type : "" there are no tap devices or bridge devices on the compute node, but still the entry in openvswitch root@computenode5:~# brctl show | grep 5b1aac7e root@computenode5:~# Is that an issue? if yes, should we remove the ports manually or reboot the compute nodes? Could that lead to networking issues? kind regards Nicolas ... --

1 0

[neutron] OpenvSwitch firewall sctp getting dropped
by thuanlk＠viettel.com.vn 08 Aug '19

08 Aug '19

I have installed Openstack Queens on CentOs 7 with OvS and I recently used the native openvswitch firewall to implement SecusiryGroup. The native OvS firewall seems to work just fine with TCP/UDP traffic but it does not forward any SCTP traffic going to the VMs no matter how I change the security groups, But it run if i disable port security completely or use iptables_hybrid firewall driver. What do I have to do to allow SCTP packets to reach the VMs?

4 7

[OpenStack Infra] Current documentations of OpenStack CI/CD architecture
by Trinh Nguyen 08 Aug '19

08 Aug '19

Hi, Is there any documents somewhere describing the current architecture of the CI/CD system that OpenStack Infrastructure is running? Best regards, -- *Trinh Nguyen* *www.edlab.xyz <https://www.edlab.xyz>*

3 5

Slow instance launch times due to RabbitMQ
by Grant Morley 07 Aug '19

07 Aug '19

Hi all, We are randomly seeing slow instance launch / deletion times and it appears to be because of RabbitMQ. We are seeing a lot of these messages in the logs for Nova and Neutron: ERROR oslo.messaging._drivers.impl_rabbit [-] [f4ab3ca0-b837-4962-95ef-dfd7d60686b6] AMQP server on 10.6.2.212:5671 is unreachable: Too many heartbeats missed. Trying again in 1 seconds. Client port: 37098: ConnectionForced: Too many heartbeats missed The RabbitMQ cluster isn't under high load and I am not seeing any packets drop over the network when I do some tracing. We are only running 15 compute nodes currently and have >1000 instances so it isn't a large deployment. Are there any good configuration tweaks for RabbitMQ running on OpenStack Queens? Many Thanks, -- Grant Morley Cloud Lead, Civo Ltd www.civo.com <https://www.civo.com/>| Signup for an account! <https://www.civo.com/signup>

7 9

[nova] Hyper-V CI broken on stable branches
by melanie witt 07 Aug '19

07 Aug '19

Dear Hyper-V CI maintainers, We noticed upstream that the Hyper-V CI has been failing on the stable/stein and stable/rocky (and perhaps older branches too) since this tempest change merged to unskip test_stamp_pattern: https://review.opendev.org/615434 Here are links to sample failed CI runs: stable/stein: http://cloudbase-ci.com/nova/674828/1/tempest/subunit.html.gz stable/rocky: http://cloudbase-ci.com/nova/674916/1/tempest/subunit.html.gz It looks like all is well on the master branch (looks like test_stamp_pattern does not run on master). Just wanted to alert you about the failures and see if anyone is available to fix it or add test_stamp_pattern to a skip list for Hyper-V CI. Cheers, -melanie

1 1

Agenda for TC Meeting 8 August 2019 at 1400 UTC
by Mohammed Naser 07 Aug '19

07 Aug '19

Hi everyone, Here’s the agenda for our monthly TC meeting. It will happen tomorrow (Thursday the 8th) at 1400 UTC in #openstack-tc and I will be your chair. If you can’t attend, please put your name in the “Apologies for Absence” section. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee * Follow up on past action items ** fungi to add himself as TC liaison for Image Encryption popup team ** fungi to draft a resolution on proper retirement procedures * Active initiatives ** Python 3: mnaser to sync up with swift team on python3 migration and mugsie to sync with dhellmann or release-team to find the code for the proposal bot ** Forum follow-up: ttx to organise Milestone 2 forum meeting with tc-members (done) ** Make goal selection a two-step process (needs reviews at https://review.opendev.org/#/c/667932/) * Discussion ** Attendance for leadership meeting during Shanghai Summit on 3 November ** Reviving Performance WG / Large deployment team into a Large scale SIG (ttx) Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

2 1

[training-labs] what is access domain
by Oscar Omar Posada Sanchez 07 Aug '19

07 Aug '19

Hi Team, I am starting to study openStack I am following this reference https://github.com/openstack/training-labs but I can not find the access domain in the first login already installed the laboratory. Could you tell me, thanks. -- Por su atención y tiempo, gracias. Que pase feliz día. -------

2 1

[kolla-ansible] Failed to complete TASK [keystone: Creating the keystone service and enpoint]
by vladimir franciz blando 07 Aug '19

07 Aug '19

OS: CentOS 7 on Baremetal nodes (3x controller, 1xcompute, 2 bonded 10G interfaces) Ansible: 2.8.3 kolla-ansible: 8.0.0 kolla_base_distro: "centos" kolla_install_type: "source" # I also tried "binary" openstack_release: "stein" I already tried redeploying (fresh OS reinstallation) 3 times already and kolla-ansible deploy always fails on this TASK ( http://paste.openstack.org/show/755599/) and won't continue and finish the deployment. And I think the issue is that the admin_url was not created ( http://paste.openstack.org/show/755600/) but why? Which task failed to not create the admin_url? Kolla-ansible only specified that 1 task failed. On keystone logs (http://paste.openstack.org/show/755601/) it says that the admin endpoint was created. The 3 keystone containers (keystone_fernet, keystone_ssh and keystone) are running without error on their logs though. - Vlad ᐧ

2 3

[scientific]How are orgs using openstack for research?
by Brenton Poke 07 Aug '19

07 Aug '19

One of the answers I'm seeking is whether or not some orgs create shareable configurations for software stacks that might do things like collect data into sinks like that of what amazon is attempting to offer with aws research cloud. The limitation I see with aws is that everything is built specifically with aws systems, like using S3 to store everything. For example, if your output would be better represented as a graph, it would make more sense to use a kafka-orientDB sink that can be reused. Make it a helm chart? I'm not sure. I was told that CERN uses openstack for research, but I have no idea as to what extent or if they contribute anything back. Does anyone know how a research org is using the infrastructure right now?

3 2

Shanghai Summit Schedule Live
by Ashlee Ferguson 07 Aug '19

07 Aug '19

Hi everyone, The agenda for the Open Infrastructure Summit <https://www.openstack.org/summit/shanghai-2019/summit-schedule>(formerly the OpenStack Summit) is now live! If you need a reason to join the Summit in Shanghai, November 4-6, here’s what you can expect: Breakout sessions spanning 30+ open source projects from technical community leaders and organizations including ARM, WalmartLabs, China Mobile, China Railway, Shanghai Electric Power Company, China UnionPay, Haitong Securities Company, CERN, and more. Project updates and onboarding from OSF projects: Airship, Kata Containers, OpenStack, StarlingX, and Zuul. Join collaborative sessions at the Forum <https://wiki.openstack.org/wiki/Forum>, where open infrastructure operators and upstream developers will gather to jointly chart the future of open source infrastructure, discussing topics ranging from upgrades to networking models and how to get started contributing. Get hands on training around open source technologies directly from the developers and operators building the software. Now what? Register <https://www.openstack.org/summit/shanghai-2019> before prices increase on August 14 at 11:59pm PT (August 15 at 2:59pm China Standard Time). Recruiting new talent? Pitching a new product? Enhance the visibility of your organization by sponsoring the Summit <https://t.e2ma.net/click/aq0kpb/ygpkur/q287bl>! Questions? Reach out to summit(a)openstack.org <mailto:summit@openstack.org> Cheers, Ashlee Ashlee Ferguson OpenStack Foundation ashlee(a)openstack.org <mailto:ashlee@openstack.org>

1 0

[scientific-sig] IRC meeting today - experiences with accounting and chargeback
by Stig Telfer 06 Aug '19

06 Aug '19

Hi all - We have a Scientific SIG IRC meeting today at 2100 UTC (about an hour’s time) in channel #openstack-meeting. Everyone is welcome. Today’s agenda is here: https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_August_6th_2019 <https://wiki.openstack.org/wiki/Scientific_SIG#IRC_Meeting_August_6th_2019> We’d like to continue with last week’s discussion, gathering notes on experiences with accounting and chargeback for scientific OpenStack deployments. Cheers, Stig

1 0

[OSSA-2019-003] Nova Server Resource Faults Leak External Exception Details (CVE-2019-14433)
by Jeremy Stanley 06 Aug '19

06 Aug '19

========================================================================== OSSA-2019-003: Nova Server Resource Faults Leak External Exception Details ========================================================================== :Date: August 06, 2019 :CVE: CVE-2019-14433 Affects ~~~~~~~ - Nova: <17.0.12,>=18.0.0<18.2.2,>=19.0.0<19.0.2 Description ~~~~~~~~~~~ Donny Davis with Intel reported a vulnerability in Nova Compute resource fault handling. If an API request from an authenticated user ends in a fault condition due to an external exception, details of the underlying environment may be leaked in the response and could include sensitive configuration or other data. Patches ~~~~~~~ - https://review.openstack.org/674908 (Ocata) - https://review.openstack.org/674877 (Pike) - https://review.openstack.org/674859 (Queens) - https://review.openstack.org/674848 (Rocky) - https://review.openstack.org/674828 (Stein) - https://review.openstack.org/674821 (Train) Credits ~~~~~~~ - Donny Davis from Intel (CVE-2019-14433) References ~~~~~~~~~~ - https://launchpad.net/bugs/1837877 - http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-14433 Notes ~~~~~ - The stable/ocata and stable/pike branches are under extended maintenance and will receive no new point releases, but patches for them are provided as a courtesy. -- Jeremy Stanley OpenStack Vulnerability Management Team

1 0

[openstack-ansible] office hours update
by Mohammed Naser 06 Aug '19

06 Aug '19

Hey everyone, Here’s the update of what happened in this week’s OpenStack Ansible Office Hours. We talked about who was attending the Shanghai Summit this year but not many of us are compared to past years. There was an issue with Manila still failing but it's been fixed and there's good progress on that front. There’s also an issue with adding novnc test to os_nova because the tempest plugins used isn’t from master, which brought up some discussion to perhaps start using the Zuul checked-out roles. Thanks for tuning in! Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

1 0

[edge][all] Edge Hacking Days
by Ildiko Vancsa 06 Aug '19

06 Aug '19

Hi, I’m reaching out with an attempt to organize hacking days to work on edge related tasks. The idea is to get together remotely on IRC/Zoom or any other platform that supports remote communication and work on items like building and testing our reference architectures or work on some project specific items like in Keystone or Ironic. Here are Doodle polls for the next three months: August: https://doodle.com/poll/ucfc9w7iewe6gdp4 September: https://doodle.com/poll/3cyqxzr9vd82pwtr October: https://doodle.com/poll/6nzziuihs65hwt7b Please mark any day when you have some availability to dedicate to hack even if it’s not a full day. Please let me know if you have any questions. As a reminder you can find the edge computing group’s resources and information about latest activities here: https://wiki.openstack.org/wiki/Edge_Computing_Group Thanks and Best Regards, Ildikó (IRC: ildikov)

1 1

[tc] Shanghai Summit Planning
by Mohammed Naser 06 Aug '19

06 Aug '19

Hey everyone! Here's the link to the Etherpad for this year's Shanghai summit initial planning. You can put your name if you're attending and also write down your topic of discussion ideas. Looking forward to seeing you there! https://etherpad.openstack.org/p/PVG-TC-PTG Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

1 1

[openstack-ansible] Shanghai Summit Planning
by Mohammed Naser 06 Aug '19

06 Aug '19

Hey everyone! Here's the link to the Etherpad for this year's Shanghai summit initial planning. You can put your name if you're attending and also write down your topic of discussion ideas. Looking forward to seeing you there! https://etherpad.openstack.org/p/PVG-OSA-PTG Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

1 1

[nova] edit flavor
by Engelmann Florian 06 Aug '19

06 Aug '19

Hi, I would like to edit flavors which we created with 0 disk size. Flavors with a disk size (root volume) of 0 are not handled well with, eg. Magnum. So I would like to add a root disk size to those flavors. The default precedure is to delete them and recreate them. But that's not what I want. I would like to edit them, eg in the database. Is there any possible impact on running instances using those flavors? I guess resize will work? All the best, Florian

3 2

[ironic] Moving to office hours as opposed to weekly meetings for the next month
by Julia Kreger 06 Aug '19

06 Aug '19

Greetings Everyone! This week, during the weekly meeting, we seemed to reach consensus that we would try taking a break from meetings[0] and moving to orienting around using the mailing list[1] and our etherpad "whiteboard" [2]. With this, we're going to want to re-evaluate in about a month. I suspect it would be a good time for us to have a "mid-cycle" style set of topical calls. I've gone ahead and created a poll to try and identify a couple days that might be ideal for contributors[3]. But in the mean time, we want to ensure that we have some times for office hours. The suggestion was also made during this week's meeting that we may want to make the office hours window a little larger to enable more discussion. So when will we have office hours? ---------------------------------- Ideally we'll start with two time windows. One to provide coverage to US and Europe friendly time zones, and another for APAC contributors. * I think 2-4 PM UTC on Mondays would be ideal. This translates to 7-9 AM US-Pacific or 10 AM to 12 PM US-Eastern. * We need to determine a time window that would be ideal for APAC contributors. I've created a poll to help facilitate discussion[4]. So what is Office Hours? ------------------------ Office hours are a time window when we expect some contributors to be on IRC and able to partake in higher bandwidth discussions. These times are not absolute. They can change and evolve, and that is the most important thing for us to keep in mind. -- If there are any questions, Please let me know! Otherwise I'll send a summary email out on next Monday. -Julia [0]: http://eavesdrop.openstack.org/meetings/ironic/2019/ironic.2019-07-01-15.00… [1]: http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007038.html [2]: https://etherpad.openstack.org/p/IronicWhiteBoard [3]: https://doodle.com/poll/652gzta6svsda343 [4]: https://doodle.com/poll/2ta5vbskytpntmgv

2 4

Re: [telemetry] ceilometer: Octavia Loadbalancer
by Blom, Merlin, NMU-OI 06 Aug '19

06 Aug '19

Hey, Has anybody experiences with amphora/Octavia Loadbalancers in ceilometer/gnoochi with OpenStack Ansible on Stein? The Metrics for load balancing are not pushed into gnocchi. <https://docs.openstack.org/ceilometer/pike/admin/telemetry-measurements.htm l#load-balancer-as-a-service-lbaas-v2> https://docs.openstack.org/ceilometer/pike/admin/telemetry-measurements.html #load-balancer-as-a-service-lbaas-v2 Looking for the amphora instances directly showed, that the instances in the service tenant don't show up in gnocchi ether. The integration in neutron for Octavia is deactivated because it is deprecated. Is there a workaround? Maybe creating the Measurements via a custom polling script? Thanks you for your help in advance! Merlin Blom

2 1

Re: [telemetry] Gnocchi: Aggregates Operation Syntax
by Blom, Merlin, NMU-OI 06 Aug '19

06 Aug '19

Thanks Bernd, now I understand the new aggregation syntax. J In my setup max is not a valid aggregation method so I would use mean instead. Problem with my use case is that I don't want the actual used cpu NS, but the reserved vcpus per hour. Using the metric vcpus is no option, because it does not reflect the status of the VM. When you shut down the VM it is recorded anyway. So I decided to correlate the cpu with the vcpus metrics. For that I have to write my own python scripts Thanks again for your supportiv answer! Merlin Von: Bernd Bausch <berndbausch(a)gmail.com> Gesendet: Dienstag, 6. August 2019 07:49 An: openstack-discuss(a)lists.openstack.org Betreff: Re: [telemetry] Gnocchi: Aggregates Operation Syntax Yes, aggregate syntax documentation has room for improvement. However, Gnocchi's API documentation has a rather useful list of supported operations at https://gnocchi.xyz/rest.html#list-of-supported-operations <https://urldefense.proofpoint.com/v2/url?u=https-3A__gnocchi.xyz_rest.html- 23list-2Dof-2Dsupported-2Doperations&d=DwMD-g&c=vo2ie5TPcLdcgWuLVH4y8lsbGPqI ayH3XbK3gK82Oco&r=hTUN4-Trlb-8Fh11dR6m5VD1uYA15z7v9WL8kYigkr8&m=s9-jJlo7pCXA 4tDqYm7rnH9PmrmPxcPeNp3ldkJ3O5E&s=ZikNNpSMmr3eG3-z2HL8maF5zlpc4pyNFOLFy2R1-q s&e=> . See also my recent issue https://github.com/gnocchixyz/gnocchi/issues/1044 <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_gnocchixyz_ gnocchi_issues_1044&d=DwMD-g&c=vo2ie5TPcLdcgWuLVH4y8lsbGPqIayH3XbK3gK82Oco&r =hTUN4-Trlb-8Fh11dR6m5VD1uYA15z7v9WL8kYigkr8&m=s9-jJlo7pCXA4tDqYm7rnH9PmrmPx cPeNp3ldkJ3O5E&s=QSibXpyW3kfu6GWLdxb49Uzcny34vtChJaL4wNdHM3s&e=> , which helped me understand how aggregation works in Gnocchi. Note that you can write an aggregate operation as a string using prefix notation, or as a JSON structure. On the command line, the string version is easier to use in my opinion. Regarding your use case, allow me to focus on CPU. Ceilometer's cpu metric accumulates the nanoseconds an instance consumes. Try max aggregation to look at the CPU usage of a single instance: gnocchi measures show --aggregation max --resource-id SERVER_UUID cpu which is equivalent to gnocchi aggregates '(metric cpu max)' id=SERVER_UUID then use sum aggregation over all instances of a project: gnocchi aggregates '(aggregate sum (metric cpu max))' project_id=PROJECT_UUID You can even divide the figures by one billion, which converts nanoseconds to seconds: gnocchi aggregates '(/ (aggregate sum (metric cpu max)) 1000000000)' project_id=PROJECT_UUID If that works, it should not be too hard to do something equivalent for memory and storage. Bernd. On 8/5/2019 5:43 PM, Blom, Merlin, NMU-OI wrote: Hey, I would like to aggregate data from the gnocchi database by using the gnocchi aggregates function of the CLI/API The documentation does not cover the operations that are available nor the syntax that has to be used: <https://urldefense.proofpoint.com/v2/url?u=https-3A__gnocchi.xyz_gnocchicli ent_shell.html-3Fhighlight-3Dreaggregation-23aggregates&d=DwMD-g&c=vo2ie5TPc LdcgWuLVH4y8lsbGPqIayH3XbK3gK82Oco&r=hTUN4-Trlb-8Fh11dR6m5VD1uYA15z7v9WL8kYi gkr8&m=s9-jJlo7pCXA4tDqYm7rnH9PmrmPxcPeNp3ldkJ3O5E&s=rBSJoFwYZ81qRuPWXModdF6 yty_dDg4r25vGRsmXedc&e=> https://gnocchi.xyz/gnocchiclient/shell.html?highlight=reaggregation#aggrega tes Searching for more information I found a GitHub Issue: <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_gnocchixyz_ gnocchi_issues_393&d=DwMD-g&c=vo2ie5TPcLdcgWuLVH4y8lsbGPqIayH3XbK3gK82Oco&r= hTUN4-Trlb-8Fh11dR6m5VD1uYA15z7v9WL8kYigkr8&m=s9-jJlo7pCXA4tDqYm7rnH9PmrmPxc PeNp3ldkJ3O5E&s=YNehDG_-fPl8zG4_leiUOiiAW_y8jxRFazvQqcpXVQs&e=> https://github.com/gnocchixyz/gnocchi/issues/393 But I cannot use the syntax from that ether. My use case: I want to aggregate the vcpus hours per month, vram hours per month, . per server or project. - when an instance is stopped only storage is counted - the exact usage is used e.g. 2 vcpus between 1st and 7th day 4vcpus between 8th and last month no mean calculations Do you have detailed documentation about the gnocchi Aggregates Operation Syntax? Do you have complex examples for gnocchi aggregations? Especially when using the python bindings: conn_gnocchi.metric.aggregation(metrics="memory", query=[XXXXXXXX], resource_type='instance', groupby='original_resource_id') Can you give me advice regarding my use case? Do's and don'ts. Thank you for your help in advance! Merlin Blom

1 0

[goals][IPv6-Only Deployments and Testing] Week R-11 Update
by Ghanshyam Mann 06 Aug '19

06 Aug '19

Hello Everyone, Below is the progress on Ipv6 goal during R11 week. At the first step, I am preparing the ipv6 jobs for the projects having zuulv3 jobs. The projects having zuulv2 jobs will be my second take. It seems like, there are a lot of works required to IPv6 deployment and testing than expected initially. 11 projects put of initial 17 projects are failing on "IPv6-only as listen address". Summary: # of Ipv6 job proposed Projects: 17 # of pass projects: 6 # of failing projects: 11 Storyboard: ========= - https://storyboard.openstack.org/#!/story/2005477 Current status: ============ 1. Base job 'devstack-tempest-ipv6' and 'tempest-ipv6-only' are merged. 2. 'tempest-ipv6-only' has been proposed to run on 5 services project date side [2]. 3. Neutron stadium projects jobs have been prepared (as first only for projects having zuulv3 job) 4. New projects ipv6 jobs patch and status: - Congress: link: https://review.opendev.org/#/c/671908/ status: job is failing, I will ping Eric to help on debugging. - Monasca: links: https://review.opendev.org/#/q/topic:ipv6-only-deployment-and-testing+(stat… status: jobs are failing. I fixed some of the IPv6 parsing on monasca-api and monasca-notoficatiob side but it seems kafka addresses still have an issue on IPv6 - Murano: links: https://review.opendev.org/#/c/673291/ Status: job is failing. I have not debugged it. I need project help on this if any bug or failing for any other issue. - cloudkitty: link: https://review.opendev.org/#/c/671909/ status: Job is passing. - quinling: link: https://review.opendev.org/#/c/673506/ status: job is failing. need to debug this. - networking-old link: https://review.opendev.org/#/c/673501/ status: working with Lajos to define the job on top of their new zuulv3 jobset. - networking-ovn: link: https://review.opendev.org/#/c/673488/ status: job is failing, Lucas and Brian already checking it on review. I will debug with them. IPv6 missing support found: ===================== 1. https://review.opendev.org/#/c/673397/ 2. https://review.opendev.org/#/c/673449/ 3. https://review.opendev.org/#/c/673266/ How you can help: ============== - Each project needs to look for and review the ipv6 job patch. - Verify it works fine on ipv6 and no ipv4 used in conf etc - Any other specific scenario needs to be added as part of project IPv6 verification. - Help on debugging and fix the bug in IPv6 job is failing. Everything related to this goal can be found under this topic: Topic: https://review.opendev.org/#/q/topic:ipv6-only-deployment-and-testing+(stat… How to define and run new IPv6 Job on project side: ======================================= - I prepared a wiki page to describe this section - https://wiki.openstack.org/wiki/Goal-IPv6-only-deployments-and-testing - I am adding this wiki info in goal doc also[4]. Review suggestion: ============== - Main goal of these jobs will be whether your service is able to listen on IPv6 and can communicate to any other services either OpenStack or DB or rabbitmq etc on IPv6 or not. So check your proposed job with that point of view. If anything missing, comment on patch. - One example was - I missed to configure novnc address to IPv6- https://review.opendev.org/#/c/672493/ - base script as part of 'devstack-tempest-ipv6' will do basic checks for endpoints on IPv6 and some devstack var setting. But if your project needs more specific varification then it can be added in project side job as post-run playbooks as described in wiki page[5]. [1] https://review.opendev.org/#/c/671231/ [2] https://review.opendev.org/#/q/topic:ipv6-only-deployment-and-testing+(stat… [3] http://eavesdrop.openstack.org/irclogs/%23openstack-neutron/%23openstack-ne… [4] https://review.opendev.org/#/c/671898/ [5] https://wiki.openstack.org/wiki/Goal-IPv6-only-deployments-and-testing -gmann

1 0

Re: [telemetry] Team meeting agenda for tomorrow
by Joseph Davis 06 Aug '19

06 Aug '19

On 5/8/19 7:12 AM, openstack-discuss-request(a)lists.openstack.org wrote: > Hello Trinh, > Where does the meeting happen? Will it be via IRC Telemetry channel? Or, in > the Etherpad (https://etherpad.openstack.org/p/telemetry-meeting-agenda)? I > would like to discuss and understand a bit better the context behind > the Telemetry > events deprecation. Unfortunately, I have a conflict at that time and will not be able to attend. I do have a little bit of context on the Events deprecation to share. First, you will note the commit message from the commit [0] when Events were deprecated: " Deprecate event subsystem This subsystem has never been finished and is not maintained. Deprecate it for future removal. " I got the impression from jd at the time that there were a number of features in Telemetry, including Panko, that were not really "finished" and that the engineers who had worked on them had moved on to other things, so the features had become unsupported. In late 2018 there was an effort to clean up things that were not well maintained or didn't fit the direction of Telemetry. See also: https://julien.danjou.info/lessons-from-openstack-telemetry-deflation/ Events is one feature that often gets requested, but the use cases and demand for it are not expressed strongly or well understood by most people. If the Telemetry project has demand to de-deprecate Event handling (including Panko), I'd suggest a review of the requirements for event handling and possibly choosing a champion for maintaining the Panko service. Also note: over in Monasca we have a spec [1] for handling Events ingestion which I hope we will be completing in Train. Contributions and comments welcome. :) joseph [0] https://github.com/openstack/ceilometer/commit/8a0245a5b3e1357d35ad6653be37… [1] https://github.com/openstack/monasca-specs/blob/master/specs/stein/approved… > > On Wed, May 8, 2019 at 12:19 AM Trinh Nguyen<dangtrinhnt(a)gmail.com> wrote: > >> Hi team, >> >> As planned, we will have a team meeting at 02:00 UTC, May 9th on >> #openstack-telemetry to discuss what we gonna do for the next milestone >> (Train-1) and continue what we left off from the last meeting. >> >> I put here [1] the agenda thinking that it should be fine for an hour >> meeting. If you have anything to talk about, please put it there too. >> >> [1]https://etherpad.openstack.org/p/telemetry-meeting-agenda >> >> >> Bests, >> >> -- >> *Trinh Nguyen* >> *www.edlab.xyz<https://www.edlab.xyz>* >> >>

14 25

[telemetry] Gnocchi: Aggregates Operation Syntax
by Blom, Merlin, NMU-OI 06 Aug '19

06 Aug '19

Hey, I would like to aggregate data from the gnocchi database by using the gnocchi aggregates function of the CLI/API The documentation does not cover the operations that are available nor the syntax that has to be used: <https://gnocchi.xyz/gnocchiclient/shell.html?highlight=reaggregation#aggreg ates> https://gnocchi.xyz/gnocchiclient/shell.html?highlight=reaggregation#aggrega tes Searching for more information I found a GitHub Issue: <https://github.com/gnocchixyz/gnocchi/issues/393> https://github.com/gnocchixyz/gnocchi/issues/393 But I cannot use the syntax from that ether. My use case: I want to aggregate the vcpus hours per month, vram hours per month, . per server or project. - when an instance is stopped only storage is counted - the exact usage is used e.g. 2 vcpus between 1st and 7th day 4vcpus between 8th and last month no mean calculations Do you have detailed documentation about the gnocchi Aggregates Operation Syntax? Do you have complex examples for gnocchi aggregations? Especially when using the python bindings: conn_gnocchi.metric.aggregation(metrics="memory", query=[XXXXXXXX], resource_type='instance', groupby='original_resource_id') Can you give me advice regarding my use case? Do's and don'ts. Thank you for your help in advance! Merlin Blom

2 1

[kuryr] [ptl] [tc] Stepping down as Kuryr PTL
by Daniel Mellado 06 Aug '19

06 Aug '19

As I have taken on a new role in my company I won't be having the time to dedicate to Kuryr in order to keep as the PTL for the current cycle. I started working on the project more than two cycles ago and it has been a real pleasure for me. Helping a project grow from an idea and a set of diagrams to a production-grade service was an awesome experience and I got help from my awesome team and upstream contributors! I would like to take this opportunity to thank everyone who contributed to the success of Kuryr – either by writing code, suggesting new use cases, participating in our discussions, or helping out with Infra! Michal Dulko (irc: dulek) has been kind enough to accept replacing me as the new Kuryr PTL [1]. I’m sure he'll make an excellent work as he's knowledgeable about every piece of the code and is tightly connected to the community. I will still be around to help if needed. Please join me congratulating Michal on his new role! Best! Daniel [1] https://review.opendev.org/674624

6 5

stestr Python 2 Support
by Matthew Treinish 05 Aug '19

05 Aug '19

Hi Everyone, I just wanted to send a quick update about the state of python 2 support in stestr, since OpenStack is the largest user of the project. With the recent release of stestr 2.4.0 we've officially deprecated the python 2.7 support in stestr. It will emit a DeprecationWarning whenever it's run from the CLI with python 2.7 now. The plan (which is not set in stone) is that we will be pushing a 3.0.0 release that removes the python 2 support and compat code from stestr sometime in early 2020 (definitely after the Python 2.7 EoL on Jan. 1st). I don't believe this conflicts with the current Python version support plans in OpenStack [1] but just wanted to make sure people were aware so that there are no surprises when stestr stops working with Python 2 in 3.0.0. Thanks, -Matt Treinish [1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecatio…

3 3

[neutron] Bug deputy report for week of July 29th
by Brian Haley 05 Aug '19

05 Aug '19

Hi, I was Neutron bug deputy last week. Below is a short summary about reported bugs. -Brian Critical bugs ------------- None High bugs --------- * https://bugs.launchpad.net/neutron/+bug/1838699 - Removing a subnet from DVR router also removes DVR MAC flows for other router on network - Confirmed, needs owner * https://bugs.launchpad.net/neutron/+bug/1838760 - Security groups don't work for trunk ports with iptables_hybrid fw driver - Confirmed, needs owner * https://bugs.launchpad.net/neutron/+bug/1838793 - "KeepalivedManagerTestCase" tests failing during namespace deletion - Rodolfo took ownership * https://bugs.launchpad.net/devstack/+bug/1838811 - /opt/stack/devstack/tools/outfilter.py failing in neutron functional jobs since 8/2 - https://review.opendev.org/#/c/674426/ merged Medium bugs ----------- * https://bugs.launchpad.net/neutron/+bug/1838396 - update port receive 500 - https://review.opendev.org/#/c/673486/ * https://bugs.launchpad.net/neutron/+bug/1838403 - Asymmetric floating IP notifications - Seems events for floating IPs/routers are not sent in some cases - Additional information was supplied in bug, need to reproduce - Need neutron version information - Needs owner * https://bugs.launchpad.net/neutron/+bug/1838449 - Router migrations failing in the gate - Miguel assigned to himself, needs further investigation * https://bugs.launchpad.net/neutron/+bug/1838431 - [scale issue] ovs-agent port processing time increases linearly and eventually timeouts - Confirmed, needs owner * https://bugs.launchpad.net/neutron/+bug/1838563 - Timeout in executing ovs command crash ovs agent - https://review.opendev.org/#/c/674085/ * https://bugs.launchpad.net/neutron/+bug/1838587 - request neutron with Incorrect body key return 500 - https://review.opendev.org/#/c/674153/ * https://bugs.launchpad.net/neutron/+bug/1838689 - rpc_workers default value ignores setting of api_workers - https://review.opendev.org/674125 merged, backport in progress Low bugs -------- Wishlist bugs ------------- * https://bugs.launchpad.net/neutron/+bug/1838621 - [RFE] Configure extra dhcp options via API and per network - Discussed at neutron drivers meeting 8/2 - Slawek will investigate questions that came up at meeting Invalid bugs ------------ Further triage required ----------------------- * https://bugs.launchpad.net/neutron/+bug/1838617 - ssh connection getting dropped frequently - Pike, but not much info given, asked for logs and/or a reproducer on more recent code. * https://bugs.launchpad.net/neutron/+bug/1838697 - DVR Mac conversion rules are only added for the first router a network is attached to - Asked for additional information * https://bugs.launchpad.net/neutron/+bug/1839004 - Rocky DVR-SNAT seems missing entries for conntrack marking - Looks like a possible mis-configuration - Asked for additional information

1 0

[Manila] Python3 support in Manila 3rd Party CI
by Tom Barron 05 Aug '19

05 Aug '19

We worked in the Pike release to get Manila unit tests running under Python3. In Train we have completed the work begun in Stein to get all first party functional test jobs running under Python 3. Now we need to push to get third party jobs converted to Python 3 since Train will be the last OpenStack release to keep support for Python2 and it will itself support Python3 first [1]. None of this will be news to those who attend the weekly Manila team meeting at 1500 UTC on Thursdays on #openstack-meetings-alt on freenode [2], but not every back end driver has regular representation at those meetings so we are communicating the need for third party CI job Python 3 support here as well. We are tracking this work on the Manila wiki [3], where you may also find or add tips to help other CI maintainers. Also, please feel free to follow up with me via email or on freenode #openstack-manila. -- Tom Barron email: tpb@dyncloudnet irc: tbarron [1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecatio… [2] https://wiki.openstack.org/wiki/Manila/Meetings [3] https://wiki.openstack.org/wiki/Manila/TrainCycle#Python3_Testing

1 0

[Airship-Seaworthy] Deployment of Airship-Seaworthy on Virtual Environment
by Anirudh Gupta 05 Aug '19

05 Aug '19

Hi Team, We want to test Production Ready Airship-Seaworthy in our virtual environment The link followed is https://airship-treasuremap.readthedocs.io/en/latest/seaworthy.html<https://ind01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairship-t…> As per the document we need 6 DELL R720xd bare-metal servers: 3 control, and 3 compute nodes. But we need to deploy our setup on Virtual Environment. Does Airship-Seaworthy support Installation on Virtual Environment? We have 2 Rack Servers with Dual-CPU Intel® Xeon® E5 26xx with 16 cores each and 128 GB RAM. Is it possible that we can create Virtual Machines on them and set up the complete environment. In that case, what possible infrastructure do we require for setting up the complete setup. Looking forward for your response. Regards अनिरुद्ध गुप्ता (वरिष्ठ अभियंता) Hughes Systique Corporation D-23,24 Infocity II, Sector 33, Gurugram, Haryana 122001 DISCLAIMER: This electronic message and all of its contents, contains information which is privileged, confidential or otherwise protected from disclosure. The information contained in this electronic mail transmission is intended for use only by the individual or entity to which it is addressed. If you are not the intended recipient or may have received this electronic mail transmission in error, please notify the sender immediately and delete / destroy all copies of this electronic mail transmission without disclosing, copying, distributing, forwarding, printing or retaining any part of it. Hughes Systique accepts no responsibility for loss or damage arising from the use of the information transmitted by this email including damage from virus.

3 5

[manila] Enabling CephFS snapshot support by default
by Goutham Pacha Ravi 05 Aug '19

05 Aug '19

Hi Zorillas and interested parties, (Copying a couple of folks explicitly as they may not be part of openstack-discuss) Manila's CephFS driver has a configuration option "cephfs_enable_snapshots" that administrators can toggle in manila.conf to allow project users to take snapshots of their shares. [1][2] This option has defaulted to False, since the time the CephFS driver was introduced (Mitaka release of OpenStack Manila) IIUC, CephFS snapshots have been "stable" since the Mimic release of CephFS, which debuted in June 2018. [3] Since then, ceph deployers/administrators don't have to toggle anything on the backend to enable snapshots. So, can we consider changing the default value of the config opt in manila to "True" in the Train release? This will allow the driver to report the capability "snapshot_support" as enabled, and makes life easier for OpenStack deployers/administrators. I would also love to deprecate this option and remove it in a future release, since it isn't really necessary, read on: Users cannot arbitrarily take snapshots in manila, there are a couple of things that can be used to prevent them: Quotas and Share Types. Unless users create their shares using a share type that explicitly requests "snapshot_support", users can't snapshot their shares. So, we're covered by the service's API in case someone is afraid this has an impact on someone in the wild using newer versions of Manila with older versions of Ceph, where snapshots were unstable. Thoughts? [1] https://opendev.org/openstack/manila/src/commit/721bb70a81b80bf065705e8790b… [2] https://docs.openstack.org/manila/queens/admin/cephfs_driver.html#enabling-… [3] http://docs.ceph.com/docs/mimic/dev/cephfs-snapshots/

2 1

[nova] local ssd disk performance
by Budai Laszlo 05 Aug '19

05 Aug '19

Hello all, we have a problem with the performance of the disk IO in a KVM instance. We are trying to provision VMs with high performance SSDs. we have investigated different possibilities with different results ... 1. configure Nova to use local LVM storage (images_types = lvm) - provided the best performance, but we could not migrate our instances (seems to be a bug). 2. use cinder with lvm backend and instance locality, we could migrate the instances, but the performance is less than half of the previous case 3. mount the ssd on /var/lib/nova/instances and use the images_type = raw in nova. We could migrate, but the write performance dropped to ~20% of the images_types = lvm performance and read performance is ~65% of the lvm case. do you have any idea to improve the performance for any of the cases 2 or 3 which allows migration. Kind regards, Laszlo

4 8

[tc] fast-approve change to update voting schedule for u release
by Mohammed Naser 05 Aug '19

05 Aug '19

Hi everyone, Doug has proposed a change to make changes to the voting schedule for the U release. I will be fast tracking this change and this serves as a notification to the TC (and the community as whole). It's a trivial change which adjusts scheduling only, nothing more. The current date is already passed so we wouldn't have documentation that makes sense in the first place. https://review.opendev.org/#/c/674465/1 If anyone objects, feel free to push a revert to that patch. Thanks, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

1 0

[openstack-dev][magnum] Project updates
by feilong 05 Aug '19

05 Aug '19

Hi all, Now we're very close to the milestone-2 of Train release. I'd like to take this opportunity to update the project status of Magnum. So far, we have done some great work in this cycle which make Magnum to achieve to a higher level. The first feature I'd like to highlight is the k8s version rolling upgrade. With this feature, user can upgrade their k8s version with zero downtime for their application. Another one is auto scaling, now Magnum can support k8s cluster auto scaling by leveraging the Cluster Autoscaler (there is a OpenStack Magnum driver contributed by CERN team). Besides, Magnum now can support auto repair, aka auto healing, by using a combination of Draino, Cluster Autoscaler and Node Problem Detector. Except above work we have done, there are still a lot of cool features coming: 1. The first one I'd like to mention is the boot volume support. With this feature, user can boot k8s cluster node from volume and user can even set the volume type which is super useful if the cloud provider can support high performance storage, e.g. NVME. And a new useful label named "etcd_volume_type" will be introduced by the way so that user can set the volume type for etcd volume to get a great performance for etcd cluster. 2. Private cluster. With this feature, user can set the network, subnet and floating IP when creating cluster with a public cluster template, which is very handy when user want to use public templates but still want to create the cluster in their existing network. 3. Another super fancy feature is introducing a new auto healing controller except the Draino+CA+NPD combination. Recently, we (Catalyst Cloud) contributed our Magnum auto healer[1] to Cloud-Provider-Openstack repo and we're trying to introduce it to Magnum as another auto healing controller, it would be also very useful for end user to get a zero downtime k8s cluster. As you can see, there are quite a lot of exciting features are being introduced into Magnum, however all of them are done by such a small team. What a cool team! As you know, Kubernetes is still evolving very fast, so we're very keen to get more help from the community. We're working hard to add some new core members to our existing team, but we do need more. In short, if you're interested in provisioning Kubernetes on OpenStack, please try Magnum and feel free to pop up on #openstack-containers for any question. Cheers. -- Cheers & Best regards, Feilong Wang (王飞龙) ------------------------------------------------------ Senior Cloud Software Engineer Tel: +64-48032246 Email: flwang(a)catalyst.net.nz Catalyst IT Limited Level 6, Catalyst House, 150 Willis Street, Wellington ------------------------------------------------------

3 4

[uc]The UC Nomination Period is now open!
by Ed Leafe 05 Aug '19

05 Aug '19

As the subject says, the nomination period for the August 2019 User Committee elections is now open. Any individual member of the Foundation who is an Active User Contributor (AUC) can propose their candidacy (except the three sitting UC members elected in the previous election). Self-nomination is common, no third party nomination is required. They do so by sending an email to the user-committee(a)lists.openstack.org mailing-list, with the subject: “UC candidacy” by August 16, 05:59 UTC. The email can include a description of the candidate platform. The candidacy is then confirmed by one of the election officials, after verification of the electorate status of the candidate. -- Ed Leafe

1 0

reply: [lists.openstack.org代发]Re: [nova][ops] Documenting nova tunables at scale
by Brin Zhang(张百林) 05 Aug '19

05 Aug '19

Agree with this approach. The large-scale test scenario configuration manual is very meaningful. When the OpenStack deployment scale reaches a certain level (for example, nodes >= 200,500… etc.), various exception scenarios will occur, rabitmq blocking conditions, and create a batch of servers. It’s success rate (There will be some configurations to be care. e.g. scheduling policies, rpc wait time, amount of works etc.). If there is a reference manual, this is very friendly. > On Sat, 3 Aug. 2019, 6:25 am Matt Riedemann, <mriedemos(a)gmail.com<mailto:mriedemos@gmail.com>> wrote: > I wanted to send this to get other people's feedback if they have > particular nova configurations once they hit a certain scale (hundreds > or thousands of nodes). Every once in awhile in IRC I'll be chatting > with someone about configuration changes they've made running at large > scale to avoid, for example, hammering the control plane. I don't know > how many times I've thought, "it would be nice if we had a doc > highlighting some of these things so a new operator could come along and > see, oh I've never tried changing that value before". > > I haven't started that doc, but I've started a bug report for people to > dump some of their settings. The most common ones could go into a simple > admin doc to start. > > I know there is more I've thought about in the past that I don't have in > here but this is just a starting point so I don't make the mistake of > not taking action on this again. > > https://bugs.launchpad.net/nova/+bug/1838819 > > -- > > Thanks, > > Matt Hi Matt, My name is Joe - docs person from years back - this looks like a good initiative and I would be up for documenting these settings at scale. Next step I can see is gathering more Info about this pain point (already started :)) and then I can draft something together for feedback.

1 0

Openstack IPv6 neutron confiuraton
by Md. Farhad Hasan Khan 03 Aug '19

03 Aug '19

Hi, We already have IPv4 vxLAN in our openstack. Now we want to add IPv6 for tenant network. Looking for help how to configure ipv6 tenant network in openstack neutron. Kindly help me to understand how ipv6 packet flow. Please give me some documents with network diagram. Thanks & B'Rgds, Rony

4 5