November 2019 - openstack-discuss

[ptg][nova][keystone] x-p meeting minutes
by Sylvain Bauza 12 Nov '19

12 Nov '19

We only discussed about the large effort needed for the new policies. https://specs.openstack.org/openstack/nova-specs/specs/ussuri/approved/poli… https://review.opendev.org/#/q/topic:bp/policy-defaults-refresh+(status:ope… We asked the Keystone team to review the first changes in the series as it's very helpful. We had concerns about completion over the Ussuri cycle since the series can be very large and we want to avoid deprecation messages for some APIs if not all the Nova APIs are touched yet. The agreement was to hold a procedural -2 on the API changes and start reviewing (for both Keystone and Nova folks) until all the changes are there. Maybe a runway could help once all the API changes are uploaded. -S

1 0

[nova][cinder] Volume-backed instance disks and some operations that do not support those yet
by Peter Penchev 12 Nov '19

12 Nov '19

Hi, First of all, thanks to everyone involved for all the work on Nova, Cinder, os-brick, and actually all the rest of OpenStack, too! So, yeah, I guess it is kind of weird that I'm asking this on the list just a couple of days after the PTG where I could have asked in person, but here goes :) There seem to still be some quirks with Nova and volume-backed instance disks; some actions on instances are not allowed, others produce somewhat weird results. From a quick look at the code it seems to me that currently these are: - taking a snapshot of an instance (produces a zero-sized file, no real data backed up) - backing an instance up (refuses outright) - rescuing an instance (refuses outright) ...and maybe there are some that I've missed. So, possibly stupid question here, but what are the project's plans about these - is there an intention to implement them at some point, or are there some very, very hard theoreitcal or practical problems (so something like "guess not for the present"), or is somebody working on something? The main reason that I am asking is that we, StorPool, have a shared-storage Cinder driver, and every now and then a customer comes up and asks about one or more of these actions. Every now and then we come back to the idea of writing a vendor-specific Nova image backend, but, first off, we are not really sure whether we want to do this, and second, we are not really sure whether it will be accepted upstream. A couple of years ago people told us "don't do that" and there was some talk about having an image backend for storage drivers supported by libvirt, but that effort seems to have stalled. Of course, we know that in all software projects, including, but certainly not limited to, the more-or-less volunteer free/libre/open-source projects, there are many tasks and many demands on the developers so that it is only natural that not everything is implemented or adapted at once; things happen, priorities shift, people get redirected, nobody else steps up to continue - it happens. With this in mind, where do things stand right now, should we consider writing an image backend, are there other options or plans? So thanks for reading through my ramblings, I guess, and keep up the great work! Best regards, Peter

4 3

[nova][ptg] Support re-configure deleted_on_termination in server
by Sylvain Bauza 12 Nov '19

12 Nov '19

Spec is https://review.opendev.org/#/c/580336/ Most people seem to think this makes sense but realize there are already other ways to do this (snapshot) and therefore it's not totally necessary. The agreement in the room was to post the code up for the change, as this will help sell people on it if it's trivial enough and document the use case (i.e. are there scenarios where this would make life 10x easier?)

1 0

BGP dynamic routing
by Volodymyr Litovka 12 Nov '19

12 Nov '19

Dear colleagues, "BGP dynamic routing" doc (https://docs.openstack.org/neutron/rocky/admin/config-bgp-dynamic-routing.h…) says only about advertisement of routes: "BGP dynamic routing enables advertisement of self-service (private) network prefixes to physical network devices that support BGP such as routers, thus removing the conventional dependency on static routes." and nothing about receiving of routes from external peers. Whether it is ever possible using Neutron to have fully dynamic routing inside the project, both advertising/receiving (and updating VRs configuration) routes to/from remote peers? Thank you. -- Volodymyr Litovka "Vision without Execution is Hallucination." -- Thomas Edison

3 5

[tacker] No IRC meeting today
by Dharmendra Kushwaha 12 Nov '19

12 Nov '19

Hello Taker team, As we have PTG in last week, lets skip today's weekly meeting. Thanks & Regards Dharmendra Kushwaha

1 0

How to set password
by zhaowx＠jxresearch.com 12 Nov '19

12 Nov '19

hello: When I use `trovestack build-image`, how to set password for the image? thanks zhaowx(a)jxresearch.com

1 0

[nova][ptg] drop the shadow table concept
by Balázs Gibizer 11 Nov '19

11 Nov '19

CERN reported two issues with archive_deleted_rows CLI: * When one record gets inserted into the shadow_instance_extra but didn't get deleted from instance_extra (I know this is in a single transaction but sometimes it happens), needs manual cleanup on the database * Also there could be two cells running this command at the same time fighting for the API db lock, TODOs: * tssurya to report bugs / improvements on archive_deleted_rows CLI based on CERN's experience with long table locking * mnaser to report a wishlist bug / specless bp about one step db purge CLI which would skip the shadow tables Cheers, gibi

4 6

[octavia][ptg] Summary of Shanghai PTG Discussion
by Adam Harwell 11 Nov '19

11 Nov '19

Fellow Octavians, We covered a lot of ground during this PTG, met a number of new folks, and got a lot of valuable feedback. I'll do my best to summarize here what was discussed. 1. Metrics 1. It would be nice to expose metrics for pools/members, though we would like to get a better understanding of the requirements / use-cases. 2. We should publish metrics using some mechanism (plugin/driver). 1. The default would be "database" and would handle the existing API-exposed metrics. 2. Additional drivers would be loaded in parallel, and might include Monasca/Ceilometer/Prometheus drivers. 3. We will switch our metrics internally to use a delta system instead of absolute values from HAProxy. This will allow us to publish in a more sane way in the future. This would not change the way metrics are exposed in the existing API. 2. Notifications 1. We will need to create a spec and gather community feedback. 2. Initial observation indicates the need for two general paths, which will most likely have their own driver systems: 1. provisioning_status changes (including all create/update/delete events) 2. operating_status changes (member up/down, etc) 3. We would provide the entire object in the notification, similar to what other services do. 4. Most likely the default driver(s) would use oslo.notify. 3. Availability Zone Support (Multi-Zone Fault Tolerance) 1. Make at least a story for tracking this, if it doesn't already exist. 2. Allow a single LB to have amphorae in multiple zones. 3. Existing patch: https://review.opendev.org/#/c/558962/ 4. Availability Zone Support (Compute AZ Awareness) 1. Make at least a story for tracking this, if it doesn't already exist. 2. Allow placing LBs in specific zones: 1. When zones are geographically separated, LBs should exist in the same zone as the members they support 2. When zones are logically separated (like PCI compliance zones, etc), users may need to place them specifically. 3. A new parameter `availability_zone` will be added to the LB create API. It will allow the user to select which Octavia AZ to use. 4. A new API section will be added for creating/configuring/listing Octavia AZs. This will allow a linkage between Compute AZs and Amphora Management Network, along with other possible options in the future. Admins can create/update, and users can list zones. 5. Update clients to support this, including further polluting the `openstack availability zone list` command to include `--loadbalancers` zones. 5. Python 2 EOL 1. Remove all jobs that test Python 2 (or update them if they're not duplicates). 2. Remove six compatibility code, which should simplify string handling significantly. 6. More Flavor Capabilities 1. Image Tag (to allow different amp images per flavor) 2. Availability Zone (to allow compute AZ pinning) 3. Management Network (to go with compute AZ) 4. Metadata (allow passing arbitrary metadata through to compute) 7. TLS Protocol/Cipher API Support 1. Allow users to select specific protocols/ciphers as a whitelist. 2. Stories: 1. Ciphers: https://storyboard.openstack.org/#!/story/2006627 2. Protocols: https://storyboard.openstack.org/#!/story/2006733 8. Performance Tuning 1. HAProxy: There are a number of knobs/dials that can be adjusted to make HAProxy behave more efficiently. Some of these that we could look at more are around TLS options, and around multiprocessing/threading. The latter will probably need to wait for us to switch to HAProxy 2.0. 2. Image Metadata: There are flags that could be added to our amphora image's metadata that might improve performance. To be further researched. 9. Testing 1. Team to evaluate existing non-voting jobs for promotion to voting. 1. Agreement was made with the Barbican team to promote both side's co-gating jobs to voting. 2. Team to evaluate merging or pruning some jobs to reduce the overall set that run on each change. 3. Grenade needs a few changes: 1. Switch to python3. 2. Upgrade to Zuul v3. 3. Test additional operations on existing LBs (old amp image), not just traffic. 4. Test more than just the most recent amphora image against the current control-plane code. Use periodic jobs for this. 5. Fix the Zuul grafana dashboard for Octavia test history. 10. Jobboard 1. This continues to be a priority for Ussuri. 2. Put together a priority list of patches specifically for jobboard. 11. HAProxy 2.0 1. A number of features are gated behind the new version, including multi-process and HTTP/2 support. 2. Need to reach out to distributions to push for backports (to cloudarchive for Ubuntu, and whatever similar thing for CentOS). 3. Possibly add an element to allow building new versions from source. 4. Perform version-based validation of options on the API-side of the amphora driver. 1. Inspect and cache Glance metadata for the LB's amphora image to get version data. 2. Provide the metadata string for the operator from our disk-image-create script. The full etherpad from the PTG including the notes I've summarized here is available at https://etherpad.openstack.org/p/octavia-shanghai-U-ptg if further review is desired. Thanks to everyone who participated, and best of luck on this (hopefully) productive new cycle! --Adam Harwell

1 0

[neutron] Bug Deputy Report Nov. 4-11
by Ryan Tidwell 11 Nov '19

11 Nov '19

Hello neutrinos, here is the bug deputy report for the week of Nov. 4th: High: * https://bugs.launchpad.net/neutron/+bug/1851659 "removing a network from a DHCP agent removes L3 rules even if it shouldn't" This was found on stable/rocky, we should obviously see if it can be reproduced on master, stein, and train as well. There does seem to be a workaround, but VM's lost connectivity for a brief period. Medium: * https://bugs.launchpad.net/neutron/+bug/1851500 ""test_show_port_chain" conflicts with Flow Classifier" This is a gate issue, with what appears to be some instability with a test. Initial triage indicates it's an intermittent issue. * https://bugs.launchpad.net/neutron/+bug/1851194 "FWaaSv2 configures iptables with invalid port name" This similar to https://bugs.launchpad.net/neutron/+bug/1798577, FWaaSv2 seems to be referencing the wrong port names. RFE: * https://bugs.launchpad.net/neutron/+bug/1851609 Add an option for graceful l3 agent shutdown -Ryan Tidwell

1 0

Change Volume Type, but in use
by Sinan Polat 11 Nov '19

11 Nov '19

Hi, I am using Ceph as the backend for Cinder. Within Ceph we have defined 2 RBD pools (ssdvolumes, sasvolumes). In OpenStack I created 2 Volume Types (ssd, sas). Each Volume Type has property "volume_backend_name='tripleo_ceph_<ssd|sas>'". In the Cinder configuration I have the following backends configured: [tripleo_ceph_ssd] backend_host=hostgroup volume_backend_name=tripleo_ceph_ssd volume_driver=cinder.volume.drivers.rbd.RBDDriver rbd_ceph_conf=/etc/ceph/ceph.conf rbd_user=openstack rbd_pool=ssdvolumes [tripleo_ceph_sas] backend_host=hostgroup volume_backend_name=tripleo_ceph_sas volume_driver=cinder.volume.drivers.rbd.RBDDriver rbd_ceph_conf=/etc/ceph/ceph.conf rbd_user=openstack rbd_pool=sasvolumes As you might have noticed, the backend name (tripleo_ceph_ssd) and the RBD pool name (ssdvolumes, not ssd) does not match. So far, we do not have any problems. But I want to correct the names and I do not want to have the mismatch anymore. So I want to change the value of key volume_backend_name for both Volume Types (tripleo_ceph_ssd => tripleo_ceph_ssdvolumes). I tried the following: $ openstack volume type show 80cb25ff-376a-4483-b4f7-d8c75839e0ce +--------------------+----------------------------------------+ | Field | Value | +--------------------+----------------------------------------+ | access_project_ids | None | | description | | | id | 80cb25ff-376a-4483-b4f7-d8c75839e0ce | | is_public | True | | name | ssd | | properties | volume_backend_name='tripleo_ceph_ssd' | | qos_specs_id | None | +--------------------+----------------------------------------+ $ $ openstack volume type set --property volume_backend_name='tripleo_ceph_ssdvolumes' 80cb25ff-376a-4483-b4f7-d8c75839e0ce Failed to set volume type property: Volume Type is currently in use. (HTTP 400) (Request-ID: req-5efaa5b7-910f-4802-8494-3115cfc4ab93) Command Failed: One or more of the operations failed $ How to solve my problem? Thanks! Sinan

4 6