January 2019 - openstack-discuss - lists.openstack.org

[goals][upgrade-checkers] Week R-14 Update
by Matt Riedemann 04 Jan '19

04 Jan '19

There has not been much progress since the R-16 update [1] let's assume because of the holiday break. There is a new trove patch which replaces the placeholder check with a real upgrade check [2]. I have left comments on that review. I am not sure if that is due to some new changes in trove which require the upgrade check, or if this is just something that has always been needed when upgrading trove. [1] http://lists.openstack.org/pipermail/openstack-discuss/2018-December/001266… [2] https://review.openstack.org/#/c/627555/ -- Thanks, Matt

1 0

[placement] update 19-00
by Chris Dent 04 Jan '19

04 Jan '19

HTML: https://anticdent.org/placement-update-19-00.html Welcome to the first placement update of 2019. May all your placements have sufficient resources this year. # Most Important A few different people have mentioned that we're approaching crunch time on pulling the trigger on deleting the placement code from nova. The week of the 14th there will be a meeting to iron out the details of what needs to be done prior to that. If this is important to you, watch out for an announcement of when it will be. This is a separate issue from putting placement under its own governance, but some of the requirements [declared](http://lists.openstack.org/pipermail/openstack-dev/2018-September… for that, notably a deployment tool demonstrating an upgrade from placement-in-nova to placement-alone, are relevant. Therefore, reviewing and tracking the deployment tool related work remains critical. Those are listed below. Also, it is spec freeze next week. There are quite a lot of specs that are relevant to placement and scheduling that are close, but not quite. Mel has sent out [an email](http://lists.openstack.org/pipermail/openstack-discuss/2019-January/… about which specs most need attention. # What's Changed * There's an [os-resource-classes](https://pypi.org/p/os-resource-classes) now, already merged in placement, with a change posted [for nova](https://review.openstack.org/#/c/628278/). It's effectively the same as os-traits, but for resource classes. * There's a release of a 0.1.0 of placement [pending](https://review.openstack.org/628400). This won't have complete documentation, but will mean that there's an actually usable openstack-placement on PyPI, with what we expect to be the final python module requirements. * This has been true for a while, but it seems worth mentioning, via coreycb: "you can install placement-api on bionic with the stein cloud archive enabled". * A `db stamp` command has been added to `placement-manage` tool which makes it possible for someone who has migrated their data from nova to say "I'm at version X". * placement functional tests have been removed from nova. * Matt did a mess of work to make initializing the scheduler report client in nova less expensive and redundant. * Improving the handling of [allocation ratios](https://docs.openstack.org/nova/latest/admin/configuration/schedule… has merged, allowing for "initial allocation ratios". # Bugs * Placement related [bugs not yet in progress](https://goo.gl/TgiPXb): 15. -2. * [In progress placement bugs](https://goo.gl/vzGGDQ) 15. +2 # Specs Spec freeze next week! Only one of the previously listed specs has merged since early December and a new one has been added (at the end). * <https://review.openstack.org/#/c/544683/> Account for host agg allocation ratio in placement (Still in rocky/) * <https://review.openstack.org/#/c/595236/> Add subtree filter for GET /resource_providers * <https://review.openstack.org/#/c/597601/> Resource provider - request group mapping in allocation candidate * <https://review.openstack.org/#/c/549067/> VMware: place instances on resource pool (still in rocky/) * <https://review.openstack.org/#/c/555081/> Standardize CPU resource tracking * <https://review.openstack.org/#/c/599957/> Allow overcommit of dedicated CPU (Has an alternative which changes allocations to a float) * <https://review.openstack.org/#/c/591037/> Modelling passthrough devices for report to placement * <https://review.openstack.org/#/c/603955/> Nova Cyborg interaction specification. * <https://review.openstack.org/#/c/601596/> supporting virtual NVDIMM devices * <https://review.openstack.org/#/c/552924/> Proposes NUMA topology with RPs * <https://review.openstack.org/#/c/569011/> Count quota based on resource class * <https://review.openstack.org/#/c/141219/> Adds spec for instance live resize * <https://review.openstack.org/#/c/612497/> Provider config YAML file * <https://review.openstack.org/#/c/509042/> Propose counting quota usage from placement and API database * <https://review.openstack.org/603545> Resource modeling in cyborg. * <https://review.openstack.org/#/c/609960/> Support filtering of allocation_candidates by forbidden aggregates * <https://review.openstack.org/601596> support virtual persistent memory # Main Themes ## Making Nested Useful Progress continues on gpu-reshaping for libvirt and xen: * <https://review.openstack.org/#/q/topic:bp/reshape-provider-tree+status:open> Also making use of nested is bandwidth-resource-provider: * <https://review.openstack.org/#/q/topic:bp/bandwidth-resource-provider> There's a [review guide](http://lists.openstack.org/pipermail/openstack-discuss/2018-December… for those patches. Eric's in the process of doing lots of cleanups to how often the ProviderTree in the resource tracker is checked against placement, and a variety of other "let's make this more right" changes in the same neighborhood: * Stack at: <https://review.openstack.org/#/c/615677> ## Extraction The [extraction etherpad](https://etherpad.openstack.org/p/placement-extract-stein-4) is starting to contain more strikethrough text than not. Progress is being made. I'll refactor that soon so it is more readable, before the week of the 14th meeting. The main tasks are the reshaper work mentioned above and the work to get deployment tools operating with an extracted placement: * [TripleO](https://review.openstack.org/#/q/topic:tripleo-placement-extractio… * [OpenStack Ansible](https://review.openstack.org/#/q/project:openstack/openstack-ansib… * [Kolla and Kolla Ansible](https://review.openstack.org/#/q/topic:split-placement) Loci's change to have an extracted placement has merged. Kolla has a patch to [include the upgrade script](https://review.openstack.org/#/q/topic:upgrade-placement). It raises the question of how or if the `mysql-migrate-db.sh` should be distributed. Should it maybe end up in the pypi distribution? Documentation tuneups: * Release-notes: <https://review.openstack.org/#/c/618708/> This is blocked until we refactor the release notes to reflect _now_ better. * The main remaining task here is participating in [openstack-manuals](https://docs.openstack.org/doc-contrib-guide/doc-index.h…, to that end: * <https://review.openstack.org/#/c/628183/> A stack of changes to nova to remove placement from the install docs. * <https://review.openstack.org/#/c/628220/> Install docs in placement. I wrote to the [mailing list](http://lists.openstack.org/pipermail/openstack-discuss/2019-January/0… asking for input on making sure these things are close to correct, especially with regard to distro-specific things like package names. * <https://review.openstack.org/#/c/628324> Change to openstack-manuals to assert that placement is publishing install docs. Depends on the above. * There is a patch to [delete placement](https://review.openstack.org/#/c/618215/) from nova that we've put an administrative -2 on while we determine where things are (see about the meeting above). * There's a pending patch to support [online data migrations](https://review.openstack.org/#/c/624942/). This is important to make sure that fixup commands like `create_incomplete_consumers` can be safely removed from nova and implemented in placement. # Other There are currently 13 [open changes](https://review.openstack.org/#/q/project:openstack/placement+statu… in placement itself. Most of the time critical work is happening elsewhere (notably the deployment tool changes listed above). Of those placement changes, the [database-related](https://review.openstack.org/#/q/owner:nakamura.tetsuro%2… ones from Tetsuro are the most important. Outside of placement: * <https://review.openstack.org/#/q/topic:minimum-bandwidth-allocation-placeme…> Neutron minimum bandwidth implementation * <https://review.openstack.org/#/c/602160/> Add OWNERSHIP $SERVICE traits * <https://review.openstack.org/#/c/586960/> zun: Use placement for unified resource management * <https://review.openstack.org/#/c/621645/> WIP: add Placement aggregates tests (in tempest) * <https://review.openstack.org/627326> blazar: Consider the number of reservation inventory * <https://review.openstack.org/622316> Add placement client for basic GET operations (to tempest) # End Lot's of good work in progress. Our main task is making sure it all gets review and merged. The sooner we do, the sooner people get to use it and find all the bugs we're sure to have left lying around. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent tw: @anticdent

1 0

[oslo] Problem when use library "oslo.messaging" for HA Openstack
by Thành Nguyễn Bá 04 Jan '19

04 Jan '19

Dear all, I have a problem when use 'notification listener' oslo-message for HA Openstack. It raise 'oslo_messaging.exceptions.MessageDeliveryFailure: Unable to connect to AMQP server on 172.16.4.125:5672 after inf tries: Exchange.declare: (406) PRECONDITION_FAILED - inequivalent arg 'durable' for exchange 'nova' in vhost '/': received 'false' but current is 'true''. How can i fix this?. I think settings default in my program set 'durable' is False so it can't listen RabbitMQ Openstack? This is my nova.conf > http://paste.openstack.org/show/738813/ And section [oslo_messaging_rabbit] [oslo_messaging_rabbit] > rabbit_ha_queues = true > rabbit_retry_interval = 1 > rabbit_retry_backoff = 2 > amqp_durable_queues= true *Nguyễn Bá Thành* *Mobile*: 0128 748 0391 *Email*: bathanhtlu(a)gmail.com

3 4

Openstack CLI using a non-zero return code for successful command.
by Rafael Weingärtner 04 Jan '19

04 Jan '19

Hello OpenStackers, I have been using "openstack" CLI for a while now, and for most of the commands (the ones I used so far), when it is successful, I receive a return code (RC) 0 (ZERO). However, when using the command "openstack federation protocol set --identity-provider <idP> --mapping <mappingName> <protocol>", I am getting an RC 1 (ONE) as the exit code for successful executions as well. Is this a known bug? -- Rafael Weingärtner

1 0

[nova][dev] tracking spec reviews ahead of spec freeze
by melanie witt 04 Jan '19

04 Jan '19

Hey all, Spec freeze is coming up shortly after the holidays on January 10, 2019. Since we don't have much time after returning to work next year before spec freeze, let's keep track of specs that are close to approval or otherwise candidates to focus on in the last stretch before freeze. Here's an etherpad where I've collected a list of possible candidates for focus the first week of January. Feel free to add notes and specs I might have missed: https://etherpad.openstack.org/p/nova-stein-blueprint-spec-freeze Cheers, -melanie

2 3

[nova][ops] Trying to get per-instance live migration timeout action spec unstuck
by Matt Riedemann 04 Jan '19

04 Jan '19

I'm looking at this (previously approved [1]) spec again [2] and trying to sort out what needs to happen to reach agreement on this feature. Note the dependent blueprint is now complete in Stein [3]. The idea is pretty simple: provide new parameters to the live migration API to (1) override [libvirt]/live_migration_completion_timeout [4] and/or (2) provide a timeout action in case the provided (or configured) timeout is reached which would override [libvirt]/live_migration_timeout_action [5]. The use case is also pretty simple: you can have a default timeout and action (abort) configured but there could be cases where you need to override that on a per-instance basis to move a set of VMs off a host for maintenance, so you want to tell nova to force complete (post-copy or pause) in case of a timeout. The abort and force-complete actions are the same as in the API ([6] and [7] respectively). There are two main sticking points against this in the review: 1. This can already be done using existing APIs (as noted) client-side if monitoring the live migration and it times out for whatever you consider a reasonable timeout at the time. 2. The libvirt driver is the only one that currently supports abort and force-complete. For #1, while valid as a workaround, is less than ideal since it would mean having to orchestrate that into any tooling that needs that kind of workaround, be that OSC, openstacksdk, python-novaclient, gophercloud, etc. I think it would be relatively simple to pass those parameters through with the live migration request down to nova-compute and have the parameters override the config options and then it's natively supported in the API. For #2, while also true, I think is not a great reason *not* to support per-instance timeouts/actions in the API when we already have existing APIs that do the same thing and have the same backend compute driver limitations. To ease this, I think we can sort out two things: a) Can other virt drivers that support live migration (xenapi, hyperv, vmware in tree, and powervm out of tree) also support abort and force-complete actions? John Garbutt at least thought it should be possible for xenapi at the Stein PTG. I don't know about the others - driver maintainers please speak up here. The next challenge would be getting driver maintainers to actually add that feature parity, but that need not be a priority for Stein as long as we know it's possible to add the support eventually. b) There are pre-live migration checks that happen on the source compute before we initiate the actual guest transfer. If a user (admin) specified these new parameters and the driver does not support them, we could fail the live migration early. This wouldn't change the instance status but the migration would fail and an instance action event would be recorded to explain why it didn't work, and then the admin can retry without those parameters. This would shield us from exposing something in the API that could give a false sense of functionality when the backend doesn't support it. Given all of this, are these reasonable compromises to continue trying to drive this feature forward, and more importantly, are other operators looking to see this functionality added to nova? Huawei public cloud operators want it because they routinely are doing live migrations as part of maintenance activities and want to be able to control these values per-instance. I assume there are other deployments that would like the same. If this is something you'd like to see move forward, please speak up soon since the nova spec freeze for Stein is January 10. [1] https://specs.openstack.org/openstack/nova-specs/specs/pike/approved/live-m… [2] https://review.openstack.org/#/c/600613/ [3] https://blueprints.launchpad.net/nova/+spec/live-migration-force-after-time… [4] https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.li… [5] https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.li… [6] https://developer.openstack.org/api-ref/compute/?expanded=#delete-abort-mig… [7] https://developer.openstack.org/api-ref/compute/?expanded=#force-migration-… -- Thanks, Matt

5 9

[openstack-dev] [neutron] Cancelling Neutron drivers meeting on January 4th
by Miguel Lavalle 03 Jan '19

03 Jan '19

Hi Neutrinos, Today I spent time triaging RFEs. I posted comments and questions in several of them in order to get them in good shape to be discussed by the Drivers team. None are ready to be discussed, though. As a consequence. I am cancelling the meeting on January 4th. Best regards Miguel

1 0

[loci] Loci Meeting for January 3, 2019
by Chris Hoge 03 Jan '19

03 Jan '19

On Friday, January 3 we will resume our Loci team meetings at 7 AM PT/ 15 UTC after an extended end-of-year break. We will revisit our plan for building stable branches (in light of several failures to build stable branches on a few distributions because of requirements failures), coming up with a more robust testing strategy, and planning new development priorities for the remainder of the cycle. Thanks, Chris https://etherpad.openstack.org/p/loci-meeting

1 1

[tripleo] OVB stable/1.0 branch created
by Ben Nemec 03 Jan '19

03 Jan '19

Just a quick update on the status of OVB. There's some discussion on [1] about how to handle importing it to Gerrit, and it looks like we first want to wrap up the 2.0-dev feature branch so we don't have to mess with that after import. As a result, I've cut the stable/1.0 branch from current OVB master. Anyone who's not ready to try out 2.0 should start pointing at stable/1.0 instead of master as the 2.0-dev branch will soon be merged back to master. This will leave us with a branch structure that better matches most other OpenStack projects. We should probably give consumers some time to get switched over, but it sounds like TripleO CI already pins to a specific commit in OVB so it may not affect that. Once a little time has passed we can do the 2.0-dev merge and proceed with the import. I also made a wordier blog post about this[2]. Let me know if you have any feedback. Thanks. -Ben 1: https://review.openstack.org/#/c/620613/ 2: http://blog.nemebean.com/content/openstack-virtual-baremetal-import-plans

2 1

[tripleo] Scenario Standalone ci jobs update - voting and promotion pipeline
by Marios Andreou 03 Jan '19

03 Jan '19

o/ TripleO's & Happy New Year \o/ if you are tracking the ci squad you may know that one area of focus recently is moving the scenario-multinode jobs to harder/better/faster/stronger (most importantly *smaller* but not as cool) scenario-standalone equivalents. Scenarios 1-4 are now merged [1]. For the current sprint [2] ci squad is doing cleanup on those. This includes making sure the new jobs are used in all the places the multinode jobs were e.g.[3][4] (& scens 2/3 will follow) and fixing any missing services or any other nits we find. Once done we can move on to the rest - scenarios 5/6 etc. We are looking for any feedback about the jobs in general or any one in particular if you have some special interest in a particular service (see [5] for reminder about services and scenarios). Most importantly those jobs are now being set as voting (e.g. already done for 1/4 at [1]) and the next natural step once voting is to add them into the master promotion pipeline. Please let us know if you think this is a bad idea or with any other feedback or suggestion. regards &thanks for reading! marios [1] https://github.com/openstack-infra/tripleo-ci/blob/3d634dc2874f95a9d4fd97a1… [2] https://tree.taiga.io/project/tripleo-ci-board/taskboard/unified-sprint-3 [3] https://review.openstack.org/#/q/topic:replace-scen1 [4] https://review.openstack.org/#/q/topic:replace-scen4 [5] https://github.com/openstack/tripleo-heat-templates#service-testing-matrix

3 2