July 2019 - openstack-discuss - lists.openstack.org

[nova] instence running, but status is Error
by Budai Laszlo 26 Jul '19

26 Jul '19

Dear all, I am testing different migration scenarios in our openstack, and I ended up with an instance that is running (I can access it by ssh), but its Status is Error. and its task_state is resize_migrating. I suppose that I got here due to the fact that the `allow_resize_to_same_host` is set to True, and the scheduler has tried to use the same compute node in the migration as described here: https://bugs.launchpad.net/nova/+bug/1811235. Is there any possibility to be able to manage again that instance using openstack (except to delete it)? I have tried the shut off (from horizon) and it gives me an error: fault: code: 400 created: '2019-07-26T14:46:05Z' message: Unavailable console type novnc. Thank you, Laszlo

2 3

[placement] update 19-29
by Chris Dent 26 Jul '19

26 Jul '19

HTML: https://anticdent.org/placement-update-19-29.html Welcome to a rushed pupdate 19-29. My morning was consumed by other things. A reminder: The Placement project holds office hours every Wednesday at 1500 UTC in the `#openstack-placement` IRC channel. If you have a topic that needs some synchronous discussion, then is an ideal time. Just start talking! # Most Important The two main things on the Placement radar are implementing Consumer Types and cleanups, performance analysis, and documentation related to nested resource providers. # What's Changed * The [api-ref](https://docs.openstack.org/api-ref/placement/) has moved to `docs.openstack.org` from `developer.openstack.org`. Redirects are in place. * Both traits and resource classes are [now cached](https://review.opendev.org/672298) per request, allowing for some name to id and id to name optimizations. * A new zuul template is [being used](https://review.opendev.org/671257) in placement that means fewer irrelevant tempest tests are run on placement changes. # Stories/Bugs (Numbers in () are the change since the last pupdate.) There are 21 (-1) stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement). 0 (0) are [untagged](https://storyboard.openstack.org/#!/worklist/580). 2 (0) are [bugs](https://storyboard.openstack.org/#!/worklist/574). 5 (0) are [cleanups](https://storyboard.openstack.org/#!/worklist/575). 10 (0) are [rfes](https://storyboard.openstack.org/#!/worklist/594). 4 (0) are [docs](https://storyboard.openstack.org/#!/worklist/637). If you're interested in helping out with placement, those stories are good places to look. * Placement related nova [bugs not yet in progress](https://goo.gl/TgiPXb) on launchpad: 17 (0). * Placement related nova [in progress bugs](https://goo.gl/vzGGDQ) on launchpad: 4 (0). # osc-placement osc-placement is currently behind by 12 microversions. * <https://review.opendev.org/666542> Add support for multiple member_of. There's been some useful discussion about how to achieve this, and a consensus has emerged on how to get the best results. # Main Themes ## Consumer Types Adding a type to consumers will allow them to be grouped for various purposes, including quota accounting. * <https://review.opendev.org/#/q/topic:bp/support-consumer-types> A WIP, as microversion 1.37, has started. ## Cleanup Cleanup is an overarching theme related to improving documentation, performance and the maintainability of the code. The changes we are making this cycle are fairly complex to use and are fairly complex to write, so it is good that we're going to have plenty of time to clean and clarify all these things. # Other Placement Miscellaneous changes can be found in [the usual place](https://review.opendev.org/#/q/project:openstack/placement+status:op…. There are two [os-traits changes](https://review.opendev.org/#/q/project:openstack/os-traits+status:… being discussed. And zero [os-resource-classes changes](https://review.opendev.org/#/q/project:openstack/os-resource-class… (yay!). # Other Service Users New discoveries are added to the end. Merged stuff is removed. Anything that has had no activity in 4 weeks has been removed. * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.opendev.org/659233> Cyborg: Placement report * <https://review.opendev.org/662229> helm: add placement chart * <https://review.opendev.org/656023> Nova: Use OpenStack SDK for placement * <https://review.opendev.org/634551> libvirt: report pmem namespaces resources by provider tree * <https://review.opendev.org/660852> Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI * <https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports> Nova: support move ops with qos ports * <https://review.opendev.org/664689> Nova: get_ksa_adapter: nix by-service-type confgrp hack * <https://review.opendev.org/666202> Blazar: Create placement client for each request * <https://review.opendev.org/667952> nova: Support filtering of hosts by forbidden aggregates * <https://review.opendev.org/669079> blazar: Send global_request_id for tracing calls * <https://review.opendev.org/668252> Nova: Update HostState.\*\_allocation_ratio earlier * <https://review.opendev.org/670112> Nova: WIP: Add a placement audit command * <https://review.opendev.org/586960> zun: [WIP] Use placement for unified resource management * <https://review.opendev.org/671478> Kayobe: Build placement images by default * <https://review.opendev.org/671312> blazar: Fix placement operations in multi-region deployments * <https://review.opendev.org/672870> Watcher: Getting data from placement when updating datamodel * <https://review.opendev.org/671793> Nova: libvirt: Start reporting PCPU inventory to placement * <https://review.opendev.org/672649> chef: placement_api: create valid apache config before installing package * <https://review.opendev.org/670696> tempest: Add placement API methods for testing routed provider nets * <https://review.opendev.org/672678> openstack-helm: Build placement in OSH-images # End If we get the chance, it will be interesting to start working with placement with 1 million providers. Just to see. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

1 0

[openstack-ansible] weekly update
by Mohammed Naser 26 Jul '19

26 Jul '19

Hi everyone, I'm going to try doing these more often to keep everyone updated on the current state of openstack-ansible (mostly around what is discussed during our office hours / meetings). - Dmitriy has been working on removing unused code in playbooks, mostly around logs as we move to 100% systemd-journald. - We're discussing the proposal to remove os-log-dir-setup.yml, as well as rsyslog_client and rsyslog_server roles as they aren’t used anymore with the move to systemd-journald - We're also discussing removing all the complex bind mounts, which would clean it and simplify the code base. - Our continuous integration has been stable but needs upgrade jobs and increased coverage. Our coverage would go up significantly by running aio_lxc for all supported systems. - Jonathan's work on binding to management IPs is making progress (and seems to be at the Galera stage). - CentOS CI has been very slow and nobody knows why. It seems to be something we've regressed at some point. - A unicast flood issue affecting deployments using linuxbridge that happened inside os-vif needs to be patched. It's still pending os-vif to patch. - Namrata mentioned an issue during the upgrade from Rocky to Stein (when upgrading WSREP SST method from xtrabackup-V2 to mariabackup) that will have to be fixed is Stein. That's something that's in progress now. Thanks everyone for reading! Regards, Mohammed -- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser(a)vexxhost.com W. http://vexxhost.com

2 1

[cinder][ceph][RBD mirroring] Error while creating a mirrored volume
by Ajitha Robert 26 Jul '19

26 Jul '19

Hi all Following picture shows my setup [image: image.png] I m facing following error when i issue a command to enable rbd mirroring on a particular image either manually or through cinder. *"rbd::mirror::InstanceWatcher: C_NotifyInstanceRequest: 0x7fb364025310 finish: resending after timeout"* in rbd-mirroring.log and getting *"failed to get omap key: client_0921b3ce-716a-4462-b3cf-86b5102bfd9"* error in osd.log. . Should I enable anything else apart from timeout parametes, if the distance between the two ceph clusters is greater than 300km? when I create a bootable volume of 100 GB with a glance image.Image get downloaded and from cinder, volume log throws with "volume is busy deleting volume that has snapshot" . Image was enabled with exclusive lock, journaling, layering, object-map, fast-diff and deep-flatten Cinder volume is in error state but the rbd image is created in primary but not in secondary. pls check the logs 1)Log for manual rbd image creation http://paste.openstack.org/show/754766/ 2)Log for 16gb volume created from cinder, status in cinder volume is available http://paste.openstack.org/show/754767/ 3)Log for 100gb volume created from cinder, status in cinder volume is error http://paste.openstack.org/show/754769/ -- *Regards,Ajitha R*

1 0

[placement] update 19-27
by Chris Dent 26 Jul '19

26 Jul '19

HTML: https://anticdent.org/placement-update-19-27.html Pupdate 19-27 is here and now. # Most Important Of the features we planned to do this cycle, all are done save one: consumer types (in progress, see below). This means we have a good opportunity to focus on documentation, performance, and improving the codebase for maintainability. You do not need permission to work on these things. If you find a problem and know how to fix it, fix it. If you are not sure about the solution, please discuss it on this email list or in the `#openstack-placement` IRC channel. This also means we're in a good position to help review changes that use placement in other projects. The Foundation needs to know how much, if any, Placement time will be needed in Shanghai. I started a [thread](http://lists.openstack.org/pipermail/openstack-discuss/2019-July/00… and an [etherpad](https://etherpad.openstack.org/p/placement-shanghai-ptg). # What's Changed * The `same_subtree` query parameter has merged as [microversion 1.36](https://docs.openstack.org/placement/latest/placement-api-microversio…. This enables a form of nested provider affinity: "these providers must all share the same ancestor". * The placement projects have been updated (pending merge) to match the [Python 3 test runtimes for Train](https://governance.openstack.org/tc/goals/train/python3-updates.html) community wide goal. Since the projects have been Python3-enabled from the start, this was mostly a matter of aligning configurations with community-wide norms. When Python 3.8 becomes available we should get on that sooner than later to catch issues early. # Specs/Features All placement specs have merged. Thanks to everyone for the frequent reviews and quick followups. Some non-placement specs are listed in the Other section below. # Stories/Bugs (Numbers in () are the change since the last pupdate.) There are 23 (0) stories in [the placement group](https://storyboard.openstack.org/#!/project_group/placement). 0 (0) are [untagged](https://storyboard.openstack.org/#!/worklist/580). 3 (1) are [bugs](https://storyboard.openstack.org/#!/worklist/574). 5 (0) are [cleanups](https://storyboard.openstack.org/#!/worklist/575). 11 (0) are [rfes](https://storyboard.openstack.org/#!/worklist/594). 4 (0) are [docs](https://storyboard.openstack.org/#!/worklist/637). If you're interested in helping out with placement, those stories are good places to look. * Placement related nova [bugs not yet in progress](https://goo.gl/TgiPXb) on launchpad: 16 (0). * Placement related nova [in progress bugs](https://goo.gl/vzGGDQ) on launchpad: 4 (0). # osc-placement osc-placement is currently behind by 11 microversions. * <https://review.opendev.org/666542> Add support for multiple member_of. This is stuck and needs input from users of the tool. # Main Themes ## Consumer Types Adding a type to consumers will allow them to be grouped for various purposes, including quota accounting. * <https://review.opendev.org/#/q/topic:bp/support-consumer-types> This is currently the migration to add the necessary table and column. ## Cleanup Cleanup is an overarching theme related to improving documentation, performance and the maintainability of the code. The changes we are making this cycle are fairly complex to use and are fairly complex to write, so it is good that we're going to have plenty of time to clean and clarify all these things. As mentioned last week, one of the important cleanup tasks that is not yet in progress is updating the [gabbit](https://opendev.org/openstack/placement/src/branch/master/gate/gabb… that creates the nested topology that's used in nested performance testing. The topology there is simple, unrealistic, and doesn't sufficiently exercise the several features that may be used during a query that desires a nested response. This needs to be someone who is more closely related to real world use of nested than me. efried? gibi? Another cleanup that needs to start is satisfying the community wide goal of [PDF doc generation](https://storyboard.openstack.org/#!/story/2006110). Anyone know if there is a cookbook for this? # Other Placement Miscellaneous changes can be found in [the usual place](https://review.opendev.org/#/q/project:openstack/placement+status:op…. There are three [os-traits changes](https://review.opendev.org/#/q/project:openstack/os-traits+status:… being discussed. And one [os-resource-classes change](https://review.opendev.org/#/q/project:openstack/os-resource-classe…. # Other Service Users New discoveries are added to the end. Merged stuff is removed. Anything that has had no activity in 4 weeks has been removed. * <https://review.openstack.org/#/q/topic:bug/1819923> Nova: nova-manage: heal port allocations * <https://review.openstack.org/650188> nova-spec: Allow compute nodes to use DISK_GB from shared storage RP * <https://review.opendev.org/659233> Cyborg: Placement report * <https://review.opendev.org/662229> helm: add placement chart * <https://review.opendev.org/656023> Nova: Use OpenStack SDK for placement * <https://review.opendev.org/612497> Nova: Spec: Provider config YAML file * <https://review.opendev.org/634551> libvirt: report pmem namespaces resources by provider tree * <https://review.opendev.org/660852> Nova: Remove PlacementAPIConnectFailure handling from AggregateAPI * <https://review.opendev.org/#/q/topic:bp/support-move-ops-with-qos-ports> Nova: support move ops with qos ports * <https://review.opendev.org/664689> Nova: get_ksa_adapter: nix by-service-type confgrp hack * <https://review.opendev.org/664862> OSA: Add nova placement to placement migration * <https://review.opendev.org/657796> Nova: Defaults missing group_policy to 'none' * <https://review.opendev.org/666202> Blazar: Create placement client for each request * <https://review.opendev.org/669309> tempest: Define the Integrated-gate-placement gate template * <https://review.opendev.org/668263> Nova: Restore RT.old_resources if ComputeNode.save() fails * <https://review.opendev.org/667952> nova: Support filtering of hosts by forbidden aggregates * <https://review.opendev.org/669079> blazar: Send global_request_id for tracing calls * <https://review.opendev.org/668252> Nova: Update HostState.\*\_allocation_ratio earlier * <https://review.opendev.org/670105> neutron: segments: fix rp inventory update * <https://review.opendev.org/670112> Nova: WIP: Add a placement audit command * <https://review.opendev.org/668526> OSA: Add placement client for neutron * <https://review.opendev.org/586960> zun: [WIP] Use placement for unified resource management # End A colleague suggested yesterday that the universe doesn't have an over subscription problem, rather there's localized contention, and what we really have is a placement problem. -- Chris Dent ٩◔̯◔۶ https://anticdent.org/ freenode: cdent

2 1

[keystone] Need clarification about Stein policies
by Bernd Bausch 25 Jul '19

25 Jul '19

The Keystone policy.json file I created with oslo-policy-generator contains lines I don't understand. For example /list_users/. The comment says: # DEPRECATED "identity:list_users":"rule:admin_required" has been # deprecated since S in favor of "identity:list_users":"(role:reader # and system_scope:all) or (role:reader and # domain_id:%(target.domain_id)s)". I do understand the expression starting with (role:reader .... , but contrarily to the comment, the policy is "identity:list_users": "rule:identity:list_users" This looks like a circular definition, and in any case, nowhere do I seerule:identity:list_users defined. Can someone in the know explain how this policy is processed? Thanks much, Bernd

2 1

[dev][nova][glance][cinder] introducing 'compressed' image container_format
by Ma, Zheng 25 Jul '19

25 Jul '19

Hello Nova developers, Cinder is working on a feature that will allow it to compress volumes uploaded to Glance [0,1]. We are proposing a new container_format for Glance named 'compressed' [2]. The idea is that a consumer of a 'compressed' image would read the file to determine the kind of compression and take appropriate action (either successfully decompress to find a payload that matches the image's disk_format, or fail with 'unsupported container_format'). Our preliminary testing with a raw image with container_format == compressed, disk_format == raw is that Nova boots a VM from the image but the VM appears to be unusable. Nova seems taking it as a real RAW image as disk_format indicates. So we want to open a discussion about handling this. Thanks! [0] cinder spec patch https://review.opendev.org/#/c/652275/ [1] cinder code implementation https://review.opendev.org/#/c/668825 [2] glance spec patch https://review.opendev.org/#/c/670454/

1 0

[Nova] migration issue
by Budai Laszlo 25 Jul '19

25 Jul '19

Dear all, we are testing the cold migratoin of instances that are using local ephemeral storage and it fails with the following error: 2019-07-24 13:47:13.115 58902 INFO nova.virt.libvirt.driver [req-356fce31-5e98-425f-89d6-8a98664d31ad 122b0950c0cc47bdbb78e63724d65105 f713e44c723e491aa67352e12f83e0d7 - default default] [instance: 5a6dae 58-00d6-4317-b635-909fdf09ac49] Instance shutdown successfully after 3 seconds. 2019-07-24 13:47:13.126 58902 INFO nova.virt.libvirt.driver [-] [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Instance destroyed successfully. 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [req-356fce31-5e98-425f-89d6-8a98664d31ad 122b0950c0cc47bdbb78e63724d65105 f713e44c723e491aa67352e12f83e0d7 - default default] [instance: 5a6dae58- 00d6-4317-b635-909fdf09ac49] Setting instance vm_state to ERROR: ProcessExecutionError: Unexpected error while running command. Command: scp -r /var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 192.168.56.46:/dev/ssd_nova/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph 0 Exit code: 1 Stdout: u'' Stderr: u'------------------------------------------------------------------------------\n* WARNING *\n* You are accessing a secured syst em and your actions will be logged along *\n* with identifying information. Disconnect immediately if you are not an *\n* authorized user of this system. * \n------------------------------------------------------------------------------\n/var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0: No suc h file or directory\n' 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Traceback (most recent call last): 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 75 18, in _error_out_instance_on_exception 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] yield 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 42 75, in _resize_instance 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] timeout, retry_interval) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", lin e 8200, in migrate_disk_and_power_off 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] shared_storage) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220 , in __exit__ 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] self.force_reraise() 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196 , in force_reraise 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", lin e 8185, in migrate_disk_and_power_off 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] compression=compression) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 226, in copy_image 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] compression=compression) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/volume/remotefs .py", line 110, in copy_file 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] compression=compression) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/volume/remotefs .py", line 196, in copy_file 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] on_execute=on_execute, on_completion=on_completion) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/utils.py", line 231, in exec ute 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] return processutils.execute(*cmd, **kwargs) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_concurrency/processutils.py" , line 424, in execute 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] cmd=sanitized_cmd) 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] ProcessExecutionError: Unexpected error while running command. 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Command: scp -r /var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 192.168.56.46:/dev/ssd_nova/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Exit code: 1 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Stdout: u'' 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Stderr: u'------------------------------------------------------------------------------\n* WARNING *\n* You are accessing a secured system and your actions will be logged along *\n* with identifying information. Disconnect immediately if you are not an *\n* authorized user of this system. *\n------------------------------------------------------------------------------\n/var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0: No such file or directory\n' 2019-07-24 13:47:14.008 58902 ERROR nova.compute.manager [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] 2019-07-24 13:47:14.380 58902 INFO nova.compute.manager [req-356fce31-5e98-425f-89d6-8a98664d31ad 122b0950c0cc47bdbb78e63724d65105 f713e44c723e491aa67352e12f83e0d7 - default default] [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Swapping old allocation on 3da11b23-5ef8-4471-a3c9-5f743b0bbfd7 held by migration da66c416-2636-421b-99cb-470bd07dad4c for instance 2019-07-24 13:47:14.979 58902 INFO nova.compute.manager [req-356fce31-5e98-425f-89d6-8a98664d31ad 122b0950c0cc47bdbb78e63724d65105 f713e44c723e491aa67352e12f83e0d7 - default default] [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] Successfully reverted task state from resize_migrating on failure for instance. 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server [req-356fce31-5e98-425f-89d6-8a98664d31ad 122b0950c0cc47bdbb78e63724d65105 f713e44c723e491aa67352e12f83e0d7 - default default] Exception during message handling: ProcessExecutionError: Unexpected error while running command. Command: scp -r /var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 192.168.56.46:/dev/ssd_nova/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 Exit code: 1 Stdout: u'' Stderr: u'------------------------------------------------------------------------------\n* WARNING *\n* You are accessing a secured system and your actions will be logged along *\n* with identifying information. Disconnect immediately if you are not an *\n* authorized user of this system. *\n------------------------------------------------------------------------------\n/var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0: No such file or directory\n' 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server function_name, call_dict, binary) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 186, in decorated_function 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 156, in decorated_function 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/utils.py", line 977, in decorated_function 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 214, in decorated_function 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 202, in decorated_function 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 4240, in resize_instance 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self._revert_allocation(context, instance, migration) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 4237, in resize_instance 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server instance_type, clean_shutdown) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/compute/manager.py", line 4275, in _resize_instance 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server timeout, retry_interval) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 8200, in migrate_disk_and_power_off 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server shared_storage) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server self.force_reraise() 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 8185, in migrate_disk_and_power_off 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server compression=compression) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 226, in copy_image 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server compression=compression) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/volume/remotefs.py", line 110, in copy_file 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server compression=compression) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/virt/libvirt/volume/remotefs.py", line 196, in copy_file 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server on_execute=on_execute, on_completion=on_completion) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/nova/utils.py", line 231, in execute 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server return processutils.execute(*cmd, **kwargs) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server File "/openstack/venvs/nova-17.1.3/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 424, in execute 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server cmd=sanitized_cmd) 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server ProcessExecutionError: Unexpected error while running command. 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server Command: scp -r /var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 192.168.56.46:/dev/ssd_nova/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server Exit code: 1 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server Stdout: u'' 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server Stderr: u'------------------------------------------------------------------------------\n* WARNING *\n* You are accessing a secured system and your actions will be logged along *\n* with identifying information. Disconnect immediately if you are not an *\n* authorized user of this system. *\n------------------------------------------------------------------------------\n/var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0: No such file or directory\n' 2019-07-24 13:47:15.000 58902 ERROR oslo_messaging.rpc.server 2019-07-24 13:47:17.065 58902 INFO nova.virt.libvirt.imagecache [req-e73474d0-6857-4c52-9c93-73f3bae247b5 - - - - -] image ca89475a-cf15-46d1-ab2d-c2b96c403248 at (/var/lib/nova/instances/_base/ab9f9dd0888a4950702b9c993c82f7ab36a3ed07): checking 2019-07-24 13:47:17.066 58902 INFO nova.virt.libvirt.imagecache [req-e73474d0-6857-4c52-9c93-73f3bae247b5 - - - - -] Active base files: /var/lib/nova/instances/_base/ab9f9dd0888a4950702b9c993c82f7ab36a3ed07 2019-07-24 13:47:27.768 58902 INFO nova.compute.manager [-] [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] VM Stopped (Lifecycle Event) 2019-07-24 13:47:27.918 58902 INFO nova.compute.manager [req-e53e928d-81fe-4185-9e62-abc2113e838c - - - - -] [instance: 5a6dae58-00d6-4317-b635-909fdf09ac49] During _sync_instance_power_state the DB power_state (1) does not match the vm_power_state from the hypervisor (4). Updating power_state in the DB to match the hypervisor. The disk has its root FS n an RBD device (a volume created by cinder) and it has its ephemeral disk on a local LVM volume created by nova. It seems that the /var/lib/nova/instances/5a6dae58-00d6-4317-b635-909fdf09ac49_resize/5a6dae58-00d6-4317-b635-909fdf09ac49_disk.eph0 is not there when scp is trying to copy it to the other side. Any ideas how to fix this? The nova config on the nodes looks like this: # Ansible managed [DEFAULT] allow_resize_to_same_host = True # Compute compute_driver = libvirt.LibvirtDriver # Scheduler cpu_allocation_ratio = 2.0 # Logs / State debug = False # Hypervisor default_ephemeral_format = ext4 disk_allocation_ratio = 1.0 # Api's enabled_apis = osapi_compute,metadata executor_thread_pool_size = 64 fatal_deprecations = False # Configdrive force_config_drive = False host = compute8.mgmt.lab.mydomain.com image_cache_manager_interval = 0 instance_name_template = instance-%08x # Ceilometer notification configurations instance_usage_audit = True instance_usage_audit_period = hour instances_path = /var/lib/nova/instances ## Vif libvirt_vif_type = ethernet log_dir = /var/log/nova # Metadata metadata_workers = 16 # Network my_ip = 192.168.56.47 notify_on_state_change = vm_and_task_state osapi_compute_workers = 16 ram_allocation_ratio = 1.0 reserved_host_disk_mb = 0 reserved_host_memory_mb = 2048 resume_guests_state_on_host_boot = False rootwrap_config = /etc/nova/rootwrap.conf rpc_response_timeout = 60 service_down_time = 120 state_path = /var/lib/nova # Rpc all transport_url = rabbit://nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e8cf8bd7@192.168.56.194:5671,nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e8cf8bd7@192.168.56.209:5671,nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e 8cf8bd7@192.168.56.179:5671//nova # Disable stderr logging use_stderr = False vif_plugging_is_fatal = True vif_plugging_timeout = 30 [api] auth_strategy = keystone enable_instance_password = True use_forwarded_for = False vendordata_jsonfile_path = /etc/nova/vendor_data.json # Cache [cache] backend = oslo_cache.memcache_pool enabled = true memcache_servers = 192.168.56.68:11211,192.168.56.85:11211,192.168.56.170:11211 # Cinder [cinder] cafile = /etc/ssl/certs/ca-certificates.crt catalog_info = volumev3:cinderv3:internalURL cross_az_attach = True os_region_name = A_Lab [conductor] workers = 16 [filter_scheduler] available_filters = nova.scheduler.filters.all_filters enabled_filters = RetryFilter, AvailabilityZoneFilter, RamFilter, AggregateRamFilter, ComputeFilter, AggregateCoreFilter, DiskFilter, AggregateDiskFilter, AggregateNumInstancesFilter, AggregateIoOpsFilter, ComputeCapabilitiesFilter, ImagePropertiesFilter, ServerGroupAntiAffinityFilter, ServerGroupAffinityFilter, NUMATopologyFilter, SameHostFilter, DifferentHostFilter host_subset_size = 10 max_instances_per_host = 50 max_io_ops_per_host = 10 ram_weight_multiplier = 5.0 tracks_instance_changes = True weight_classes = nova.scheduler.weights.all_weighers # Glance [glance] api_servers = https://vip.mgmt.lab.mydomain.com:9292 cafile = /etc/ssl/certs/ca-certificates.crt [key_manager] fixed_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx [keystone_authtoken] auth_type = password auth_uri = https://vip.mgmt.lab.mydomain.com:5000 auth_url = https://vip.mgmt.lab.mydomain.com:35357 cafile = /etc/ssl/certs/ca-certificates.crt insecure = False memcache_secret_key = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx # if your memcached server is shared, use these settings to avoid cache poisoning memcache_security_strategy = ENCRYPT memcached_servers = 192.168.56.68:11211,192.168.56.85:11211,192.168.56.170:11211 password = xxxxxxxxxxxxxxxxxxxxxxx project_domain_id = default project_name = service region_name = A_Lab token_cache_time = 300 user_domain_id = default username = nova [libvirt] disk_cachemodes = network=writeback hw_disk_discard = unmap images_rbd_ceph_conf = /etc/ceph/ceph.conf images_rbd_pool = vms images_type = lvm images_volume_group = ssd_nova inject_key = False inject_partition = -2 inject_password = False live_migration_tunnelled = True live_migration_uri = "qemu+ssh://nova@%s/system?no_verify=1&keyfile=/var/lib/nova/.ssh/id_rsa" rbd_secret_uuid = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx # ceph rbd support rbd_user = cinder remove_unused_resized_minimum_age_seconds = 3600 use_virtio_for_bridges = True virt_type = kvm # Neutron [neutron] auth_type = password # Keystone client plugin authentication URL option auth_url = https://vip.mgmt.lab.mydomain.com:35357/v3 cafile = /etc/ssl/certs/ca-certificates.crt default_floating_pool = public insecure = False metadata_proxy_shared_secret = xxxxxxxxxxxxxxxxxxxxxxxxxxxx # Keystone client plugin password option password = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx project_domain_name = Default project_name = service region_name = A_Lab service_metadata_proxy = True url = https://vip.mgmt.lab.mydomain.com:9696 user_domain_name = Default # Keystone client plugin username option username = neutron [oslo_concurrency] lock_path = /var/lock/nova # Notifications [oslo_messaging_notifications] driver = messagingv2 notification_topics = notifications transport_url = rabbit://nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e8cf8bd7@192.168.56.194:5671,nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e8cf8bd7@192.168.56.209:5671,nova:09c23ad7680fc920ab466c5ad5f6c88c2e2e8cf8bd7@192.168.56.179:5671//nova [oslo_messaging_rabbit] rpc_conn_pool_size = 30 ssl = True # Placement [placement] auth_type = "password" auth_url = https://vip.mgmt.lab.mydomain.com:35357/v3 cafile = /etc/ssl/certs/ca-certificates.crt insecure = False os_interface = internal os_region_name = A_Lab password = xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx project_domain_name = Default project_name = service user_domain_name = Default username = placement [quota] cores = 20 injected_file_content_bytes = 10240 injected_file_path_length = 255 injected_files = 5 instances = 10 key_pairs = 100 max_age = 0 metadata_items = 128 ram = 32768 server_group_members = 10 server_groups = 10 [scheduler] discover_hosts_in_cells_interval = 60 host_manager = host_manager max_attempts = 5 periodic_task_interval = 60 scheduler_driver = filter_scheduler [spice] agent_enabled = True enabled = True # Console Url and binds html5proxy_base_url = https://dashboard.iaas.lab.mydomain.com:6080/spice_auto.html server_listen = 192.168.56.47 server_proxyclient_address = 192.168.56.47 [upgrade_levels] compute = auto [vnc] enabled = False [wsgi] api_paste_config = /etc/nova/api-paste.ini secure_proxy_ssl_header = HTTP_X_FORWARDED_PROTO Thank you in advance for any suggestions. Kind regards, Laszlo

5 10

[nova] Train Spec Freeze
by Eric Fried 25 Jul '19

25 Jul '19

Hi Nova contributors and maintainers. Spec freeze for Nova is Thursday, July 25th. This means that any specs not approved by EOD [1] become drastically less likely to land in Train. For your convenience, here is a search that's intended to catch open Train specs while filtering out things like updates to existing specs: https://review.opendev.org/#/q/project:openstack/nova-specs+status:open+pat… At the time of this writing, there are 15 on that list. Of those, IMO two are "close": Add Unified Limits Spec - https://review.opendev.org/602201 - Two +1s, needs review from "Nova API subteam" Spec for the Nova part of Image Encryption - https://review.opendev.org/608696 - +3, needs a second core If you own a spec you think should be able to land [2], please: - Check it to make sure any comments have been addressed (even if there's no -1). - Explicitly ask for reviews, here on the mailing list and/or in IRC in #openstack-nova. Reviewers, please make an effort to ensure that all the specs on that list have at least one vote. (Currently there are six with no vote.) I'll send an update Friday or Monday with a final tally. Thanks, efried [1] i.e. by the time cores have signed off for the day, wherever they are [2] meaning it has already been through some reviews and seems to have support. If you just proposed your spec this week, it's not reasonable to expect it to land.

2 1

Heat resource types - aodh and ceilometer
by Dhairyasheel Ghadge 25 Jul '19

25 Jul '19

Hello all, I have installed heat engine along with heat dashboard for orchestration in openstack cloud. I am creating a template to autoscale a web server pool and spin a new VM when load on other one increases. For the autoscaling purpose, I require to have an alarming system which will monitor the cp meter of the instance, which is provided by ceilometer and aodh services of openstack. I cannot find resource types of aodh and ceilometer on the heat dashboard. These same resource types are present in the heat openstack resources directory. Are there any steps to be carried out to make it available on the dashboard? Thanks & Regards Dhairyasheel

1 0