Strange situation regarding Availability Zones
Hello everyone,
We have a strange situation regarding Availability Zones in our Openstack environment (version: Queens).
We have 3 AZs: room1, room2, and room3.
There are duplicate entries in the output of this command (openstack-pythonclient version 6.0.0):
➜ openstack availability zone list +-----------+-------------+ | Zone Name | Zone Status | +-----------+-------------+ | internal | available | | room2 | available | | room1 | available | | room3 | available | | nova | available | | room2 | available | | room3 | available | | room1 | available | | room2 | available | | room3 | available | +-----------+-------------+
However, in our database, I cannot find any reference to these duplicate entries.
What's going on here?
And how should we go about removing the duplicate entries if they don't show in the database?
MariaDB [nova_api]> select * from aggregate_metadata; +---------------------+------------+----+--------------+-------------------+-----------------+ | created_at | updated_at | id | aggregate_id | key | value | +---------------------+------------+----+--------------+-------------------+-----------------+ | 2019-08-07 06:11:47 | NULL | 1 | 1 | hypervisor | standard | | 2019-08-07 06:22:39 | NULL | 10 | 10 | availability_zone | room1 | | 2019-08-07 06:22:43 | NULL | 13 | 13 | availability_zone | room2 | | 2019-08-15 11:31:39 | NULL | 16 | 16 | availability_zone | room3 | | 2019-09-11 07:50:22 | NULL | 17 | 17 | hypervisor | test | | 2021-08-06 08:43:28 | NULL | 20 | 20 | hypervisor | ephemeral_local | +---------------------+------------+----+--------------+-------------------+-----------------+ 6 rows in set (0.00 sec)
MariaDB [nova_api]> select * from aggregates; +---------------------+------------+----+--------------------------------------+------------------+ | created_at | updated_at | id | uuid | name | +---------------------+------------+----+--------------------------------------+------------------+ | 2019-08-07 06:11:34 | NULL | 1 | 5f6baa56-252d-4f06-998f-5d166957490d | compute_standard | | 2019-08-07 06:22:39 | NULL | 10 | f7805dfb-00d2-458b-bb30-24919b8c9cd2 | room1 | | 2019-08-07 06:22:43 | NULL | 13 | b03f7028-f303-47a2-b6ca-128065f23030 | room2 | | 2019-08-15 11:31:39 | NULL | 16 | 1aff7a43-695a-4a10-9b24-116af73d18c1 | room3 | | 2019-09-11 07:50:02 | NULL | 17 | 952fa0aa-2ef6-443e-a906-76021acced0a | compute_test | | 2021-08-06 08:42:40 | NULL | 20 | bab386c4-f1a3-4482-b0ea-74c74fb172bd | ephemeral_local | +---------------------+------------+----+--------------------------------------+------------------+ 6 rows in set (0.00 sec)
Thank you.
Best regards, Lennart
On Tue, 2022-11-22 at 10:55 +0000, Lennart van Gijtenbeek | Routz wrote:
Hello everyone,
We have a strange situation regarding Availability Zones in our Openstack environment (version: Queens).
We have 3 AZs: room1, room2, and room3.
There are duplicate entries in the output of this command (openstack- pythonclient version 6.0.0):
➜ openstack availability zone list +-----------+-------------+ | Zone Name | Zone Status | +-----------+-------------+ | internal | available | | room2 | available | | room1 | available | | room3 | available | | nova | available | | room2 | available | | room3 | available | | room1 | available | | room2 | available | | room3 | available | +-----------+-------------+
Please run this command with debug mode enabled (--debug). That should give you far more information regarding what's going on here.
Stephen
However, in our database, I cannot find any reference to these duplicate entries.
What's going on here?
And how should we go about removing the duplicate entries if they don't show in the database?
MariaDB [nova_api]> select * from aggregate_metadata; +---------------------+------------+----+--------------+-------------------+-- ---------------+ | created_at | updated_at | id | aggregate_id | key | value | +---------------------+------------+----+--------------+-------------------+-- ---------------+ | 2019-08-07 06:11:47 | NULL | 1 | 1 | hypervisor | standard | | 2019-08-07 06:22:39 | NULL | 10 | 10 | availability_zone | room1 | | 2019-08-07 06:22:43 | NULL | 13 | 13 | availability_zone | room2 | | 2019-08-15 11:31:39 | NULL | 16 | 16 | availability_zone | room3 | | 2019-09-11 07:50:22 | NULL | 17 | 17 | hypervisor | test | | 2021-08-06 08:43:28 | NULL | 20 | 20 | hypervisor | ephemeral_local | +---------------------+------------+----+--------------+-------------------+-- ---------------+ 6 rows in set (0.00 sec)
MariaDB [nova_api]> select * from aggregates; +---------------------+------------+----+------------------------------------- -+------------------+ | created_at | updated_at | id | uuid | name | +---------------------+------------+----+------------------------------------- -+------------------+ | 2019-08-07 06:11:34 | NULL | 1 | 5f6baa56-252d-4f06-998f-5d166957490d | compute_standard | | 2019-08-07 06:22:39 | NULL | 10 | f7805dfb-00d2-458b-bb30-24919b8c9cd2 | room1 | | 2019-08-07 06:22:43 | NULL | 13 | b03f7028-f303-47a2-b6ca-128065f23030 | room2 | | 2019-08-15 11:31:39 | NULL | 16 | 1aff7a43-695a-4a10-9b24-116af73d18c1 | room3 | | 2019-09-11 07:50:02 | NULL | 17 | 952fa0aa-2ef6-443e-a906-76021acced0a | compute_test | | 2021-08-06 08:42:40 | NULL | 20 | bab386c4-f1a3-4482-b0ea-74c74fb172bd | ephemeral_local | +---------------------+------------+----+------------------------------------- -+------------------+ 6 rows in set (0.00 sec)
Thank you.
Best regards, Lennart
Thanks for the tip. I will investigate further.
It seems that the AZs Zone Resource are of type 'network' and 'router'. I was not aware of that distinction.
➜ openstack availability zone list --network --long +-----------+-------------+---------------+-----------+--------------+----------------+ | Zone Name | Zone Status | Zone Resource | Host Name | Service Name | Service Status | +-----------+-------------+---------------+-----------+--------------+----------------+ | room2 | available | network | | | | | room3 | available | router | | | | | room1 | available | network | | | | | room2 | available | router | | | | | room3 | available | network | | | | +-----------+-------------+---------------+-----------+--------------+----------------+
________________________________ From: Stephen Finucane stephenfin@redhat.com Sent: Tuesday, November 22, 2022 12:45 PM To: Lennart van Gijtenbeek | Routz; openstack-discuss Subject: Re: Strange situation regarding Availability Zones
CAUTION: This email originated from outside the organization.
On Tue, 2022-11-22 at 10:55 +0000, Lennart van Gijtenbeek | Routz wrote:
Hello everyone,
We have a strange situation regarding Availability Zones in our Openstack environment (version: Queens).
We have 3 AZs: room1, room2, and room3.
There are duplicate entries in the output of this command (openstack-pythonclient version 6.0.0):
➜ openstack availability zone list +-----------+-------------+ | Zone Name | Zone Status | +-----------+-------------+ | internal | available | | room2 | available | | room1 | available | | room3 | available | | nova | available | | room2 | available | | room3 | available | | room1 | available | | room2 | available | | room3 | available | +-----------+-------------+
Please run this command with debug mode enabled (--debug). That should give you far more information regarding what's going on here.
Stephen
However, in our database, I cannot find any reference to these duplicate entries.
What's going on here?
And how should we go about removing the duplicate entries if they don't show in the database?
MariaDB [nova_api]> select * from aggregate_metadata; +---------------------+------------+----+--------------+-------------------+-----------------+ | created_at | updated_at | id | aggregate_id | key | value | +---------------------+------------+----+--------------+-------------------+-----------------+ | 2019-08-07 06:11:47 | NULL | 1 | 1 | hypervisor | standard | | 2019-08-07 06:22:39 | NULL | 10 | 10 | availability_zone | room1 | | 2019-08-07 06:22:43 | NULL | 13 | 13 | availability_zone | room2 | | 2019-08-15 11:31:39 | NULL | 16 | 16 | availability_zone | room3 | | 2019-09-11 07:50:22 | NULL | 17 | 17 | hypervisor | test | | 2021-08-06 08:43:28 | NULL | 20 | 20 | hypervisor | ephemeral_local | +---------------------+------------+----+--------------+-------------------+-----------------+ 6 rows in set (0.00 sec)
MariaDB [nova_api]> select * from aggregates; +---------------------+------------+----+--------------------------------------+------------------+ | created_at | updated_at | id | uuid | name | +---------------------+------------+----+--------------------------------------+------------------+ | 2019-08-07 06:11:34 | NULL | 1 | 5f6baa56-252d-4f06-998f-5d166957490d | compute_standard | | 2019-08-07 06:22:39 | NULL | 10 | f7805dfb-00d2-458b-bb30-24919b8c9cd2 | room1 | | 2019-08-07 06:22:43 | NULL | 13 | b03f7028-f303-47a2-b6ca-128065f23030 | room2 | | 2019-08-15 11:31:39 | NULL | 16 | 1aff7a43-695a-4a10-9b24-116af73d18c1 | room3 | | 2019-09-11 07:50:02 | NULL | 17 | 952fa0aa-2ef6-443e-a906-76021acced0a | compute_test | | 2021-08-06 08:42:40 | NULL | 20 | bab386c4-f1a3-4482-b0ea-74c74fb172bd | ephemeral_local | +---------------------+------------+----+--------------------------------------+------------------+ 6 rows in set (0.00 sec)
Thank you.
Best regards, Lennart
On Tue, 2022-11-22 at 12:32 +0000, Lennart van Gijtenbeek | Routz wrote:
Thanks for the tip. I will investigate further.
It seems that the AZs Zone Resource are of type 'network' and 'router'. I was not aware of that distinction.
➜ openstack availability zone list --network --long +-----------+-------------+---------------+-----------+--------------+----------------+
Zone Name | Zone Status | Zone Resource | Host Name | Service Name | Service Status |
+-----------+-------------+---------------+-----------+--------------+----------------+
room2 | available | network | | | | room3 | available | router | | | | room1 | available | network | | | | room2 | available | router | | | | room3 | available | network | | | |
+-----------+-------------+---------------+-----------+--------------+----------------+
that sounds like two diffent types of AZ have been put under the same command
openstack availability zone list is ment to return just the nova viablitiey zones
other service like cinder can declare tehre services are algined to the nova AZ but there is no such thing as a zone resouce or zone avaiablety form a nova perspective.
so it sound like osc has incorreclty started including other concepts in teh ouput of the "openstack availability zone list" command
it was added a long time ago however https://github.com/openstack/python-openstackclient/commit/4d332defbc4231f77... so its proably too late to undo that now but the concepts fo an aviableity zone in cinder and neutron are just refence to the nvoa AZs you should not have indepentdly listabel network ro volume azs. its true that neutron has an AZ api now https://docs.openstack.org/api-ref/network/v2/?expanded=#list-all-availabili... i hope that is admin only because it shoul dnot be leaked to normal users.
cidner does not expsoe AZ as a toplevel api although i think nova can get the infor form the backend as aprt of the attachemt/connection info
my guess was there has been a regression somewhere and osc started including the network info by default however that seams to have always been the case https://github.com/openstack/python-openstackclient/blob/master/openstackcli...
you can workaround thsi by passing --compute to osc but it feels wrong to be mixing tow differnet concepts unter the same command.
From: Stephen Finucane stephenfin@redhat.com Sent: Tuesday, November 22, 2022 12:45 PM To: Lennart van Gijtenbeek | Routz; openstack-discuss Subject: Re: Strange situation regarding Availability Zones
CAUTION: This email originated from outside the organization.
On Tue, 2022-11-22 at 10:55 +0000, Lennart van Gijtenbeek | Routz wrote:
Hello everyone,
We have a strange situation regarding Availability Zones in our Openstack environment (version: Queens).
We have 3 AZs: room1, room2, and room3.
There are duplicate entries in the output of this command (openstack-pythonclient version 6.0.0):
➜ openstack availability zone list +-----------+-------------+
Zone Name | Zone Status |
+-----------+-------------+
internal | available | room2 | available | room1 | available | room3 | available | nova | available | room2 | available | room3 | available | room1 | available | room2 | available | room3 | available |
+-----------+-------------+
Please run this command with debug mode enabled (--debug). That should give you far more information regarding what's going on here.
Stephen
However, in our database, I cannot find any reference to these duplicate entries.
What's going on here?
And how should we go about removing the duplicate entries if they don't show in the database?
MariaDB [nova_api]> select * from aggregate_metadata; +---------------------+------------+----+--------------+-------------------+-----------------+
created_at | updated_at | id | aggregate_id | key | value |
+---------------------+------------+----+--------------+-------------------+-----------------+
2019-08-07 06:11:47 | NULL | 1 | 1 | hypervisor | standard | 2019-08-07 06:22:39 | NULL | 10 | 10 | availability_zone | room1 | 2019-08-07 06:22:43 | NULL | 13 | 13 | availability_zone | room2 | 2019-08-15 11:31:39 | NULL | 16 | 16 | availability_zone | room3 | 2019-09-11 07:50:22 | NULL | 17 | 17 | hypervisor | test | 2021-08-06 08:43:28 | NULL | 20 | 20 | hypervisor | ephemeral_local |
+---------------------+------------+----+--------------+-------------------+-----------------+ 6 rows in set (0.00 sec)
MariaDB [nova_api]> select * from aggregates; +---------------------+------------+----+--------------------------------------+------------------+
created_at | updated_at | id | uuid | name |
+---------------------+------------+----+--------------------------------------+------------------+
2019-08-07 06:11:34 | NULL | 1 | 5f6baa56-252d-4f06-998f-5d166957490d | compute_standard | 2019-08-07 06:22:39 | NULL | 10 | f7805dfb-00d2-458b-bb30-24919b8c9cd2 | room1 | 2019-08-07 06:22:43 | NULL | 13 | b03f7028-f303-47a2-b6ca-128065f23030 | room2 | 2019-08-15 11:31:39 | NULL | 16 | 1aff7a43-695a-4a10-9b24-116af73d18c1 | room3 | 2019-09-11 07:50:02 | NULL | 17 | 952fa0aa-2ef6-443e-a906-76021acced0a | compute_test | 2021-08-06 08:42:40 | NULL | 20 | bab386c4-f1a3-4482-b0ea-74c74fb172bd | ephemeral_local |
+---------------------+------------+----+--------------------------------------+------------------+ 6 rows in set (0.00 sec)
Thank you.
Best regards, Lennart
participants (3)
-
Lennart van Gijtenbeek | Routz
-
Sean Mooney
-
Stephen Finucane