[nova][ptg] Allow compute nodes to use DISK_GB from shared storage RP by using aggregate relationship
spec: https://review.opendev.org/#/c/650188/
Agreements from the room[1]: * new config option 'using_shared_disk_provider' (just an example name) on the compute level to ignore the DISK_GB reporting from the driver * new config option 'sharing_disk_aggregate' (just an example name) on the compute level to tell nova compute what is the UUID of the placement aggregate that contains the sharing DISK_GB providers in placement. * the "using_shared_disk_provider" flag necessarly has to be explicit since if not, it would be a chicken-and-egg problem on a greenfields install as the shared RP wouldn't be created * deployer needs to create the sharing disk RP and report inventory / traits on it * deployer needs to define the placement aggregate and add the sharing disk RP into it * when compute restarts and sees that 'using_shared_disk_provider' = True in the config, it adds the its compute RP to the aggregate defined in 'sharing_disk_aggregate' Then if it sees that the root RP still has DISK_GB inventory then trigger a reshape * os-hypervisor API response (in a new microversion) will have a link to the sharing disk RP if the compute is so configured.
TODO: * tpatil to update the spec
Hi All,
TODO:
- tpatil to update the spec
I will update the specs next week and upload it for review.
Regards, tpatil
________________________________________ From: Balázs Gibizer balazs.gibizer@est.tech Sent: Friday, November 8, 2019 5:01 PM To: openstack-discuss Subject: [nova][ptg] Allow compute nodes to use DISK_GB from shared storage RP by using aggregate relationship
spec: https://review.opendev.org/#/c/650188/
Agreements from the room[1]: * new config option 'using_shared_disk_provider' (just an example name) on the compute level to ignore the DISK_GB reporting from the driver * new config option 'sharing_disk_aggregate' (just an example name) on the compute level to tell nova compute what is the UUID of the placement aggregate that contains the sharing DISK_GB providers in placement. * the "using_shared_disk_provider" flag necessarly has to be explicit since if not, it would be a chicken-and-egg problem on a greenfields install as the shared RP wouldn't be created * deployer needs to create the sharing disk RP and report inventory / traits on it * deployer needs to define the placement aggregate and add the sharing disk RP into it * when compute restarts and sees that 'using_shared_disk_provider' = True in the config, it adds the its compute RP to the aggregate defined in 'sharing_disk_aggregate' Then if it sees that the root RP still has DISK_GB inventory then trigger a reshape * os-hypervisor API response (in a new microversion) will have a link to the sharing disk RP if the compute is so configured.
TODO: * tpatil to update the spec
[1] https://etherpad.openstack.org/p/nova-shanghai-ptg
Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding.
On 11/8/2019 2:01 AM, Balázs Gibizer wrote:
- deployer needs to create the sharing disk RP and report inventory /
traits on it
- deployer needs to define the placement aggregate and add the sharing
disk RP into it
- when compute restarts and sees that 'using_shared_disk_provider' =
True in the config, it adds the its compute RP to the aggregate defined in 'sharing_disk_aggregate' Then if it sees that the root RP still has DISK_GB inventory then trigger a reshape
Does the compute host also get added to a nova host aggregate which mirrors the resource provider aggregate in placmeent or do we only need the placement resource provider sharing DISK_GB aggregate?
On Fri, Nov 8, 2019 at 08:03, Matt Riedemann mriedemos@gmail.com wrote:
On 11/8/2019 2:01 AM, Balázs Gibizer wrote:
- deployer needs to create the sharing disk RP and report inventory /
traits on it
- deployer needs to define the placement aggregate and add the
sharing disk RP into it
- when compute restarts and sees that 'using_shared_disk_provider' =
True in the config, it adds the its compute RP to the aggregate defined in 'sharing_disk_aggregate' Then if it sees that the root RP still has DISK_GB inventory then trigger a reshape
Does the compute host also get added to a nova host aggregate which mirrors the resource provider aggregate in placmeent or do we only need the placement resource provider sharing DISK_GB aggregate?
As far as I see we only need the placement aggregate to make this work.
--
Thanks,
Matt
On 11/8/2019 2:01 AM, Balázs Gibizer wrote:
- when compute restarts and sees that 'using_shared_disk_provider' =
True in the config, it adds the its compute RP to the aggregate defined in 'sharing_disk_aggregate' Then if it sees that the root RP still has DISK_GB inventory then trigger a reshape
Conversely, if the deployer decides to use local disk for the host again, what are the steps?
1. Change using_shared_disk_provider=False 2. Restart/SIGHUP compute service 3. Compute removes itself from the aggregate 4. Compute reshapes to add DISK_GB inventory on the root compute node resource provider and moves DISK_GB allocations from the sharing provider back to the root compute node provider.
Correct?
On Fri, Nov 8, 2019 at 08:05, Matt Riedemann mriedemos@gmail.com wrote:
On 11/8/2019 2:01 AM, Balázs Gibizer wrote:
- when compute restarts and sees that 'using_shared_disk_provider' =
True in the config, it adds the its compute RP to the aggregate defined in 'sharing_disk_aggregate' Then if it sees that the root RP still has DISK_GB inventory then trigger a reshape
Conversely, if the deployer decides to use local disk for the host again, what are the steps?
- Change using_shared_disk_provider=False
- Restart/SIGHUP compute service
- Compute removes itself from the aggregate
- Compute reshapes to add DISK_GB inventory on the root compute node
resource provider and moves DISK_GB allocations from the sharing provider back to the root compute node provider.
Correct?
Seems correct to me.
gibi
--
Thanks,
Matt
On Wed, Nov 13, 2019 at 3:32 PM Balázs Gibizer balazs.gibizer@est.tech wrote:
On Fri, Nov 8, 2019 at 08:05, Matt Riedemann mriedemos@gmail.com wrote:
On 11/8/2019 2:01 AM, Balázs Gibizer wrote:
- when compute restarts and sees that 'using_shared_disk_provider' =
True in the config, it adds the its compute RP to the aggregate defined in 'sharing_disk_aggregate' Then if it sees that the root RP still has DISK_GB inventory then trigger a reshape
Conversely, if the deployer decides to use local disk for the host again, what are the steps?
- Change using_shared_disk_provider=False
- Restart/SIGHUP compute service
- Compute removes itself from the aggregate
- Compute reshapes to add DISK_GB inventory on the root compute node
resource provider and moves DISK_GB allocations from the sharing provider back to the root compute node provider.
Correct?
Seems correct to me.
gibi
Me too. To be clear, I don't think operators would modify the above but if so, they would need reshapes.
--
Thanks,
Matt
On 11/13/2019 8:34 AM, Sylvain Bauza wrote:
Me too. To be clear, I don't think operators would modify the above but if so, they would need reshapes.
Maybe not, but this is the kind of detail that should be in the spec and functional tests to make sure it's solid since this is a big architectural change in nova.
On 11/13/2019 8:34 AM, Sylvain Bauza wrote:
Me too. To be clear, I don't think operators would modify the above but if so, they would need reshapes.
Maybe not, but this is the kind of detail that should be in the spec and functional tests to make sure it's solid since this is a big architectural change in nova.
It depends on how the aggregates are created on the nova and placement side.
A) From placement point of view, operator can create a new aggregate and add shared storage RP to it (tag MISC_SHARES_VIA_AGGREGATE trait to this RP). The newly created valid UUID would then be set in the config option ``sharing_disk_aggregate`` on the compute node side. This aggregate UUID wouldn't be present in the nova aggregate. so it's not possible to add host to the nova aggregate unless a new aggregate is created on nova side.
B) If nova aggregates are synced to the placement service and say below is the picture:
Nova:
Agg1 - metadata (pinned=True) - host1 - host2
Now, operator adds a new shared storage RP to Agg1 on placement side and then set Agg1 UUID in ``sharing_disk_aggregate`` on compute nodes along with ``using_shared_disk_provider`=True``, then it would add compute node RP to the Agg1 on the placement without any issues but when you want to reverse the configuration, using_shared_disk_provider=False, then it not that straight to remove the host from the placement/nova aggregate because there would be other traits set to compute RPs which could cause those functions stop working.
We had same kind of discussion [1] when implementing forbidden aggregates where we want to sync traits set to the aggregates but later it was concluded that operator will do it manually.
I will include the details Matt has pointed out in this email in my next patchset.
[1] : http://lists.openstack.org/pipermail/openstack-discuss/2019-June/006950.html
Regards, tpatil
________________________________________ From: Matt Riedemann mriedemos@gmail.com Sent: Wednesday, November 13, 2019 11:41 PM To: openstack-discuss@lists.openstack.org Subject: Re: [nova][ptg] Allow compute nodes to use DISK_GB from shared storage RP by using aggregate relationship
On 11/13/2019 8:34 AM, Sylvain Bauza wrote:
Me too. To be clear, I don't think operators would modify the above but if so, they would need reshapes.
Maybe not, but this is the kind of detail that should be in the spec and functional tests to make sure it's solid since this is a big architectural change in nova.
--
Thanks,
Matt
Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding.
participants (4)
-
Balázs Gibizer
-
Matt Riedemann
-
Patil, Tushar
-
Sylvain Bauza