[designate] Architecture help for designate on multi-region/multi-AZ deployments
Hello folks,
I currently have designate installed in a single region in a cloud. Now we are thinking about the next steps: A multi-region designate for a simple use case: Bring your own domain, have A/AAAA records for your instances regardless of the region. Side note: That requirement can be modified if what I am asking is impossible.
I didn't find anything in the ML, nor in the bug tracker [1] outside a neutron bug [2] (am I blind?). I had a look at the designate documentation, but there are no best practices for multi-region (or am I blind again?) The only relevant part for integration with neutron is in [3], which is not region related.
I am looking for advice from designate experts here, to see what we can do.
Right now, we are thinking about two ways to deal with this: A) Install a separate openstack with designate, and make all the regions's neutrons configuration point to a single designate. Neutron would work, designate would be simple to manage (as we have a single designate). The downside here is the clear-cut separation of regions: I am afraid of how records are stored in the central designate, and if there won't be multi-tenancy issues (uuid could technically overlap, as different regions means different keystones here). B) Install a designate in each region, with DNS servers in each region. We have strict region separation this way. The downside is the user friendliness: A customer _can_ decide which region in which the domain will be hosted. That means that we now need to worry about synchronizing region's state (if you have created a domain in that region, you can't do it elsewhere).
Am I misunderstanding/not seeing something obvious here?
Thank you in advance.
Jean-Philippe Evrard (evrardjp)
[0]: https://www.google.com/search?q=designate+multi+region+site%3Ahttp%3A%2F%2Fl... [1]: https://bugs.launchpad.net/designate?field.searchtext=region [2]: https://review.opendev.org/c/openstack/neutron/+/684854/ [3]: https://docs.openstack.org/neutron/latest/admin/config-dns-int-ext-serv.html
Hey JP,
I was hanging out waiting to see if others had feedback for you. I have not done a multi-region deployment, so I can't talk much to the Keystone/Identity issues with a multi-region setup, but I can give you some thoughts on DNS/Designate.
From a DNS perspective, the zones are hierarchical in nature. This
means if you need resources in each region to share a zone, example.com. for example, the zones will need to be managed via a centralized control plane. However, if you can clearly separate the zones by region, deploying Designate in each region would provide complete separation.
High availability is not really an issue as it's easy to setup Designate pools to update multiple DNS server instances and you always have the option of multiple secondary DNS servers distributed to the regions. It's really a question of control plane (API) availability should a region fail.
Personally, if it was my deployment, I would probably go down the path of a Designate deployment for each region. This would mean that any one DNS zone could not (automatically) be managed from multiple regions, but aligns to my understanding of the spirit of a "region" in OpenStack.
For non-bring-your-own-domain scenarios, this can be easily managed in the DNS hierarchy by creating regional parent zones. I.e. west.example.com., east.example.com., moon.example.com., etc. Since it's also likely that routable IP address space will be unique per region, the reverse zones would naturally be unique.
In the case of bring-your-own-domain, you would need a policy that any one domain can only exist in one region. However, there would be nothing stopping users from doing a similar regional breakdown of sub-domains if they would like.
As for the synchronization question, I think that is more of an issue that a customer would need to address if they feel they want a multiple primary scenario. Technically there would be no reason they could not create the zone in multiple regions, keep it in sync themselves, and add the DNS servers for both regions as authoritative for the zone.
If you want to provide a "premium" service where DNS HA is provided across regions, this can be done using the Designate pools configuration or via the normal secondary DNS server model configured in your DNS servers. This would maintain DNS resolution should a region go down, but if the zone was created in a specific region that is unavailable, you would be unable to update it until the region is restored.
Just some thoughts, please feel free to reach out if you want to chat about it more or bounce ideas around,
Michael
On Wed, Mar 9, 2022 at 12:34 AM Jean-Philippe Evrard openstack@a.spamming.party wrote:
Hello folks,
I currently have designate installed in a single region in a cloud. Now we are thinking about the next steps: A multi-region designate for a simple use case: Bring your own domain, have A/AAAA records for your instances regardless of the region. Side note: That requirement can be modified if what I am asking is impossible.
I didn't find anything in the ML, nor in the bug tracker [1] outside a neutron bug [2] (am I blind?). I had a look at the designate documentation, but there are no best practices for multi-region (or am I blind again?) The only relevant part for integration with neutron is in [3], which is not region related.
I am looking for advice from designate experts here, to see what we can do.
Right now, we are thinking about two ways to deal with this: A) Install a separate openstack with designate, and make all the regions's neutrons configuration point to a single designate. Neutron would work, designate would be simple to manage (as we have a single designate). The downside here is the clear-cut separation of regions: I am afraid of how records are stored in the central designate, and if there won't be multi-tenancy issues (uuid could technically overlap, as different regions means different keystones here). B) Install a designate in each region, with DNS servers in each region. We have strict region separation this way. The downside is the user friendliness: A customer _can_ decide which region in which the domain will be hosted. That means that we now need to worry about synchronizing region's state (if you have created a domain in that region, you can't do it elsewhere).
Am I misunderstanding/not seeing something obvious here?
Thank you in advance.
Jean-Philippe Evrard (evrardjp)
Hello,
On Thu, Mar 10, 2022, at 22:25, Michael Johnson wrote:
Personally, if it was my deployment, I would probably go down the path of a Designate deployment for each region. This would mean that any one DNS zone could not (automatically) be managed from multiple regions, but aligns to my understanding of the spirit of a "region" in OpenStack.
Yes, that's what I was leaning towards. It makes it very clear. In a way, anything that's "region specific" belongs to "region designate".
For non-bring-your-own-domain scenarios, this can be easily managed in the DNS hierarchy by creating regional parent zones. I.e. west.example.com., east.example.com., moon.example.com., etc. Since it's also likely that routable IP address space will be unique per region, the reverse zones would naturally be unique.
Yes, that's what we thought of doing too. It's pretty easy for our domains and for many operational aspects.
In the case of bring-your-own-domain, you would need a policy that any one domain can only exist in one region. However, there would be nothing stopping users from doing a similar regional breakdown of sub-domains if they would like.
That's where we were stopping. Is there a way to _technically enforce_ such kind of policy in an automated fashion? That's the most important part for me, and (sadly) the part that's very fuzzy.
As for the synchronization question, I think that is more of an issue that a customer would need to address if they feel they want a multiple primary scenario. Technically there would be no reason they could not create the zone in multiple regions, keep it in sync themselves, and add the DNS servers for both regions as authoritative for the zone.
I had the same argument, glad to see we are aligned :) I still think this is more of a "hack".
If you want to provide a "premium" service where DNS HA is provided across regions, this can be done using the Designate pools configuration or via the normal secondary DNS server model configured in your DNS servers. This would maintain DNS resolution should a region go down, but if the zone was created in a specific region that is unavailable, you would be unable to update it until the region is restored.
We were only thinking of secondary + anycast, and only in the case of a "central" model. I will have a look at pools. Thanks for the pointer.
Just some thoughts, please feel free to reach out if you want to chat about it more or bounce ideas around,
It's all what I was looking for, so really _thank you_ for doing that :) I am more and more thinking about a "central model" + "region model" mixed together for covering all use cases, now.
Regards, JP (evrardjp)
participants (2)
-
Jean-Philippe Evrard
-
Michael Johnson