Neutron bandwidth metering based on remote address
Dear list, We are trying to implement tenant bandwidth metering at the neutron router level. Since some of the network spaces connected to the external interface of the neutron router are supposed to be unmetered, we need to match on the remote address. Conveniently, there exists a --remote-ip-prefix option on meter label create; however, since [1], its meaning was changed to the exact opposite: Instead of matching on the *remote* prefix (towards the external interface), it matches on the *local* prefix (towards the OS tenant network). In an ideal world, we would want to revert that change and instead introduce a --local-ip-prefix option which covers that use-case. I suppose this is not a thing we *should* do though, given that this change made it into a few releases already. Instead, we’ll have to create a new option (which whatever name) + associated database schema + iptables rule patterns to implement the feature. The questions associated with this are now: - Does this make absolutely no sense to anyone? - What is the process for this? I suppose since this change was made intentionally and passed review, our desired change needs to go through a feature request process (blueprints maybe?). kind regards, Jonas Schäfer [1]: https://opendev.org/openstack/neutron/commit/ 92db1d4a2c49b1f675b6a9552a8cc5a417973b64 -- Jonas Schäfer DevOps Engineer Cloud&Heat Technologies GmbH Königsbrücker Straße 96 | 01099 Dresden +49 351 479 367 37 jonas.schaefer@cloudandheat.com | www.cloudandheat.com New Service: Managed Kubernetes designed for AI & ML https://managed-kubernetes.cloudandheat.com/ Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann Authorized signatory: Kristina Rübenkamp
Hallo Jonas, I have worked to address this specific use case. First, the part of the solution that is already implemented. If you only need to gather metrics in a tenant fashion, you can take a look into this PR: https://review.opendev.org/#/c/735605/. That pull request enables operators to configure shared traffic labels, and then, these traffic labels will be exposed/published with different granularities. The different granularities are router, tenant, label, router-label, and tenant-label. The complete explanation can be found in the "RST" document that the PR also introduces, where we wrote a complete description of neutron metering, its configs, and usage. You are welcome to review and help us get this PR merged :) So far, if all you need is to measure the whole traffic, but in different granularities, that PR will probably be enough. On the other hand, if you need to create more complex rules to filter by source/destination IPs, then we need something else. Interestingly enough, we are working towards that. We will extend neutron API, and neutron metering to allow operators to use "remote-ip" and "source-ip" to create metering labels rules. We also saw the PR that changed the behavior of the "remote-ip" property, and the whole confusion it caused (at least for us). However, instead of proposing to revert it, we are working towards enabling the API to handle "remote-ip" and "source-ip", which will cover the use case of the person that introduced that commit, and many others such as ours and yours (probably). On Tue, Jul 7, 2020 at 5:47 AM Jonas Schäfer < jonas.schaefer@cloudandheat.com> wrote:
Dear list,
We are trying to implement tenant bandwidth metering at the neutron router level. Since some of the network spaces connected to the external interface of the neutron router are supposed to be unmetered, we need to match on the remote address.
Conveniently, there exists a --remote-ip-prefix option on meter label create; however, since [1], its meaning was changed to the exact opposite: Instead of matching on the *remote* prefix (towards the external interface), it matches on the *local* prefix (towards the OS tenant network).
In an ideal world, we would want to revert that change and instead introduce a --local-ip-prefix option which covers that use-case. I suppose this is not a thing we *should* do though, given that this change made it into a few releases already.
Instead, we’ll have to create a new option (which whatever name) + associated database schema + iptables rule patterns to implement the feature.
The questions associated with this are now:
- Does this make absolutely no sense to anyone? - What is the process for this? I suppose since this change was made intentionally and passed review, our desired change needs to go through a feature request process (blueprints maybe?).
kind regards, Jonas Schäfer
[1]: https://opendev.org/openstack/neutron/commit/ 92db1d4a2c49b1f675b6a9552a8cc5a417973b64
-- Jonas Schäfer DevOps Engineer
Cloud&Heat Technologies GmbH Königsbrücker Straße 96 | 01099 Dresden +49 351 479 367 37 jonas.schaefer@cloudandheat.com | www.cloudandheat.com
New Service: Managed Kubernetes designed for AI & ML https://managed-kubernetes.cloudandheat.com/
Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann Authorized signatory: Kristina Rübenkamp
-- Rafael Weingärtner
Hello Rafael, On Dienstag, 7. Juli 2020 14:09:29 CEST Rafael Weingärtner wrote:
Hallo Jonas, I have worked to address this specific use case.
First, the part of the solution that is already implemented. If you only need to gather metrics in a tenant fashion, you can take a look into this PR: https://review.opendev.org/#/c/735605/. That pull request enables operators to configure shared traffic labels, and then, these traffic labels will be exposed/published with different granularities. The different granularities are router, tenant, label, router-label, and tenant-label. The complete explanation can be found in the "RST" document that the PR also introduces, where we wrote a complete description of neutron metering, its configs, and usage. You are welcome to review and help us get this PR merged :)
This already looks very useful to us, since it saves us from creating labels for each and every project.
So far, if all you need is to measure the whole traffic, but in different granularities, that PR will probably be enough.
Not quite; as mentioned, we’ll need to carve out specific network areas from metering, those which are in our DCs, but on the other side of the router from the customer perspective.
On the other hand, if you need to create more complex rules to filter by source/destination IPs, then we need something else. Interestingly enough, we are working towards that. We will extend neutron API, and neutron metering to allow operators to use "remote-ip" and "source-ip" to create metering labels rules.
That sounds exactly like what we’d need.
We also saw the PR that changed the behavior of the "remote-ip" property, and the whole confusion it caused (at least for us). However, instead of proposing to revert it, we are working towards enabling the API to handle "remote-ip" and "source-ip", which will cover the use case of the person that introduced that commit, and many others such as ours and yours (probably).
Sounds good. Is there a way we can collaborate on this? Is there a launchpad bug which tracks that? (Also, is there a launchpad thing for the shared label granularity you’re doing already? I didn’t find one mentioned on the gerrit page.) kind regards, Jonas Schäfer
On Tue, Jul 7, 2020 at 5:47 AM Jonas Schäfer <
jonas.schaefer@cloudandheat.com> wrote:
Dear list,
We are trying to implement tenant bandwidth metering at the neutron router level. Since some of the network spaces connected to the external interface of the neutron router are supposed to be unmetered, we need to match on the remote address.
Conveniently, there exists a --remote-ip-prefix option on meter label create; however, since [1], its meaning was changed to the exact opposite: Instead of matching on the *remote* prefix (towards the external interface), it matches on the *local* prefix (towards the OS tenant network).
In an ideal world, we would want to revert that change and instead introduce a --local-ip-prefix option which covers that use-case. I suppose this is not a thing we *should* do though, given that this change made it into a few releases already.
Instead, weâll have to create a new option (which whatever name) + associated database schema + iptables rule patterns to implement the feature.
The questions associated with this are now:
- Does this make absolutely no sense to anyone? - What is the process for this? I suppose since this change was made intentionally and passed review, our desired change needs to go through a feature request process (blueprints maybe?).
kind regards, Jonas Schäfer
[1]: https://opendev.org/openstack/neutron/commit/
92db1d4a2c49b1f675b6a9552a8cc5a417973b64
-- Jonas Schäfer DevOps Engineer
Cloud&Heat Technologies GmbH Königsbrücker StraÃe 96 | 01099 Dresden +49 351 479 367 37 jonas.schaefer@cloudandheat.com | www.cloudandheat.com
New Service: Managed Kubernetes designed for AI & ML https://managed-kubernetes.cloudandheat.com/
Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann Authorized signatory: Kristina Rübenkamp
-- Jonas Schäfer DevOps Engineer Cloud&Heat Technologies GmbH Königsbrücker Straße 96 | 01099 Dresden +49 351 479 367 37 jonas.schaefer@cloudandheat.com | www.cloudandheat.com New Service: Managed Kubernetes designed for AI & ML https://managed-kubernetes.cloudandheat.com/ Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann Authorized signatory: Kristina Rübenkamp
I created a bug track for the extension of the neutron metering granularities: https://bugs.launchpad.net/neutron/+bug/1886949 I am never sure about those "paper work", I normally propose the pull requests, and wait for the guidance of the community. About the source/destination filtering, I have not published anything yet. So far, we defined/specified what we need/want from the Neutron metering sub-system. Next week I am supposed to start on this matter. Therefore, as soon as I have updates, I will create the bug report, and pull requests. You can help me now by reviewing the PR I already have open, and of course, testing/using it :) On Thu, Jul 9, 2020 at 3:54 AM Jonas Schäfer < jonas.schaefer@cloudandheat.com> wrote:
Hello Rafael,
On Dienstag, 7. Juli 2020 14:09:29 CEST Rafael Weingärtner wrote:
Hallo Jonas, I have worked to address this specific use case.
First, the part of the solution that is already implemented. If you only need to gather metrics in a tenant fashion, you can take a look into this PR: https://review.opendev.org/#/c/735605/. That pull request enables operators to configure shared traffic labels, and then, these traffic labels will be exposed/published with different granularities. The different granularities are router, tenant, label, router-label, and tenant-label. The complete explanation can be found in the "RST" document that the PR also introduces, where we wrote a complete description of neutron metering, its configs, and usage. You are welcome to review and help us get this PR merged :)
This already looks very useful to us, since it saves us from creating labels for each and every project.
So far, if all you need is to measure the whole traffic, but in different granularities, that PR will probably be enough.
Not quite; as mentioned, we’ll need to carve out specific network areas from metering, those which are in our DCs, but on the other side of the router from the customer perspective.
On the other hand, if you need to create more complex rules to filter by source/destination IPs, then we need something else. Interestingly enough, we are working towards that. We will extend neutron API, and neutron metering to allow operators to use "remote-ip" and "source-ip" to create metering labels rules.
That sounds exactly like what we’d need.
We also saw the PR that changed the behavior of the "remote-ip" property, and the whole confusion it caused (at least for us). However, instead of proposing to revert it, we are working towards enabling the API to handle "remote-ip" and "source-ip", which will cover the use case of the person that introduced that commit, and many others such as ours and yours (probably).
Sounds good. Is there a way we can collaborate on this? Is there a launchpad bug which tracks that? (Also, is there a launchpad thing for the shared label granularity you’re doing already? I didn’t find one mentioned on the gerrit page.)
kind regards, Jonas Schäfer
On Tue, Jul 7, 2020 at 5:47 AM Jonas Schäfer <
jonas.schaefer@cloudandheat.com> wrote:
Dear list,
We are trying to implement tenant bandwidth metering at the neutron
level. Since some of the network spaces connected to the external interface of the neutron router are supposed to be unmetered, we need to match on
remote address.
Conveniently, there exists a --remote-ip-prefix option on meter label create; however, since [1], its meaning was changed to the exact opposite: Instead of matching on the *remote* prefix (towards the external interface), it matches on the *local* prefix (towards the OS tenant network).
In an ideal world, we would want to revert that change and instead introduce a --local-ip-prefix option which covers that use-case. I suppose this is not a thing we *should* do though, given that this change made it into a few releases already.
Instead, we’ll have to create a new option (which whatever name) + associated database schema + iptables rule patterns to implement the feature.
The questions associated with this are now:
- Does this make absolutely no sense to anyone? - What is the process for this? I suppose since this change was made intentionally and passed review, our desired change needs to go
router the through a
feature request process (blueprints maybe?).
kind regards, Jonas Schäfer
[1]: https://opendev.org/openstack/neutron/commit/
92db1d4a2c49b1f675b6a9552a8cc5a417973b64
-- Jonas Schäfer DevOps Engineer
Cloud&Heat Technologies GmbH Königsbrücker Straße 96 | 01099 Dresden +49 351 479 367 37 jonas.schaefer@cloudandheat.com | www.cloudandheat.com
New Service: Managed Kubernetes designed for AI & ML https://managed-kubernetes.cloudandheat.com/
Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann Authorized signatory: Kristina Rübenkamp
-- Jonas Schäfer DevOps Engineer
Cloud&Heat Technologies GmbH Königsbrücker Straße 96 | 01099 Dresden +49 351 479 367 37 jonas.schaefer@cloudandheat.com | www.cloudandheat.com
New Service: Managed Kubernetes designed for AI & ML https://managed-kubernetes.cloudandheat.com/
Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann Authorized signatory: Kristina Rübenkamp
-- Rafael Weingärtner
Hello Jonas, I created the proposal for the extension concerning local IP addresses in metering label rules. You can find the proposal here: https://bugs.launchpad.net/neutron/+bug/1889431. I am starting today the implementation. On Thu, Jul 9, 2020 at 8:53 AM Rafael Weingärtner < rafaelweingartner@gmail.com> wrote:
I created a bug track for the extension of the neutron metering granularities: https://bugs.launchpad.net/neutron/+bug/1886949 I am never sure about those "paper work", I normally propose the pull requests, and wait for the guidance of the community.
About the source/destination filtering, I have not published anything yet. So far, we defined/specified what we need/want from the Neutron metering sub-system. Next week I am supposed to start on this matter. Therefore, as soon as I have updates, I will create the bug report, and pull requests. You can help me now by reviewing the PR I already have open, and of course, testing/using it :)
On Thu, Jul 9, 2020 at 3:54 AM Jonas Schäfer < jonas.schaefer@cloudandheat.com> wrote:
Hello Rafael,
Hallo Jonas, I have worked to address this specific use case.
First, the part of the solution that is already implemented. If you only need to gather metrics in a tenant fashion, you can take a look into
On Dienstag, 7. Juli 2020 14:09:29 CEST Rafael Weingärtner wrote: this
PR: https://review.opendev.org/#/c/735605/. That pull request enables operators to configure shared traffic labels, and then, these traffic labels will be exposed/published with different granularities. The different granularities are router, tenant, label, router-label, and tenant-label. The complete explanation can be found in the "RST" document that the PR also introduces, where we wrote a complete description of neutron metering, its configs, and usage. You are welcome to review and help us get this PR merged :)
This already looks very useful to us, since it saves us from creating labels for each and every project.
So far, if all you need is to measure the whole traffic, but in different granularities, that PR will probably be enough.
Not quite; as mentioned, we’ll need to carve out specific network areas from metering, those which are in our DCs, but on the other side of the router from the customer perspective.
On the other hand, if you need to create more complex rules to filter by source/destination IPs, then we need something else. Interestingly enough, we are working towards that. We will extend neutron API, and neutron metering to allow operators to use "remote-ip" and "source-ip" to create metering labels rules.
That sounds exactly like what we’d need.
We also saw the PR that changed the behavior of the "remote-ip" property, and the whole confusion it caused (at least for us). However, instead of proposing to revert it, we are working towards enabling the API to handle "remote-ip" and "source-ip", which will cover the use case of the person that introduced that commit, and many others such as ours and yours (probably).
Sounds good. Is there a way we can collaborate on this? Is there a launchpad bug which tracks that? (Also, is there a launchpad thing for the shared label granularity you’re doing already? I didn’t find one mentioned on the gerrit page.)
kind regards, Jonas Schäfer
On Tue, Jul 7, 2020 at 5:47 AM Jonas Schäfer <
jonas.schaefer@cloudandheat.com> wrote:
Dear list,
We are trying to implement tenant bandwidth metering at the neutron
level. Since some of the network spaces connected to the external interface of the neutron router are supposed to be unmetered, we need to match on
remote address.
Conveniently, there exists a --remote-ip-prefix option on meter label create; however, since [1], its meaning was changed to the exact opposite: Instead of matching on the *remote* prefix (towards the external interface), it matches on the *local* prefix (towards the OS tenant network).
In an ideal world, we would want to revert that change and instead introduce a --local-ip-prefix option which covers that use-case. I suppose this is not a thing we *should* do though, given that this change made it into a few releases already.
Instead, we’ll have to create a new option (which whatever name) + associated database schema + iptables rule patterns to implement the feature.
The questions associated with this are now:
- Does this make absolutely no sense to anyone? - What is the process for this? I suppose since this change was made intentionally and passed review, our desired change needs to go
router the through a
feature request process (blueprints maybe?).
kind regards, Jonas Schäfer
[1]: https://opendev.org/openstack/neutron/commit/
92db1d4a2c49b1f675b6a9552a8cc5a417973b64
-- Jonas Schäfer DevOps Engineer
Cloud&Heat Technologies GmbH Königsbrücker Straße 96 | 01099 Dresden +49 351 479 367 37 jonas.schaefer@cloudandheat.com | www.cloudandheat.com
New Service: Managed Kubernetes designed for AI & ML https://managed-kubernetes.cloudandheat.com/
Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann Authorized signatory: Kristina Rübenkamp
-- Jonas Schäfer DevOps Engineer
Cloud&Heat Technologies GmbH Königsbrücker Straße 96 | 01099 Dresden +49 351 479 367 37 jonas.schaefer@cloudandheat.com | www.cloudandheat.com
New Service: Managed Kubernetes designed for AI & ML https://managed-kubernetes.cloudandheat.com/
Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann Authorized signatory: Kristina Rübenkamp
-- Rafael Weingärtner
-- Rafael Weingärtner
participants (2)
-
Jonas Schäfer
-
Rafael Weingärtner