<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><font face="DejaVu Sans">Hello Brian,<br>
<br>
Thanks for the info.<br>
<br>
I looked into the code for the metering agent and discovered the
following:</font></p>
<p><font face="DejaVu Sans">1. The metering agent on compute nodes
doesn't get notified about DVR routers running on the node.<br>
</font></p>
<p><font face="DejaVu Sans">2. For DVR routers it tries to meter the
rfp- interface</font><font face="DejaVu Sans"><font face="DejaVu
Sans"> (for floating ips)</font> on the snat- namespace. This
is wrong. The rfp- interface lies on the qrouter- namespace on
compute nodes. Also, there is a qg- interface on the snat-
namespace on network nodes (for performing NAT) which should be
metered too.<br>
</font></p>
<p><font face="DejaVu Sans">3. There is a race condition whereby the
metering agent is notified about a router before its namespaces
are created. The agent ends up not adding the metering rules for
those namespaces and this leads to unrecorded traffic.<br>
<br>
I addressed those issues in a change:
<a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/621165/">https://review.openstack.org/#/c/621165/</a>. Any feedback is
appreciated.<br>
<br>
Best regards,<br>
Alex<br>
<br>
</font></p>
<div class="moz-cite-prefix">On 26/10/2018 21:49, Brian Haley wrote:<br>
</div>
<blockquote type="cite"
cite="mid:e8be927b-0cd0-5dfc-5936-589fd1bb135e@gmail.com">On
10/25/2018 08:06 AM, Alexandru Sorodoc wrote:
<br>
<blockquote type="cite">Hello,
<br>
<br>
I'm trying to set up metering for neutron in Pike. I tested it
with a
<br>
centralized router and it works, but when I try with a
distributed router it
<br>
doesn't record any usage samples. I have one compute node and
one network node
<br>
and I've created an instance with a floating ip.
<br>
</blockquote>
<br>
The metering agent isn't very well maintained, and I don't see any
open bugs similar to this issue. The only thing I can remember is
this abandoned change regarding traffic counters for DVR routers -
<a class="moz-txt-link-freetext" href="https://review.openstack.org/#/c/486493/">https://review.openstack.org/#/c/486493/</a> but there was no
follow-on from the author.
<br>
<br>
The best thing to do would be to try and reproduce it on the
master branch (or Rocky) and file a bug.
<br>
<br>
> I think this is because the router is running on network1.
Why is it
<br>
> running on
<br>
> network1 and why does it seem that the l3 agent on compute1
does the actual
<br>
> routing?
<br>
<br>
The compute node will do all the routing when a floating IP is
associated, the router on network1 is for default snat traffic
when there is no floating IP and the instance tries to communicate
out the external network.
<br>
<br>
-Brian
<br>
<br>
<blockquote type="cite">
<br>
openstack router show public-router2
<br>
+-------------------------+----------------------------------------------------+
<br>
| Field |
Value |
<br>
+-------------------------+----------------------------------------------------+
<br>
| admin_state_up |
UP |
<br>
| availability_zone_hints
| |
<br>
| availability_zones |
nova |
<br>
| created_at |
2018-10-05T12:07:32Z |
<br>
| description
| |
<br>
| distributed |
True |
<br>
| external_gateway_info | {"network_id":
"b96473ce- |
<br>
| | 94f6-464f-a703-5285fb8ff3d3",
"enable_snat": true, |
<br>
| | "external_fixed_ips":
[{"subnet_id": |
<br>
| |
"6c08c3d9-7df1-4bec-b847-19f80b9d1764", |
<br>
| | "ip_address":
"192.168.252.102"}]} |
<br>
| flavor_id |
None |
<br>
| ha |
False |
<br>
| id |
37c1794b-58d1-4d0d-b34b-944ca411b86b |
<br>
| name |
public-router2 |
<br>
| project_id |
fe203109e67f4e39b066c9529f9fc35d |
<br>
| revision_number |
5 |
<br>
| routes | |
<br>
| status |
ACTIVE |
<br>
| tags | |
<br>
| updated_at |
2018-10-05T12:09:36Z |
<br>
+-------------------------+----------------------------------------------------+
<br>
<br>
openstack network agent list
<br>
+-----------+------------+-----------+-------------------+-------+-------+--------------+
<br>
| ID | Agent Type | Host | Availability Zone | Alive
| State | Binary |
<br>
+-----------+------------+-----------+-------------------+-------+-------+--------------+
<br>
| 14b9ea75- | L3 agent | compute1. | nova | :-) | UP |
neutron-l3-a |
<br>
| 1dc1-4e37 | | localdoma | | | |
gent |
<br>
| -a2b0-508 | | in | | |
| |
<br>
| 3d336916d | | | | |
| |
<br>
| 26139ec1- | Metering | compute1. | None | :-) | UP |
neutron- |
<br>
| f4f9-4bb3 | agent | localdoma | | | |
metering- |
<br>
| -aebb-c35 | | in | | | |
agent |
<br>
| 3a36ed79c | | | | |
| |
<br>
| 2a54971f- | DHCP agent | network1. | nova | :-) | UP |
neutron- |
<br>
| 9849-4ed2 | | localdoma | | | |
dhcp-agent |
<br>
| -b009-00e | | in | | |
| |
<br>
| 45eb4d255 | | | | |
| |
<br>
| 443c0b49- | Open | compute1. | None | :-) | UP |
neutron- |
<br>
| 4484-44d2 | vSwitch | localdoma | | | |
openvswitch- |
<br>
| -a704-32a | agent | in | | | |
agent |
<br>
| 92ffe6982 | | | | |
| |
<br>
| 5d00a219 | L3 agent | network1. | nova | :-) | UP |
neutron-vpn- |
<br>
| -abce- | | localdoma | | | |
agent |
<br>
| 48ca- | | in | | |
| |
<br>
| ba1e-d962 | | | | |
| |
<br>
| 01bd7de3 | | | | |
| |
<br>
| bc3458b4 | Open | network1. | None | :-) | UP |
neutron- |
<br>
| -250e- | vSwitch | localdoma | | | |
openvswitch- |
<br>
| 4adf-90e0 | agent | in | | | |
agent |
<br>
| -110a1a7f | | | | |
| |
<br>
| 6ccb | | | | |
| |
<br>
| c29f9da8- | Metering | network1. | None | :-) | UP |
neutron- |
<br>
| ca58-4a11 | agent | localdoma | | | |
metering- |
<br>
| -b500-a25 | | in | | | |
agent |
<br>
| 3f820808e | | | | |
| |
<br>
| cdce667d- | Metadata | network1. | None | :-) | UP |
neutron- |
<br>
| faa4 | agent | localdoma | | | |
metadata- |
<br>
| -49ed- | | in | | | |
agent |
<br>
| 83ee-e0e5 | | | | |
| |
<br>
| a352d482 | | | | |
| |
<br>
| cf5ae104- | Metadata | compute1. | None | :-) | UP |
neutron- |
<br>
| 49d7-4c85 | agent | localdoma | | | |
metadata- |
<br>
| -a252-cc5 | | in | | | |
agent |
<br>
| 9a9a12789 | | | | |
| |
<br>
+-----------+------------+-----------+-------------------+-------+-------+--------------+
<br>
<br>
If I check the node on which my distributed router is running it
tells me that
<br>
it's running on the network node:
<br>
<br>
neutron l3-agent-list-hosting-router
37c1794b-58d1-4d0d-b34b-944ca411b86b
<br>
+--------------------------------------+----------------------+----------------+-------+----------+
<br>
| id | host |
admin_state_up | alive | ha_state |
<br>
+--------------------------------------+----------------------+----------------+-------+----------+
<br>
| 5d00a219-abce-48ca-ba1e-d96201bd7de3 | network1.localdomain |
True | :-) | |
<br>
+--------------------------------------+----------------------+----------------+-------+----------+
<br>
<br>
If I check the iptable rules for the router on the compute and
network nodes by running:
<br>
<br>
ip netns exec qrouter-37c1794b-58d1-4d0d-b34b-944ca411b86b
iptables -nv -L
<br>
<br>
I see that compute1 records the traffic while network1 doesn't.
Also, I did some
<br>
debugging and found out that the metering agent on compute1
receives an empty
<br>
list of routers when querying the routers that it should
monitor.
<br>
<br>
Source:
<br>
<br>
<a class="moz-txt-link-freetext" href="https://github.com/openstack/neutron/blob/stable/pike/neutron/services/metering/agents/metering_agent.py#L177-L189">https://github.com/openstack/neutron/blob/stable/pike/neutron/services/metering/agents/metering_agent.py#L177-L189</a>
<br>
<br>
<a class="moz-txt-link-freetext" href="https://github.com/openstack/neutron/blob/stable/pike/neutron/db/metering/metering_rpc.py#L33-L57">https://github.com/openstack/neutron/blob/stable/pike/neutron/db/metering/metering_rpc.py#L33-L57</a>
<br>
<br>
I think this is because the router is running on network1. Why
is it running on
<br>
network1 and why does it seem that the l3 agent on compute1 does
the actual
<br>
routing?
<br>
<br>
Thanks,
<br>
Alex
<br>
<br>
<br>
<br>
_______________________________________________
<br>
Mailing list:
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
<br>
Post to : <a class="moz-txt-link-abbreviated" href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a>
<br>
Unsubscribe :
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
<br>
<br>
</blockquote>
<br>
</blockquote>
</body>
</html>