[Openstack] [Pike][Neutron] L3 metering with DVR doesn't work

Alexandru Sorodoc alex at privacysystems.eu
Fri Nov 30 14:24:18 UTC 2018


Hello Brian,

Thanks for the info.

I looked into the code for the metering agent and discovered the following:

1. The metering agent on compute nodes doesn't get notified about DVR 
routers running on the node.

2. For DVR routers it tries to meter the rfp- interface(for floating 
ips) on the snat- namespace. This is wrong. The rfp- interface lies on 
the qrouter- namespace on compute nodes. Also, there is a qg- interface 
on the snat- namespace on network nodes (for performing NAT) which 
should be metered too.

3. There is a race condition whereby the metering agent is notified 
about a router before its namespaces are created. The agent ends up not 
adding the metering rules for those namespaces and this leads to 
unrecorded traffic.

I addressed those issues in a change: 
https://review.openstack.org/#/c/621165/. Any feedback is appreciated.

Best regards,
Alex

On 26/10/2018 21:49, Brian Haley wrote:
> On 10/25/2018 08:06 AM, Alexandru Sorodoc wrote:
>> Hello,
>>
>> I'm trying to set up metering for neutron in Pike. I tested it with a
>> centralized router and it works, but when I try with a distributed 
>> router it
>> doesn't record any usage samples. I have one compute node and one 
>> network node
>> and I've created an instance with a floating ip.
>
> The metering agent isn't very well maintained, and I don't see any 
> open bugs similar to this issue.  The only thing I can remember is 
> this abandoned change regarding traffic counters for DVR routers - 
> https://review.openstack.org/#/c/486493/ but there was no follow-on 
> from the author.
>
> The best thing to do would be to try and reproduce it on the master 
> branch (or Rocky) and file a bug.
>
> > I think this is because the router is running on network1. Why is it
> > running on
> > network1 and why does it seem that the l3 agent on compute1 does the 
> actual
> > routing?
>
> The compute node will do all the routing when a floating IP is 
> associated, the router on network1 is for default snat traffic when 
> there is no floating IP and the instance tries to communicate out the 
> external network.
>
> -Brian
>
>>
>> openstack router show public-router2
>> +-------------------------+----------------------------------------------------+ 
>>
>> | Field                   | 
>> Value                                              |
>> +-------------------------+----------------------------------------------------+ 
>>
>> | admin_state_up          | 
>> UP                                                 |
>> | availability_zone_hints 
>> |                                                    |
>> | availability_zones      | 
>> nova                                               |
>> | created_at              | 
>> 2018-10-05T12:07:32Z                               |
>> | description |                                                    |
>> | distributed             | 
>> True                                               |
>> | external_gateway_info   | {"network_id": 
>> "b96473ce-                          |
>> |                         | 94f6-464f-a703-5285fb8ff3d3", 
>> "enable_snat": true, |
>> |                         | "external_fixed_ips": 
>> [{"subnet_id":               |
>> |                         | 
>> "6c08c3d9-7df1-4bec-b847-19f80b9d1764",            |
>> |                         | "ip_address": 
>> "192.168.252.102"}]}                 |
>> | flavor_id               | 
>> None                                               |
>> | ha                      | 
>> False                                              |
>> | id                      | 
>> 37c1794b-58d1-4d0d-b34b-944ca411b86b               |
>> | name                    | 
>> public-router2                                     |
>> | project_id              | 
>> fe203109e67f4e39b066c9529f9fc35d                   |
>> | revision_number         | 
>> 5                                                  |
>> | routes |                                                    |
>> | status                  | 
>> ACTIVE                                             |
>> | tags |                                                    |
>> | updated_at              | 
>> 2018-10-05T12:09:36Z                               |
>> +-------------------------+----------------------------------------------------+ 
>>
>>
>> openstack network agent list
>> +-----------+------------+-----------+-------------------+-------+-------+--------------+ 
>>
>> | ID        | Agent Type | Host      | Availability Zone | Alive | 
>> State | Binary       |
>> +-----------+------------+-----------+-------------------+-------+-------+--------------+ 
>>
>> | 14b9ea75- | L3 agent   | compute1. | nova | :-)   | UP    | 
>> neutron-l3-a |
>> | 1dc1-4e37 |            | localdoma | |       |       | gent         |
>> | -a2b0-508 |            | in        | |       | |              |
>> | 3d336916d |            |           | |       | |              |
>> | 26139ec1- | Metering   | compute1. | None | :-)   | UP    | 
>> neutron-     |
>> | f4f9-4bb3 | agent      | localdoma | |       |       | metering-    |
>> | -aebb-c35 |            | in        | |       |       | agent        |
>> | 3a36ed79c |            |           | |       | |              |
>> | 2a54971f- | DHCP agent | network1. | nova | :-)   | UP    | 
>> neutron-     |
>> | 9849-4ed2 |            | localdoma | |       |       | dhcp-agent   |
>> | -b009-00e |            | in        | |       | |              |
>> | 45eb4d255 |            |           | |       | |              |
>> | 443c0b49- | Open       | compute1. | None | :-)   | UP    | 
>> neutron-     |
>> | 4484-44d2 | vSwitch    | localdoma | |       |       | openvswitch- |
>> | -a704-32a | agent      | in        | |       |       | agent        |
>> | 92ffe6982 |            |           | |       | |              |
>> | 5d00a219  | L3 agent   | network1. | nova | :-)   | UP    | 
>> neutron-vpn- |
>> | -abce-    |            | localdoma | |       |       | agent        |
>> | 48ca-     |            | in        | |       | |              |
>> | ba1e-d962 |            |           | |       | |              |
>> | 01bd7de3  |            |           | |       | |              |
>> | bc3458b4  | Open       | network1. | None | :-)   | UP    | 
>> neutron-     |
>> | -250e-    | vSwitch    | localdoma | |       |       | openvswitch- |
>> | 4adf-90e0 | agent      | in        | |       |       | agent        |
>> | -110a1a7f |            |           | |       | |              |
>> | 6ccb      |            |           | |       | |              |
>> | c29f9da8- | Metering   | network1. | None | :-)   | UP    | 
>> neutron-     |
>> | ca58-4a11 | agent      | localdoma | |       |       | metering-    |
>> | -b500-a25 |            | in        | |       |       | agent        |
>> | 3f820808e |            |           | |       | |              |
>> | cdce667d- | Metadata   | network1. | None | :-)   | UP    | 
>> neutron-     |
>> | faa4      | agent      | localdoma | |       |       | metadata-    |
>> | -49ed-    |            | in        | |       |       | agent        |
>> | 83ee-e0e5 |            |           | |       | |              |
>> | a352d482  |            |           | |       | |              |
>> | cf5ae104- | Metadata   | compute1. | None | :-)   | UP    | 
>> neutron-     |
>> | 49d7-4c85 | agent      | localdoma | |       |       | metadata-    |
>> | -a252-cc5 |            | in        | |       |       | agent        |
>> | 9a9a12789 |            |           | |       | |              |
>> +-----------+------------+-----------+-------------------+-------+-------+--------------+ 
>>
>>
>> If I check the node on which my distributed router is running it 
>> tells me that
>> it's running on the network node:
>>
>> neutron l3-agent-list-hosting-router 
>> 37c1794b-58d1-4d0d-b34b-944ca411b86b
>> +--------------------------------------+----------------------+----------------+-------+----------+ 
>>
>> | id                                   | host                 | 
>> admin_state_up | alive | ha_state |
>> +--------------------------------------+----------------------+----------------+-------+----------+ 
>>
>> | 5d00a219-abce-48ca-ba1e-d96201bd7de3 | network1.localdomain | 
>> True           | :-)   |          |
>> +--------------------------------------+----------------------+----------------+-------+----------+ 
>>
>>
>> If I check the iptable rules for the router on the compute and 
>> network nodes by running:
>>
>> ip netns exec qrouter-37c1794b-58d1-4d0d-b34b-944ca411b86b iptables 
>> -nv -L
>>
>> I see that compute1 records the traffic while network1 doesn't. Also, 
>> I did some
>> debugging and found out that the metering agent on compute1 receives 
>> an empty
>> list of routers when querying the routers that it should monitor.
>>
>> Source:
>>
>> https://github.com/openstack/neutron/blob/stable/pike/neutron/services/metering/agents/metering_agent.py#L177-L189 
>>
>>
>> https://github.com/openstack/neutron/blob/stable/pike/neutron/db/metering/metering_rpc.py#L33-L57 
>>
>>
>> I think this is because the router is running on network1. Why is it 
>> running on
>> network1 and why does it seem that the l3 agent on compute1 does the 
>> actual
>> routing?
>>
>> Thanks,
>> Alex
>>
>>
>>
>> _______________________________________________
>> Mailing list: 
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to     : openstack at lists.openstack.org
>> Unsubscribe : 
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20181130/4dcafae1/attachment.html>


More information about the Openstack mailing list