Open Stack

On Wed, 2023-03-15 at 16:10 +0000, Felix Hüttner wrote:
> Hi,
> 
> > Subject: Re: [neutron] detecting l3-agent readiness
> > 
> > Hi,
> > 
> > Dnia poniedziałek, 13 marca 2023 16:35:43 CET Felix Hüttner pisze:
> > > Hi Mohammed,
> > > 
> > > > Subject: [neutron] detecting l3-agent readiness
> > > > 
> > > > Hi folks,
> > > > 
> > > > I'm working on improving the stability of rollouts when using Kubernetes as a control
> > plane, specifically around the L3 agent, it seems that I have not found a clear way to
> > detect in the code path where the L3 agent has finished it's initial sync..
> > > > 
> > > 
> > > We build such a solution here: https://gitlab.com/yaook/images/neutron-l3-agent/-
> > /blob/devel/files/startup_wait_for_ns.py
> > > Basically we are checking against the neutron api what routers should be on the node and
> > then validate that all keepalived processes are up and running.
> > 
> > That would work only for HA routers. If You would also have routers which aren't "ha" this
> > method may fail.
> > 
> 
> Yep, since we only have HA routers that works fine for us. But I guess it should also work for non-ha routers without too much adoption (maybe just check for namespaces instead of keepalived).
> 

Instead of counting processes I have been using the l3 agent's
`configurations.routers` field to determine its readiness.
From my understanding comparing this number with the number of active
routers hosted by the agent should be a good indicator of its sync
status.
Using two api calls for this is inherently racy, but could be a
sufficient workaround for environments with a moderate number of
router events.
So a simple test snippet for the sync status of all agents could be:

```
import sys
import openstack
client = openstack.connection.Connection(
   ...
)
l3_agent_synced = [
    len([
        router
        for router in client.network.agent_hosted_routers(agent)
            if router.is_admin_state_up
    ]) <= client.network.get_agent(agent).configuration["routers"]
    for agent in client.network.agents()
        if agent.agent_type == "L3 agent"
           and (agent.configuration["agent_mode"] == "dvr_snat"
                or agent.configuration["agent_mode"] == "legacy")
]
if not all(l3_agent_synced):
    sys.exit(1)
```

Please let me know if I am way off with this approach :)


> > > 
> > > > Am I missing it somewhere or is the architecture built in a way that doesn't really
> > answer that question?
> > > > 
> > > 
> > > Adding a option in the neutron api would be a lot nicer. But i guess that also counts
> > for l2 and dhcp agents.
> > > 
> > > 
> > > > Thanks
> > > > Mohammed
> > > > 
> > > > 
> > > > --
> > > > Mohammed Naser
> > > > VEXXHOST, Inc.
> > > 
> > > --
> > > Felix Huettner
> > > Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für die Verwertung
> > durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der vorgesehene Empfänger
> > sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und löschen diese E Mail.
> > Hinweise zum Datenschutz finden Sie hier<https://www.datenschutz.schwarz>.
> > > 
> > 
> > 
> > --
> > Slawek Kaplonski
> > Principal Software Engineer
> > Red Hat
> 
> --
> Felix Huettner
> Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie hier<https://www.datenschutz.schwarz>.

-- 
Jan Horstmann