Open Stack

Hello:

I think I'm repeating myself here but we have two approaches to solve this
problem:
* The easiest one, at least for the L3 agent, is to report an INFO level
log before and after the full sync. That could be parsed by any tool to
detect that. You can propose a patch to the Neutron repository.
* https://bugs.launchpad.net/neutron/+bug/2011422: a more elaborated way to
report the agent status. That could provide the start flag, the revived
flag, the sync processing flag and many other ones that could be defined
only for this specific agent.

Regards.

On Mon, Mar 20, 2023 at 4:33 PM Jan Horstmann <J.Horstmann at mittwald.de>
wrote:

> On Wed, 2023-03-15 at 16:10 +0000, Felix Hüttner wrote:
> > Hi,
> >
> > > Subject: Re: [neutron] detecting l3-agent readiness
> > >
> > > Hi,
> > >
> > > Dnia poniedziałek, 13 marca 2023 16:35:43 CET Felix Hüttner pisze:
> > > > Hi Mohammed,
> > > >
> > > > > Subject: [neutron] detecting l3-agent readiness
> > > > >
> > > > > Hi folks,
> > > > >
> > > > > I'm working on improving the stability of rollouts when using
> Kubernetes as a control
> > > plane, specifically around the L3 agent, it seems that I have not
> found a clear way to
> > > detect in the code path where the L3 agent has finished it's initial
> sync..
> > > > >
> > > >
> > > > We build such a solution here:
> https://gitlab.com/yaook/images/neutron-l3-agent/-
> > > /blob/devel/files/startup_wait_for_ns.py
> > > > Basically we are checking against the neutron api what routers
> should be on the node and
> > > then validate that all keepalived processes are up and running.
> > >
> > > That would work only for HA routers. If You would also have routers
> which aren't "ha" this
> > > method may fail.
> > >
> >
> > Yep, since we only have HA routers that works fine for us. But I guess
> it should also work for non-ha routers without too much adoption (maybe
> just check for namespaces instead of keepalived).
> >
>
> Instead of counting processes I have been using the l3 agent's
> `configurations.routers` field to determine its readiness.
> From my understanding comparing this number with the number of active
> routers hosted by the agent should be a good indicator of its sync
> status.
> Using two api calls for this is inherently racy, but could be a
> sufficient workaround for environments with a moderate number of
> router events.
> So a simple test snippet for the sync status of all agents could be:
>
> ```
> import sys
> import openstack
> client = openstack.connection.Connection(
>    ...
> )
> l3_agent_synced = [
>     len([
>         router
>         for router in client.network.agent_hosted_routers(agent)
>             if router.is_admin_state_up
>     ]) <= client.network.get_agent(agent).configuration["routers"]
>     for agent in client.network.agents()
>         if agent.agent_type == "L3 agent"
>            and (agent.configuration["agent_mode"] == "dvr_snat"
>                 or agent.configuration["agent_mode"] == "legacy")
> ]
> if not all(l3_agent_synced):
>     sys.exit(1)
> ```
>
> Please let me know if I am way off with this approach :)
>
>
> > > >
> > > > > Am I missing it somewhere or is the architecture built in a way
> that doesn't really
> > > answer that question?
> > > > >
> > > >
> > > > Adding a option in the neutron api would be a lot nicer. But i guess
> that also counts
> > > for l2 and dhcp agents.
> > > >
> > > >
> > > > > Thanks
> > > > > Mohammed
> > > > >
> > > > >
> > > > > --
> > > > > Mohammed Naser
> > > > > VEXXHOST, Inc.
> > > >
> > > > --
> > > > Felix Huettner
> > > > Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur
> für die Verwertung
> > > durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht der
> vorgesehene Empfänger
> > > sein, setzen Sie den Absender bitte unverzüglich in Kenntnis und
> löschen diese E Mail.
> > > Hinweise zum Datenschutz finden Sie hier<
> https://www.datenschutz.schwarz>.
> > > >
> > >
> > >
> > > --
> > > Slawek Kaplonski
> > > Principal Software Engineer
> > > Red Hat
> >
> > --
> > Felix Huettner
> > Diese E Mail enthält möglicherweise vertrauliche Inhalte und ist nur für
> die Verwertung durch den vorgesehenen Empfänger bestimmt. Sollten Sie nicht
> der vorgesehene Empfänger sein, setzen Sie den Absender bitte unverzüglich
> in Kenntnis und löschen diese E Mail. Hinweise zum Datenschutz finden Sie
> hier<https://www.datenschutz.schwarz>.
>
> --
> Jan Horstmann
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230320/00b0e5a7/attachment.htm>