Open Stack

  Hi,

   We find a problem that neutron ovs-agent deletes taas flows.

   o) Problem description:

      Background:
       At Liberty, a bug fix to drop only old flows was merged
       to Neutron.
       When ovs-agent is restarted, the cleanup logic drops flow
       entries unless they are stamped by agent_uuid (recorded as
       a cookie).

       bug: #1383674
            "Restarting neutron openvswitch agent causes network
             hiccup by throwing away all flows"
            https://bugs.launchpad.net/neutron/+bug/1383674

       commit: 73673beacd75a2d9f51f15b284f1b458d32e992e (patch)
 
https://git.openstack.org/cgit/openstack/neutron/commit/?id=73673beacd75a2d9f51f15b284f1b458d32e992e


      Problem:
       Cleanup will be done only once, but it seems not to work
       until port configuration is changed.

       Therefore, taas flows will be deleted as follows:
        1. Start a new compute node or restart an existing compute node.
        2. Start taas agent on the compute node.
           --> taas agent creates flows
               (these flows are not stamped by using ovs-agent's uuid)
        3. Deploy a vm on the compute node.
           --> 1. neutron changes port configuration
               2. subsequently, the cleanup logic is invoked
               3. ovs-agent drops taas flows

      Specifically, following taas flows in br_tun are dropped:
      -----
       table=35, priority=2,reg0=0x0 actions=resubmit(,36)
       table=35, priority=1,reg0=0x1 actions=resubmit(,36)
       table=35, priority=1,reg0=0x2 actions=resubmit(,37)
      -----

      log in q-agt.log
      -----
 
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.ovs_ofctl.ofswitch
         req-e5739280-7116-4802-b5ba-d6964b4c5557 Deleting flow
         cookie=0x0, duration=434.59s, table=35, n_packets=0, n_bytes=0,
         idle_age=434, priority=2,reg0=0x0 actions=resubmit(,36)
 
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.ovs_ofctl.ofswitch
         req-e5739280-7116-4802-b5ba-d6964b4c5557 Deleting flow
         cookie=0x0, duration=434.587s, table=35, n_packets=0, n_bytes=0,
         idle_age=434, priority=1,reg0=0x1 actions=resubmit(,36)
 
neutron.plugins.ml2.drivers.openvswitch.agent.openflow.ovs_ofctl.ofswitch
         req-e5739280-7116-4802-b5ba-d6964b4c5557 Deleting flow
         cookie=0x0, duration=434.583s, table=35, n_packets=0, n_bytes=0,
         idle_age=434, priority=1,reg0=0x2 actions=resubmit(,37)
      -----


   o) Impact for TaaS:

      Because flows in br_tun are dropped by the cleanup logic, mirrored
      packets will not send to a monitoring vm running on another host.

      Note: Mirrored packets are sent in case of both source vm and
            monitoring vm are running on the same host. (not affected by
            flows in br_tun)


   o) How to reproduce:

      1. Start a new compute node or restart an existing compute node.
         (Actually, restarting ovs-agent is enough.)
      2. Start (or restart) taas agent on the compute node.
      3. Deploy a vm on the compute node.
         --> The cleanup logic drops taas flows.


   o) Workaround:

      After a vm is deployed on a (re)started compute node, restart taas
      agent before creating a tap-service or tap-flow.
      That is, create taas flows after cleanup has been done.

      Note that cleanup will be done only once during an ovs-agent is
      running.


   o) An idea to fix:

      1. Set "taas" stamp(*) to taas flows.
      2. Modify the cleanup logic in ovs-agent not to delete entries
         stamped as "taas".

      * Maybe a static string.
        If we need to use a string which generated dynamically
        (e.g. uuid), API to interact with ovs-agent is required.


  ---
  Soichi Shigeta