Open Stack

Indeed i ran cleanup while the VMs were running, but trust me, i won't do
it again hahha. So there should be a less painfull way to delete the ports
that arent related to any running VM. Maybe some manual check and delete
would be enough.


2013/10/8 Akihiro Motoki <amotoki at gmail.com>

> Hi,
>
> quantum-ovs-cleanup is originally designed to be run after rebooting the
> node.
> OVS keeps existing ports in its database even if the corresponding
> network devices does not exists.
> This may leads to port number mismatch. Just after rebooting a compute
> node,
> I think all actual network devices connected to VMs do not exist.
> I guess you ran quantum-ovs-cleanup when VM is running.
>
> Thanks,
> Akihiro
>
> On Wed, Oct 9, 2013 at 12:52 AM, Juan José Pavlik Salles
> <jjpavlik at gmail.com> wrote:
> > Hi Juan, i tried the cleanup and it actually cleaned too much haha.
> >
> > root at acelga:~# service quantum-plugin-openvswitch-agent stop
> > quantum-plugin-openvswitch-agent stop/waiting
> > root at acelga:~# quantum-ovs-cleanup -v
> > 2013-10-08 14:29:10     INFO [quantum.common.config] Logging enabled!
> > 2013-10-08 14:29:11     INFO [quantum.agent.ovs_cleanup_util] Cleaning
> > br-int
> > 2013-10-08 14:29:12     INFO [quantum.agent.ovs_cleanup_util] Delete
> > qvoa017fc49-48
> > 2013-10-08 14:29:12     INFO [quantum.agent.ovs_cleanup_util] OVS cleanup
> > completed successfully
> > root at acelga:~# service quantum-plugin-openvswitch-agent start
> > quantum-plugin-openvswitch-agent start/running, process 26631
> > root at acelga:~# ovs-vsctl show
> > bb36ccf6-ba02-48df-970a-4b2b39b12e84
> >     Bridge "br-eth1"
> >         Port "eth1"
> >             Interface "eth1"
> >         Port "br-eth1"
> >             Interface "br-eth1"
> >                 type: internal
> >     Bridge br-int
> >         Port "int-br-eth1"
> >             Interface "int-br-eth1"
> >         Port br-int
> >             Interface br-int
> >                 type: internal
> >     ovs_version: "1.4.0+build0"
> > root at acelga:~#
> >
> > "qvoa017fc49-48" port belonged to instance-00000077 and is gone now,
> > eventough i can see it in ifconfig i can't ping the VM anymore:
> >
> > root at acelga:~# ifconfig
> > ...
> > qbra017fc49-48 Link encap:Ethernet  HWaddr fe:16:3e:fc:48:06
> >           inet6 addr: fe80::b0f7:bbff:feeb:12c2/64 Scope:Link
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:812186 errors:0 dropped:220 overruns:0 frame:0
> >           TX packets:3682 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:0
> >           RX bytes:55501848 (55.5 MB)  TX bytes:316604 (316.6 KB)
> > ...
> > root at acelga:~#
> >
> > This brings up some questions:
> >
> > Is this some kind of known problem?
> > Should i clean ovs frequently?
> > What about the running VMs, will they loose connection during the
> cleanup?
> >
> > Thanks guys!
> >
> >
> > 2013/10/8 JuanFra Rodriguez Cardoso <juanfra.rodriguez.cardoso at gmail.com
> >
> >>
> >> You can try:
> >>
> >> 1) Stop quantum agents
> >> 2) Run 'quantum-ovs-cleanup' (with -v flag for enable verbose)
> >> 3) Start quantum agents
> >>
> >> Regards,
> >> ---
> >> JuanFra
> >>
> >>
> >> 2013/10/7 Juan José Pavlik Salles <jjpavlik at gmail.com>
> >>>
> >>> A few hours ago i was checking my compute nodes when i noticed that
> they
> >>> have a load of 5 but they just run a few VMs. I started looking for
> problems
> >>> and i found that quantum-openvswitch-agent was taking too much CPU
> time.
> >>>
> >>> I checked the logs and found lots of these:
> >>>
> >>> root at acelga:~# tail -f /var/log/quantum/openvswitch-agent.log
> >>> Exit code: 0
> >>> Stdout: '{attached-mac="fa:16:3e:66:8a:cb",
> >>> iface-id="b8d3bb55-90bb-4a93-9daf-9d5da52171db", iface-status=active,
> >>> vm-uuid="5406eca9-3b99-473d-b4e4-3ac53568a5a8"}\n'
> >>> Stderr: ''
> >>> 2013-10-07 02:58:11    DEBUG [quantum.agent.linux.utils] Running
> command:
> >>> ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl',
> >>> '--timeout=2', 'get', 'Interface', 'qvobe75299f-e5', 'external_ids']
> >>> 2013-10-07 02:58:11    DEBUG [quantum.agent.linux.utils]
> >>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> >>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'qvof13c912d-35',
> >>> 'external_ids']
> >>> Exit code: 0
> >>> Stdout: '{attached-mac="fa:16:3e:8e:7d:54",
> >>> iface-id="f13c912d-3516-4ec2-9d60-5cc01db0b2e3", iface-status=active,
> >>> vm-uuid="de1f79cd-7289-40d8-a5cf-acf43c089727"}\n'
> >>> Stderr: ''
> >>> 2013-10-07 02:58:11    DEBUG [quantum.agent.linux.utils] Running
> command:
> >>> ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl',
> >>> '--timeout=2', 'get', 'Interface', 'qvof4329543-aa', 'external_ids']
> >>> 2013-10-07 02:58:12    DEBUG [quantum.agent.linux.utils]
> >>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> >>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'qvobe75299f-e5',
> >>> 'external_ids']
> >>> Exit code: 0
> >>> Stdout: '{attached-mac="fa:16:3e:4f:92:9f",
> >>> iface-id="be75299f-e597-4f37-85b5-e517883bbfeb", iface-status=active,
> >>> vm-uuid="dd9783c1-344a-43b7-9cc1-b969c003f838"}\n'
> >>> Stderr: ''
> >>> 2013-10-07 02:58:12    DEBUG [quantum.agent.linux.utils] Running
> command:
> >>> ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf', 'ovs-vsctl',
> >>> '--timeout=2', 'get', 'Interface', 'qvod00a1408-ad', 'external_ids']
> >>> 2013-10-07 02:58:12    DEBUG [quantum.agent.linux.utils]
> >>> Command: ['sudo', 'quantum-rootwrap', '/etc/quantum/rootwrap.conf',
> >>> 'ovs-vsctl', '--timeout=2', 'get', 'Interface', 'qvof4329543-aa',
> >>> 'external_ids']
> >>> ...
> >>>
> >>> It seems to be just a normal check interface message, but i just have 1
> >>> instance running on that compute node:
> >>>
> >>> 4oot at acelga:~# virsh list
> >>> setlocale: No such file or directory
> >>>  Id    Name                           State
> >>> ----------------------------------------------------
> >>>  1     instance-00000077              running
> >>>
> >>> root at acelga:~#
> >>>
> >>> With just one network interface attached to it. So i checked ovs to see
> >>> if there was something wrong.. and :
> >>>
> >>> root at acelga:~# ovs-vsctl show
> >>> bb36ccf6-ba02-48df-970a-4b2b39b12e84
> >>>     Bridge "br-eth1"
> >>>         Port "eth1"
> >>>             Interface "eth1"
> >>>         Port "phy-br-eth1"
> >>>             Interface "phy-br-eth1"
> >>>         Port "br-eth1"
> >>>             Interface "br-eth1"
> >>>                 type: internal
> >>>     Bridge br-int
> >>>         Port "qvoa017fc49-48"
> >>>             tag: 1
> >>>             Interface "qvoa017fc49-48"
> >>>         Port "qvo38b6ba57-81"
> >>>             tag: 1
> >>>             Interface "qvo38b6ba57-81"
> >>>         Port "qvo27d0051a-0d"
> >>>             tag: 1
> >>>             Interface "qvo27d0051a-0d"
> >>>         Port "int-br-eth1"
> >>>             Interface "int-br-eth1"
> >>>         Port "qvof4329543-aa"
> >>>             tag: 4095
> >>>             Interface "qvof4329543-aa"
> >>>         Port "qvo6bf34ce3-25"
> >>>             tag: 1
> >>>             Interface "qvo6bf34ce3-25"
> >>>         Port "qvo0c4c9cd8-2b"
> >>>             tag: 1
> >>>             Interface "qvo0c4c9cd8-2b"
> >>>            ...
> >>>         Port "qvo54a38640-30"
> >>>             tag: 2
> >>>             Interface "qvo54a38640-30"
> >>>     ovs_version: "1.4.0+build0"
> >>>
> >>> Too many ports, way too many actually considering i'm running just one
> >>> instance. A few weeks ago i did some tests, creating many instances
> deleting
> >>> them and so on, so it could be related to that, maybe during delete
> >>> operations the ovs ports weren't actually deleted and that's why i
> have so
> >>> many of them. Did someone notice this behaviour before? How could i
> cleanly
> >>> get rid of them? I'm runing most of the services in debug mode because
> the
> >>> cloud is not in production yet.
> >>>
> >>> --
> >>> Pavlik Salles Juan José
> >>>
> >>> _______________________________________________
> >>> OpenStack-operators mailing list
> >>> OpenStack-operators at lists.openstack.org
> >>>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> >>>
> >>
> >
> >
> >
> > --
> > Pavlik Salles Juan José
> >
> > _______________________________________________
> > OpenStack-operators mailing list
> > OpenStack-operators at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> >
>
>
>
> --
> Akihiro MOTOKI <amotoki at gmail.com>
>



-- 
Pavlik Salles Juan José
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20131009/c17afd16/attachment.html>