<html><head><meta http-equiv="content-type" content="text/html; charset=us-ascii"><style>body { line-height: 1.5; }body { font-size: 10.5pt; font-family: 'Segoe UI'; color: rgb(0, 0, 0); line-height: 1.5; }</style></head><body>
<div><span></span>Hi, all.</div><div>I hit an error about ovs agents state report after deployed openstack by chef. I check the code in agents_db. Found there are some greenthread.sleep(0) in _create_or_update_agent ( <span style="font-family: ''; font-size: 10.5pt; line-height: 1.5; background-color: window;"></span><a href="https://github.com/openstack/neutron/commit/21bf6f7e4945bc7e8c303273ad10c28b6cfc8b08" segoe="" ui'";="" font-size:="" 14px;="" font-weight:="" normal;="" font-style:="" normal;'="" style="font-family: ''; font-size: 10.5pt; line-height: 1.5; background-color: window;">https://github.com/openstack/neutron/commit/21bf6f7e4945bc7e8c303273ad10c28b6cfc8b08</a><span segoe="" ui'";="" font-size:="" 14px;="" color:="" rgb(0,="" 0,="" 0);="" background-color:="" rgb(255,="" 255,="" 255);="" font-weight:="" normal;="" font-style:="" normal;text-decoration:="" none;'="" style="font-family: ''; font-size: 10.5pt; line-height: 1.5; background-color: window;"> </span><span style="background-color: window; font-size: 10.5pt; line-height: 1.5;">). I'm not sure why we need those sleep here. But I think this can let the user hit </span><span style="font-family: Arial, sans-serif; font-size: 12px; line-height: normal; background-color: window;">DBDuplicateEntry excetpion(there already have an retry to avoid this error, but it still not safe, see </span><span style="font-family: 'Arial, sans-serif'; font-size: 12px; line-height: 1.5; background-color: window;"></span><a href="https://github.com/openstack/neutron/blob/ebc3ccfdd070fb69274069057c7c1cd8f37b508a/neutron/db/agents_db.py#L198" arial,="" sans-serif'";="" font-size:="" 12px;="" font-weight:="" normal;="" font-style:="" normal;'="" style="font-family: ''; font-size: 10.5pt; line-height: 1.5; background-color: window;">https://github.com/openstack/neutron/blob/ebc3ccfdd070fb69274069057c7c1cd8f37b508a/neutron/db/agents_db.py#L198</a><span arial,="" sans-serif'";="" font-size:="" 12px;="" color:="" rgb(0,="" 0,="" 0);="" background-color:="" rgb(255,="" 255,="" 255);="" font-weight:="" normal;="" font-style:="" normal;text-decoration:="" none;'="" style="font-family: ''; font-size: 10.5pt; line-height: 1.5; background-color: window;"> </span><span style="background-color: window; font-family: Arial, sans-serif; font-size: 12px; line-height: normal;">). This error is not easy to reproduce. I think if we make several rpc calls for </span><span style="color: rgb(209, 32, 137); font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 12px; line-height: 16.7999992370605px; white-space: pre; background-color: window;">create_or_update_agent </span><span style="font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 12px; line-height: 16.7999992370605px; white-space: pre; background-color: rgb(255, 255, 255);">at the same time before this agents has already been stored in db, we can encounter this error again.</span></div><div><span style="background-color: window; font-family: Arial, sans-serif; font-size: 12px; line-height: normal;"><br></span></div><div><font face="Arial, sans-serif"><span style="font-size: 12px; line-height: normal;">Remove those sleep can avoid this issue, but seems those sleep is used to fix some bugs. Need someone's help to fix this issue.</span></font></div>
<div><br></div><div><span style="font-family: "" segoe="" ui'";="" font-size:="" 14px;="" color:="" rgb(0,="" 0,="" 0);="" background-color:="" rgba(0,="" font-weight:="" normal;="" font-style:="" normal;text-decoration:="" none;'="">2014-11-10 23:14:36.050 31989 ERROR oslo.messaging.rpc.dispatcher [req-e07a7ed7-d22f-40e9-b299-aba6e06f9470 ] Exception during message handling: (IntegrityError) ibm_db_dbi::IntegrityError: Statement Execute Failed: [IBM][CLI Driver][DB2/LINUXX8664] SQL0803N One or more values in the INSERT statement, UPDATE statement, or foreign key update caused by a DELETE statement are not valid because the primary key, unique constraint or unique index identified by "2" constrains table "NEUTRON.AGENTS" from having duplicate values for the index key. SQLSTATE=23505 SQLCODE=-803 'INSERT INTO agents (id, agent_type, "binary", topic, host, admin_state_up, created_at, started_at, heartbeat_timestamp, description, configurations) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' ('5028b520-1a90-46c2-9e97-b1cb80d6ebe0', 'Open vSwitch agent', 'neutron-openvswitch-agent', 'N/A', 'testhl12.sce.ibm.com', '1', datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), None, '{"arp_responder_enabled": false, "tunneling_ip": "10.11.1.12", "devices": 0, "l2_population": false, "tunnel_types": ["gre", "vxlan"], "enable_distributed_routing": false, "bridge_mappings": {"default": "br-eth1"}}')
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher incoming.message))
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher result = getattr(endpoint, method)(ctxt, **new_args)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/neutron/db/agents_db.py", line 237, in report_state
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher self.plugin.create_or_update_agent(context, agent_state)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/neutron/db/agents_db.py", line 214, in create_or_update_agent
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher return self._create_or_update_agent(context, agent)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/neutron/openstack/common/excutils.py", line 82, in __exit__
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/neutron/db/agents_db.py", line 197, in create_or_update_agent
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher return self._create_or_update_agent(context, agent)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/neutron/db/agents_db.py", line 191, in _create_or_update_agent
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher greenthread.sleep(0)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 447, in __exit__
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher self.rollback()
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/util/langhelpers.py", line 58, in __exit__
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher compat.reraise(exc_type, exc_value, exc_tb)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 444, in __exit__
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher self.commit()
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 354, in commit
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher self._prepare_impl()
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 334, in _prepare_impl
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher self.session.flush()
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 1818, in flush
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher self._flush(objects)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 1936, in _flush
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher transaction.rollback(_capture_exception=True)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/util/langhelpers.py", line 58, in __exit__
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher compat.reraise(exc_type, exc_value, exc_tb)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 1900, in _flush
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher flush_context.execute()
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher rec.execute(self)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/unitofwork.py", line 525, in execute
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher uow
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/persistence.py", line 64, in save_obj
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher table, insert)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/persistence.py", line 569, in _emit_insert_statements
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher execute(statement, params)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 662, in execute
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher params)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 761, in _execute_clauseelement
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher compiled_sql, distilled_params
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 874, in _execute_context
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher context)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/oslo/db/sqlalchemy/compat/handle_error.py", line 125, in _handle_dbapi_exception
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher six.reraise(type(newraise), newraise, sys.exc_info()[2])
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/oslo/db/sqlalchemy/compat/handle_error.py", line 102, in _handle_dbapi_exception
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher per_fn = fn(ctx)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/oslo/db/sqlalchemy/exc_filters.py", line 323, in handler
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher context.is_disconnect)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher File "/usr/lib/python2.6/site-packages/oslo/db/sqlalchemy/exc_filters.py", line 223, in _db2_dupe_key_error
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher raise exception.DBDuplicateEntry([], integrity_error)
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher DBDuplicateEntry: (IntegrityError) ibm_db_dbi::IntegrityError: Statement Execute Failed: [IBM][CLI Driver][DB2/LINUXX8664] SQL0803N One or more values in the INSERT statement, UPDATE statement, or foreign key update caused by a DELETE statement are not valid because the primary key, unique constraint or unique index identified by "2" constrains table "NEUTRON.AGENTS" from having duplicate values for the index key. SQLSTATE=23505 SQLCODE=-803 'INSERT INTO agents (id, agent_type, "binary", topic, host, admin_state_up, created_at, started_at, heartbeat_timestamp, description, configurations) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' ('5028b520-1a90-46c2-9e97-b1cb80d6ebe0', 'Open vSwitch agent', 'neutron-openvswitch-agent', 'N/A', 'testhl12.sce.ibm.com', '1', datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), None, '{"arp_responder_enabled": false, "tunneling_ip": "10.11.1.12", "devices": 0, "l2_population": false, "tunnel_types": ["gre", "vxlan"], "enable_distributed_routing": false, "bridge_mappings": {"default": "br-eth1"}}')
<br>2014-11-10 23:14:36.050 31989 TRACE oslo.messaging.rpc.dispatcher
<br>2014-11-10 23:14:36.064 31989 ERROR oslo.messaging._drivers.common [req-e07a7ed7-d22f-40e9-b299-aba6e06f9470 ] Returning exception (IntegrityError) ibm_db_dbi::IntegrityError: Statement Execute Failed: [IBM][CLI Driver][DB2/LINUXX8664] SQL0803N One or more values in the INSERT statement, UPDATE statement, or foreign key update caused by a DELETE statement are not valid because the primary key, unique constraint or unique index identified by "2" constrains table "NEUTRON.AGENTS" from having duplicate values for the index key. SQLSTATE=23505 SQLCODE=-803 'INSERT INTO agents (id, agent_type, "binary", topic, host, admin_state_up, created_at, started_at, heartbeat_timestamp, description, configurations) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' ('5028b520-1a90-46c2-9e97-b1cb80d6ebe0', 'Open vSwitch agent', 'neutron-openvswitch-agent', 'N/A', 'testhl12.sce.ibm.com', '1', datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), None, '{"arp_responder_enabled": false, "tunneling_ip": "10.11.1.12", "devices": 0, "l2_population": false, "tunnel_types": ["gre", "vxlan"], "enable_distributed_routing": false, "bridge_mappings": {"default": "br-eth1"}}') to caller
<br>2014-11-10 23:14:36.064 31989 ERROR oslo.messaging._drivers.common [req-e07a7ed7-d22f-40e9-b299-aba6e06f9470 ] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch\n result = getattr(endpoint, method)(ctxt, **new_args)\n', ' File "/usr/lib/python2.6/site-packages/neutron/db/agents_db.py", line 237, in report_state\n self.plugin.create_or_update_agent(context, agent_state)\n', ' File "/usr/lib/python2.6/site-packages/neutron/db/agents_db.py", line 214, in create_or_update_agent\n return self._create_or_update_agent(context, agent)\n', ' File "/usr/lib/python2.6/site-packages/neutron/openstack/common/excutils.py", line 82, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', ' File "/usr/lib/python2.6/site-packages/neutron/db/agents_db.py", line 197, in create_or_update_agent\n return self._create_or_update_agent(context, agent)\n', ' File "/usr/lib/python2.6/site-packages/neutron/db/agents_db.py", line 191, in _create_or_update_agent\n greenthread.sleep(0)\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 447, in __exit__\n self.rollback()\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/util/langhelpers.py", line 58, in __exit__\n compat.reraise(exc_type, exc_value, exc_tb)\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 444, in __exit__\n self.commit()\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 354, in commit\n self._prepare_impl()\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 334, in _prepare_impl\n self.session.flush()\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 1818, in flush\n self._flush(objects)\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 1936, in _flush\n transaction.rollback(_capture_exception=True)\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/util/langhelpers.py", line 58, in __exit__\n compat.reraise(exc_type, exc_value, exc_tb)\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/session.py", line 1900, in _flush\n flush_context.execute()\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute\n rec.execute(self)\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/unitofwork.py", line 525, in execute\n uow\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/persistence.py", line 64, in save_obj\n table, insert)\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/orm/persistence.py", line 569, in _emit_insert_statements\n execute(statement, params)\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 662, in execute\n params)\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 761, in _execute_clauseelement\n compiled_sql, distilled_params\n', ' File "/usr/lib64/python2.6/site-packages/sqlalchemy/engine/base.py", line 874, in _execute_context\n context)\n', ' File "/usr/lib/python2.6/site-packages/oslo/db/sqlalchemy/compat/handle_error.py", line 125, in _handle_dbapi_exception\n six.reraise(type(newraise), newraise, sys.exc_info()[2])\n', ' File "/usr/lib/python2.6/site-packages/oslo/db/sqlalchemy/compat/handle_error.py", line 102, in _handle_dbapi_exception\n per_fn = fn(ctx)\n', ' File "/usr/lib/python2.6/site-packages/oslo/db/sqlalchemy/exc_filters.py", line 323, in handler\n context.is_disconnect)\n', ' File "/usr/lib/python2.6/site-packages/oslo/db/sqlalchemy/exc_filters.py", line 223, in _db2_dupe_key_error\n raise exception.DBDuplicateEntry([], integrity_error)\n', 'DBDuplicateEntry: (IntegrityError) ibm_db_dbi::IntegrityError: Statement Execute Failed: [IBM][CLI Driver][DB2/LINUXX8664] SQL0803N One or more values in the INSERT statement, UPDATE statement, or foreign key update caused by a DELETE statement are not valid because the primary key, unique constraint or unique index identified by "2" constrains table "NEUTRON.AGENTS" from having duplicate values for the index key. SQLSTATE=23505 SQLCODE=-803 \'INSERT INTO agents (id, agent_type, "binary", topic, host, admin_state_up, created_at, started_at, heartbeat_timestamp, description, configurations) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)\' (\'5028b520-1a90-46c2-9e97-b1cb80d6ebe0\', \'Open vSwitch agent\', \'neutron-openvswitch-agent\', \'N/A\', \'testhl12.sce.ibm.com\', \'1\', datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), datetime.datetime(2014, 11, 11, 4, 14, 35, 580590), None, \'{"arp_responder_enabled": false, "tunneling_ip": "10.11.1.12", "devices": 0, "l2_population": false, "tunnel_types": ["gre", "vxlan"], "enable_distributed_routing": false, "bridge_mappings": {"default": "br-eth1"}}\')\n']</span></div><div><br></div><div><br></div><hr style="width: 210px; height: 1px;" color="#b5c4df" size="1" align="left">
<div><span><div style="MARGIN: 10px; FONT-FAMILY: verdana; FONT-SIZE: 10pt"><div>huwei.xtu@gmail.com</div></div></span></div>
</body></html>