Open Stack

Jon,

The value in quota_usages was wrong.  Fixed.  Thanks for the help.

JR

On 12/11/2013 2:36 PM, Jonathan Proulx wrote:
> I've seen issues where the quota_use table in the nova database gets
> out of sync with the resources actually used.
> 
> try this query and see if the fixed_ips matches what is in use:
> 
> mysql nova -e 'select * from quota_usages where
> project_id="60d776fe573f44a4810cb294b95e09d6" and
> resource="fixed_ips";
> 
> if it doesn't you'll need to manually update it and check the other
> resources for this tenant (if one is wrong usually they all are)
> 
> I was just ranting privately after running into this issue today (not
> for the first time) about how having this in its own table is bad for
> these reasons & that I'd rather have the right answer slowly than the
> wrong answer quickly. Though this may not be your problem & I haven't
> yet reproduced the issue to see if I can file a proper bug (a previous
> bug related to this was fixed a long time ago)
> 
> -Jon
> 
> On Wed, Dec 11, 2013 at 2:11 PM, JR <botemout at gmail.com> wrote:
>> Hi all,
>>
>> I'm having an odd issue where I can't spin up new VMs; the error is below.
>> I've confirmed that the tenant starting the instance has sufficient
>> capacity.
>> I have another tenant who is running a similar number of VMs and it's not
>> having the problem.
>>
>> This is ubuntu 12.04 running grizzly.
>>
>> Any thoughts?  I assume that something in the DB is wedged ...
>>
>> Thanks
>> JR
>>
>> root at nebula01:/mnt/gdm_source# su - dd_test -c 'nova quota-show'
>> +-----------------------------+--------+
>> | Property                    | Value  |
>> +-----------------------------+--------+
>> | metadata_items              | 128    |
>> | injected_file_content_bytes | 10240  |
>> | ram                         | 256000 |
>> | floating_ips                | 128    |
>> | key_pairs                   | 100    |
>> | instances                   | 128    |
>> | security_group_rules        | 20     |
>> | injected_files              | 5      |
>> | cores                       | 128    |
>> | fixed_ips                   | 128    |
>> | injected_file_path_bytes    | 255    |
>> | security_groups             | 50     |
>> +-----------------------------+--------+
>>
>> root at nebula01:/mnt/gdm_source# su - dd_test -c 'nova show gdm-worker17-t'
>> +-----------------------------+---------------------------------------------------------------------------------+
>> | Property                    | Value
>> |
>> +-----------------------------+---------------------------------------------------------------------------------+
>> | status                      | ERROR
>> |
>> | updated                     | 2013-12-11T19:05:15Z
>> |
>> | OS-EXT-STS:task_state       | None
>> |
>> | key_name                    | dd_test
>> |
>> | image                       | gdm-worker
>> (fe0147b8-d355-4d7e-826f-195ba97d7594)                               |
>> | hostId                      |
>> 5f1a4bd11ee027f10f8fc37b0e2ce091904b94befdf5abeb196c7f67
>> |
>> | OS-EXT-STS:vm_state         | error
>> |
>> | public network              |
>> |
>> | flavor                      | i3.small_d10g
>> (7d1037f1-dfd0-4a0e-8da7-097258c9934d)                            |
>> | id                          | d7b9499b-7cc4-4d2d-80f1-50b1e4a7fc85
>> |
>> | security_groups             | [{u'name': u'dd_test'}]
>> |
>> | user_id                     | dcc3374a58984a92981d09f27c76a869
>> |
>> | name                        | gdm-worker17-t
>> |
>> | created                     | 2013-12-11T19:05:10Z
>> |
>> | fault                       | {u'message': u'NoValidHost', u'code': 500,
>> u'created': u'2013-12-11T19:05:15Z'} |
>> | OS-DCF:diskConfig           | MANUAL
>> |
>> | metadata                    | {}
>> |
>> | accessIPv4                  |
>> |
>> | accessIPv6                  |
>> |
>> | tenant_id                   | 60d776fe573f44a4810cb294b95e09d6
>> |
>> | OS-EXT-STS:power_state      | 0
>> |
>> | OS-EXT-AZ:availability_zone | nova
>> |
>> | config_drive                |
>> |
>> +-----------------------------+---------------------------------------------------------------------------------+
>> root at nebula01:/mnt/gdm_source# keystone tenant-list|grep 60d
>> | 60d776fe573f44a4810cb294b95e09d6 | dd_test |   True  |
>>
>>
>> Error:
>>
>> 2013-12-11 12:05:15.153 ERROR nova.compute.manager
>> [req-2363d68e-a947-41af-9359-2a6e6720a0ee dcc3374a58984a92981d09f27c76a869
>> 60d776fe573f44a4810cb294b95e09d6] [instance:
>> d7b9499b-7cc4-4d2d-80f1-50b1e4a7fc85] Error: ['Traceback (most recent call
>> last):\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 847, in
>> _run_instance\n    requested_networks, macs, security_groups)\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1091, in
>> _allocate_network\n    instance=instance)\n', '  File
>> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__\n
>> self.gen.next()\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1087, in
>> _allocate_network\n    security_groups=security_groups)\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 102, in
>> wrapped\n    return func(self, context, *args, **kwargs)\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 47, in
>> wrapper\n    res = f(self, context, *args, **kwargs)\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 283, in
>> allocate_for_instance\n    nw_info =
>> self.network_rpcapi.allocate_for_instance(context, **args)\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line 171, in
>> allocate_for_instance\n    topic=topic, version=\'1.9\')\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py", line
>> 80, in call\n    return rpc.call(context, self._get_topic(topic), msg,
>> timeout)\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py",
>> line 140, in call\n    return _get_impl().call(CONF, context, topic, msg,
>> timeout)\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py",
>> line 798, in call\n    rpc_amqp.get_connection_pool(conf, Connection))\n', '
>> File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py",
>> line 612, in call\n    rv = list(rv)\n', '  File
>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
>> 561, in __iter__\n    raise result\n', 'FixedIpLimitExceeded_Remote: Maximum
>> number of fixed ips exceeded\nTraceback (most recent call last):\n\n  File
>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
>> 430, in _process_data\n    rval = self.proxy.dispatch(ctxt, version, method,
>> **args)\n\n  File
>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py",
>> line 133, in dispatch\n    return getattr(proxyobj, method)(ctxt,
>> **kwargs)\n\n  File
>> "/usr/lib/python2.7/dist-packages/nova/network/floating_ips.py", line 116,
>> in allocate_for_instance\n    **kwargs)\n\n  File
>> "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 513, in
>> allocate_for_instance\n    requested_networks=requested_networks)\n\n  File
>> "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 217, in
>> _allocate_fixed_ips\n    vpn=vpn, address=address)\n\n  File
>> "/usr/lib/python2.7/dist-packages/nova/network/manager.py", line 836, in
>> allocate_fixed_ip\n    raise
>> exception.FixedIpLimitExceeded()\n\nFixedIpLimitExceeded: Maximum number of
>> fixed ips exceeded\n\n']
>>
>> --
>> Your electronic communications are being monitored; strong encryption is an
>> answer. My public key
>>
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to     : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>

-- 
Your electronic communications are being monitored; strong encryption is
an answer. My public key
<http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x4F08C504BD634953>