[Openstack-operators] Problem with multiple nova-consoleauth

Daneyon Hansen (danehans) danehans at cisco.com
Mon May 13 18:55:24 UTC 2013


Are you using 2013.1.1?

Regards,
Daneyon Hansen, CCIE 9950
Software Engineer
Office of the Cloud CTO
Mobile: 303-718-0400
Office: 720-875-2936
Email: danehans at cisco.com<mailto:danehans at cisco.com>

On May 13, 2013, at 12:16 PM, "Juan José Pavlik Salles" <jjpavlik at gmail.com<mailto:jjpavlik at gmail.com>> wrote:

Hi guys, i just tried using memcached_servers flag in my nova.conf file, but when i go to the dashboard i get "Error: Unable to retrieve quota information.".

memcached_servers=172.19.136.10:11211<http://172.19.136.10:11211>,172.19.136.11:11211<http://172.19.136.11:11211>

I tried using just one server instead of both of them, but it doesn't work either, i get the exact same message in the dashboard. Maybe i'm missing something else, i also changed local_settings.py from the dashboard, but i didn't keep the changes because they didn work.



2013/5/13 Daneyon Hansen (danehans) <danehans at cisco.com<mailto:danehans at cisco.com>>

I created 1158958 when using memcached_servers instead of the Nova in-process cache.  I needed to use memcached to provide resiliency for storing the consoleauth session token.  I would get the following error whenever I would configure Nova for memcached (memcached_servers=).  I observed the same behavior whether I used a VIP or a string of actual IP's for my memcached servers.

2013-03-22 19:22:03.921 ERROR nova.api.openstack [req-3b839f39-ebe5-460a-9499-736323132e43 cd9a6b253278436eb3fcbe4f92a96694 3eccdb2a9331419c96ac9ff336110b65] Caught error: Keys must be str()'s, not unicode. Convert your unicode strings using mystring.encode(charset)!
2013-03-22 19:22:03.921 10563 TRACE nova.api.openstack Traceback (most recent call last):

Here's my nova.conf:


[DEFAULT]

network_api_class=nova.network.quantumv2.api.API

quantum_url=http://192.168.220.40:9696

quantum_admin_auth_url=http://192.168.220.40:35357/v2.0

quantum_auth_strategy=keystone

quantum_admin_tenant_name=services

quantum_admin_username=quantum

quantum_admin_password=keystone_admin

firewall_driver=nova.virt.firewall.NoopFirewallDriver

service_quantum_metadata_proxy=true

quantum_metadata_proxy_shared_secret=quantum_proxy_secret

dhcpbridge_flagfile=/etc/nova/nova.conf

dhcpbridge=/usr/bin/nova-dhcpbridge

logdir=/var/log/nova

state_path=/var/lib/nova

lock_path=/var/lock/nova

iscsi_helper=tgtadm

libvirt_use_virtio_for_bridges=True

verbose=false

ec2_private_dns_show_ip=True

api_paste_config=/etc/nova/api-paste.ini

image_service=nova.image.glance.GlanceImageService

rpc_backend=nova.rpc.impl_kombu

rabbit_ha_queues=True

rabbit_hosts=control01:5672,control02:5672,control03:5672

glance_api_servers=192.168.220.40:9292<http://192.168.220.40:9292>

service_down_time=60

rabbit_port=5672

novncproxy_port=6080

rabbit_virtual_host=/

sql_connection=mysql://nova:nova_pass@192.168.220.40/nova<http://nova:nova_pass@192.168.220.40/nova>

memcached_servers=192.168.220.41:11211<http://192.168.220.41:11211>,192.168.220.42:11211<http://192.168.220.42:11211>,192.168.220.43:11211<http://192.168.220.43:11211>

rabbit_userid=openstack_rabbit_user

rabbit_password=openstack_rabbit_password

novncproxy_host=0.0.0.0

metadata_listen=192.168.221.41

ec2_listen=192.168.220.41

enabled_apis=ec2,osapi_compute,metadata

osapi_compute_listen=192.168.220.41

volume_api_class=nova.volume.cinder.API

auth_strategy=keystone

rootwrap_config= /etc/nova/rootwrap.conf

novncproxy_base_url=http://192.168.220.40:6080/vnc_auto.html

novncproxy_host=192.168.220.41


Regards,
Daneyon Hansen
Software Engineer
Email: danehans at cisco.com<mailto:danehans at cisco.com>
Phone: 303-718-0400
http://about.me/daneyon_hansen

From: Emilien Macchi <emilien.macchi at enovance.com<mailto:emilien.macchi at enovance.com>>
Date: Monday, May 13, 2013 8:08 AM
To: Cisco Employee <danehans at cisco.com<mailto:danehans at cisco.com>>
Cc: Juan José Pavlik Salles <jjpavlik at gmail.com<mailto:jjpavlik at gmail.com>>, "openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>" <openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>>

Subject: Re: [Openstack-operators] Problem with multiple nova-consoleauth

Interesting bugs, thank's. But I'm not sure they concern nova-consoleauth in my opinion.

Is somebody here still working on Folsom ? I'm curious if nova-consoleauth is working in active / active too.


________________________________
From: "Daneyon Hansen (danehans)" <danehans at cisco.com<mailto:danehans at cisco.com>>
To: "Emilien Macchi" <emilien.macchi at enovance.com<mailto:emilien.macchi at enovance.com>>, "Juan José Pavlik Salles" <jjpavlik at gmail.com<mailto:jjpavlik at gmail.com>>
Cc: openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>
Sent: Monday, May 13, 2013 3:52:25 PM
Subject: Re: [Openstack-operators] Problem with multiple nova-consoleauth


Keep in mind there are a couple memcached bugs that were recently resolved in 2013.1.1:

https://bugs.launchpad.net/nova/+bug/1158958
https://bugs.launchpad.net/nova/+bug/1160900

Regards,
Daneyon Hansen
Software Engineer
Email: danehans at cisco.com<mailto:danehans at cisco.com>
Phone: 303-718-0400
http://about.me/daneyon_hansen

From: Emilien Macchi <emilien.macchi at enovance.com<mailto:emilien.macchi at enovance.com>>
Date: Monday, May 13, 2013 1:38 AM
To: Juan José Pavlik Salles <jjpavlik at gmail.com<mailto:jjpavlik at gmail.com>>
Cc: "openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>" <openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>>
Subject: Re: [Openstack-operators] Problem with multiple nova-consoleauth

Hi,


We are running :

  *   2 memcached servers on dedicated nodes
  *   2 nova-consoleauth with memcached_servers flag in nova.conf (don't forget to install python-memcache on nova-consoleauth nodes)
  *   2 nova-html5proxy in load balancing with HAproxy in front-end.

It works fine.


Cheers !


Emilien


________________________________
From: "Juan José Pavlik Salles" <jjpavlik at gmail.com<mailto:jjpavlik at gmail.com>>
To: "Sam Morrison" <sorrison at gmail.com<mailto:sorrison at gmail.com>>
Cc: openstack-operators at lists.openstack.org<mailto:openstack-operators at lists.openstack.org>
Sent: Monday, May 13, 2013 2:37:56 AM
Subject: Re: [Openstack-operators] Problem with multiple nova-consoleauth

Great, thank guys!!, Should i run one memcache server??? instead of running many of them, one in every compute node, using the configuration John Dewey says.


2013/5/12 Sam Morrison <sorrison at gmail.com<mailto:sorrison at gmail.com>>
If you run multiple consoleauths they need to talk to the same memcached server.

Sam


On 12/05/2013, at 11:46 AM, Juan José Pavlik Salles <jjpavlik at gmail.com<mailto:jjpavlik at gmail.com>> wrote:


Hi guys, on friday i was trying to reach my vms through novnc proxy and i ran into some kind of odd behaviour. One every two attempts were successfull so we started watching logs. after a couple of hours stopping and starting services we realized that this strange behaviour was due to we had two nova-consoleauth instances running, one per compute node. We stopped one of the instances a everything started to work just fine, even both novncproxies.
I found this https://bugs.launchpad.net/nova/+bug/989337 but it's suppossed to be fixed in folsom, and i'm running grizzly.

Should i run just one instance of nova-consoleauth?

_______________________________________________

OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators




--
Pavlik Salles Juan José

_______________________________________________
OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators





--
Pavlik Salles Juan José
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20130513/e155636e/attachment.html>


More information about the OpenStack-operators mailing list