[openstack-dev] [Neutron] minimal device driver for VPN
Paul Michali (pcm)
pcm at cisco.com
Mon Jul 28 10:44:59 UTC 2014
It doesn’t sound like you have the stack running with your drivers. Here’s what I remember happens with DevStack…
When stack.sh is run, it will take the /opt/stack/neutron/etc/vpn_agent.ini file and copy it to /etc/neutron/vpn_agent.ini. If prior to stacking, you have not added and uncommented your device driver in /opt/stack/neutron/etc/vpn_agent.ini, then it will not be loaded. From you log, it looks like maybe that is not happening.
What I do is add my driver to /opt/stack/neutron/etc/vpn_agent.ini, like this:
diff --git a/etc/vpn_agent.ini b/etc/vpn_agent.ini
index c3089df..e1cf777 100644
--- a/etc/vpn_agent.ini
+++ b/etc/vpn_agent.ini
@@ -6,7 +6,7 @@
# vpn device drivers which vpn agent will use
# If we want to use multiple drivers, we need to define this option multiple times.
# vpn_device_driver=neutron.services.vpn.device_drivers.ipsec.OpenSwanDriver
-# vpn_device_driver=neutron.services.vpn.device_drivers.cisco_ipsec.CiscoCsrIPsecDriver
+vpn_device_driver=neutron.services.vpn.device_drivers.cisco_ipsec.CiscoCsrIPsecDriver
# vpn_device_driver=another_driver
[ipsec]
You can enable both, if desired.
In addition, when running stack.sh the neutron.conf file is also placed into /etc/neutron/ and I think it will use the default VPN service driver. I think you could probably make your service driver the default in /etc/opt/stack/etc/neutron.conf, instead of the reference driver. What I do is apply this patch, so that I can specify my service driver as the default driver to be loaded at stack.sh time:
diff --git a/lib/neutron b/lib/neutron
index 2b8b176..755a120 100644
--- a/lib/neutron
+++ b/lib/neutron
@@ -792,6 +795,7 @@ function _configure_neutron_fwaas {
function _configure_neutron_vpn {
neutron_vpn_install_agent_packages
neutron_vpn_configure_common
+ neutron_vpnaas_configure_driver
}
function _configure_neutron_cisco_router {
diff --git a/lib/neutron_plugins/services/vpn b/lib/neutron_plugins/services/vpn
index d920ba6..26d60dd 100644
--- a/lib/neutron_plugins/services/vpn
+++ b/lib/neutron_plugins/services/vpn
@@ -18,6 +18,12 @@ function neutron_vpn_configure_common {
_neutron_service_plugin_class_add $VPN_PLUGIN
}
+function neutron_vpnaas_configure_driver() {
+ echo "Setting service driver"
+ iniset_multiline $NEUTRON_CONF service_providers service_provider "VPN:cisco:neutron.services.vpn.service_drivers.cisco_ipsec.CiscoCsrIPsecVPNDriver:default"
+ grep "VPN" /etc/neutron/neutron.conf
+}
+
function neutron_vpn_stop {
local ipsec_data_dir=$DATA_DIR/neutron/ipsec
local pids
I was doing this as a holdover of using the service type framework, when you could specify multiple service drivers, with one as the default. The framework was abandoned, and will be replaced by flavor framework, but I kept the modification in place. Just changing neutron.conf is easier, just keep in mind that if you stack with RECLONE=yes, you loose all the mods in /opt/stack/neutron/etc/.
The alternative to all this is to stack.sh, then go to /etc/neutron/ and change neutron.conf and vpn_agent.ini to have the service and device driver you want, and then restart the q-svc and q-vpn processes (you can use "screen -x stack”, select the screen for the process, control-C, up arrow, and then re-run the process). If you want, add debug messages that show that your drivers are getting loaded.
PCM (Paul Michali)
MAIL …..…. pcm at cisco.com
IRC ……..… pcm_ (irc.freenode.com)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
On Jul 27, 2014, at 2:05 PM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
> I have in my q-svc log this line:
>
> 2014-07-27 19:46:02.243 DEBUG neutron.service [-] service_providers.service_provider = ['VPN:fake_junos_vpnaas:neutron.services.vpn.service_drivers.fake_service_driver.FakeServiceDriver:default'] from (pid=2770) log_opt_values /usr/local/lib/python2.7/dist-packages/oslo/config/cfg.py:1988
>
> proving that service driver is loaded.
>
> When I have default configuration in /etc/neutron/vpn_agent.ini I see this line, loading OpenSwan driver:
>
> 014-07-27 19:41:48.955 DEBUG neutron.openstack.common.service [req-4cdcc041-cc13-4fe8-97d3-e743eab43f84 None None] vpnagent.vpn_device_driver = ['neutron.services.vpn.device_drivers.ipsec.OpenSwanDriver'] from (pid=30265) log_opt_values /usr/local/lib/python2.7/dist-packages/oslo/config/cfg.py:1988
>
> Using my configuration I don't see the equivalent line loading my device driver and I don't see any error.
>
> I don't know how to proceed.
>
>
> Julio C. Barrera Juez
> Office phone: (+34) 93 357 99 27 (ext. 527)
> Office mobile phone: (+34) 625 66 77 26
> Distributed Applications and Networks Area (DANA)
> i2CAT Foundation, Barcelona
>
>
> On 24 July 2014 20:34, Paul Michali (pcm) <pcm at cisco.com> wrote:
> Check /etc/neutron/neutron.conf and see if your service driver is correctly specified for VPN. You can also check the q-svc and q-vpn logs at the beginning to see if the service and device drivers were actually loaded by the plugin and agent, respectively. You can check vpn_agent.ini in same area, to see if your device driver is called out.
>
> Regards,
>
> PCM (Paul Michali)
>
> MAIL …..…. pcm at cisco.com
> IRC ……..… pcm_ (irc.freenode.com)
> TW ………... @pmichali
> GPG Key … 4525ECC253E31A83
> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>
>
>
> On Jul 24, 2014, at 2:11 PM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
>
>> Hi again.
>>
>> With previous days code, we don't experience any error in our logs, but we don't see any logs in q-svc nor q-vpn. When we execute any Neutron VPN command like neutron vpn-ikepolicy-list we receive:
>>
>> 404 Not Found
>>
>> The resource could not be found.
>>
>> And in q-svc logs we see:
>>
>> 2014-07-24 19:50:37.587 DEBUG routes.middleware [req-8efb06d9-36fb-44e4-ab94-2221daadd2a5 demo 4af34184cec14e70a15dee0508f16e7e] No route matched for GET /vpn/ikepolicies.json from (pid=4998) __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:97
>> 2014-07-24 19:50:37.588 DEBUG routes.middleware [req-8efb06d9-36fb-44e4-ab94-2221daadd2a5 demo 4af34184cec14e70a15dee0508f16e7e] No route matched for GET /vpn/ikepolicies.json from (pid=4998) __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:97
>>
>> Why logs in our plugin are not printed? Why /usr/lib/python2.7/dist-packages/routes/middleware.py is not finding our service driver?
>>
>> Thanks.
>>
>>
>>
>> Julio C. Barrera Juez
>> Office phone: (+34) 93 357 99 27 (ext. 527)
>> Office mobile phone: (+34) 625 66 77 26
>> Distributed Applications and Networks Area (DANA)
>> i2CAT Foundation, Barcelona
>>
>>
>> On 18 July 2014 12:56, Paul Michali (pcm) <pcm at cisco.com> wrote:
>> No docs, it’s an internal API between service and device driver (so you can implement it however you desire. You can look at the reference and Cisco ones for examples (they are currently both the same, although the Cisco one will likely change in the future). You’ll need to define a “topic” for the RPC between the two drivers that is unique to your implementation. Again, look at the existing ones and look for “topic” variable to see what strings they map to.
>>
>> From service driver to device driver, there is only one API, vpnservice_updated(), and in the other direction there are two, get_vpn_services_on_host() and udpate_status().
>>
>> Regards,
>>
>>
>> PCM (Paul Michali)
>>
>> MAIL …..…. pcm at cisco.com
>> IRC ……..… pcm_ (irc.freenode.com)
>> TW ………... @pmichali
>> GPG Key … 4525ECC253E31A83
>> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>>
>>
>>
>> On Jul 18, 2014, at 2:30 AM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
>>
>>> Is there any documentation about these RPC messages? Or de we need to use examples as guide?
>>>
>>> Once again, thank you Paul.
>>>
>>>
>>> Julio C. Barrera Juez
>>> Office phone: (+34) 93 357 99 27 (ext. 527)
>>> Office mobile phone: (+34) 625 66 77 26
>>> Distributed Applications and Networks Area (DANA)
>>> i2CAT Foundation, Barcelona
>>>
>>>
>>> On 17 July 2014 20:37, Paul Michali (pcm) <pcm at cisco.com> wrote:
>>> So you have your driver loading… great!
>>>
>>> The service driver will log in screen-q-svc.log, provided you have the service driver called out in neutron.conf (as the only one for VPN).
>>>
>>> Later, you’ll need the supporting RPC classes to send messages from service driver to device driver…
>>>
>>>
>>> Regards,
>>>
>>>
>>> PCM (Paul Michali)
>>>
>>> MAIL …..…. pcm at cisco.com
>>> IRC ……..… pcm_ (irc.freenode.com)
>>> TW ………... @pmichali
>>> GPG Key … 4525ECC253E31A83
>>> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>>>
>>>
>>>
>>> On Jul 17, 2014, at 2:18 PM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
>>>
>>>> We have followed your advices:
>>>>
>>>> - We created our fake device driver located in the same level as other device drivers (/opt/stack/neutron/neutron/services/vpn//device_drivers/fake_device_driver.py):
>>>>
>>>> import abc
>>>> import six
>>>>
>>>> from neutron.openstack.common import log
>>>> from neutron.services.vpn import device_drivers
>>>>
>>>>
>>>> LOG = log.getLogger(__name__)
>>>>
>>>> @six.add_metaclass(abc.ABCMeta)
>>>> class FakeDeviceDriver(device_drivers.DeviceDriver):
>>>> '''
>>>> classdocs
>>>> '''
>>>>
>>>> def __init__(self, agent, host):
>>>> pass
>>>>
>>>> def sync(self, context, processes):
>>>> pass
>>>>
>>>> def create_router(self, process_id):
>>>> pass
>>>>
>>>> def destroy_router(self, process_id):
>>>> pass
>>>>
>>>> - Our service driver located in /opt/stack/neutron/neutron/services/vpn/service_drivers/fake_service_driver.py:
>>>>
>>>> from neutron.openstack.common import log
>>>>
>>>> LOG = log.getLogger(__name__)
>>>>
>>>> class FakeServiceDriver():
>>>> '''
>>>> classdocs
>>>> '''
>>>>
>>>> def get_vpnservices(self, context, filters=None, fields=None):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def get_vpnservice(self, context, vpnservice_id, fields=None):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def create_vpnservice(self, context, vpnservice):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def update_vpnservice(self, context, vpnservice_id, vpnservice):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def delete_vpnservice(self, context, vpnservice_id):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def get_ipsec_site_connections(self, context, filters=None, fields=None):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def get_ipsec_site_connection(self, context,
>>>> ipsecsite_conn_id, fields=None):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def get_ikepolicy(self, context, ikepolicy_id, fields=None):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def get_ikepolicies(self, context, filters=None, fields=None):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def create_ikepolicy(self, context, ikepolicy):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def update_ikepolicy(self, context, ikepolicy_id, ikepolicy):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def delete_ikepolicy(self, context, ikepolicy_id):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def get_ipsecpolicies(self, context, filters=None, fields=None):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def get_ipsecpolicy(self, context, ipsecpolicy_id, fields=None):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def create_ipsecpolicy(self, context, ipsecpolicy):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def update_ipsecpolicy(self, context, ipsecpolicy_id, ipsecpolicy):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>> def delete_ipsecpolicy(self, context, ipsecpolicy_id):
>>>> LOG.info('XXXXXXXXXXXXXX Calling method: ' + __name__)
>>>> pass
>>>>
>>>>
>>>> - Our /etc/neutron/vpn_agent.ini:
>>>>
>>>> [DEFAULT]
>>>> # VPN-Agent configuration file
>>>> # Note vpn-agent inherits l3-agent, so you can use configs on l3-agent also
>>>>
>>>> [vpnagent]
>>>> # vpn device drivers which vpn agent will use
>>>> # If we want to use multiple drivers, we need to define this option multiple times.
>>>> # vpn_device_driver=neutron.services.vpn.device_drivers.ipsec.OpenSwanDriver
>>>> # vpn_device_driver=neutron.services.vpn.device_drivers.cisco_ipsec.CiscoCsrIPsecDriver
>>>> # vpn_device_driver=another_driver
>>>>
>>>> # custom config
>>>> # implementation location: /opt/stack/neutron/neutron/services/vpn//device_drivers/fake_device_driver.py
>>>> vpn_device_driver=neutron.services.vpn.device_drivers.fake_device_driver.FakeDeviceDriver
>>>>
>>>> [ipsec]
>>>> # Status check interval
>>>> # ipsec_status_check_interval=60
>>>>
>>>>
>>>> It seems now everything is working and q-vpn starts. In one line in his log we see:
>>>>
>>>> 2014-07-16 21:59:45.009 DEBUG neutron.openstack.common.service [req-fb6ed9ca-0e71-4783-804b-81ea34b16679 None None] service_providers.service_provider = ['VPN:fake_junos_vpnaas:neutron.services.vpn.service_drivers.fake_service_driver.FakeServiceDriver:default'] from (pid=14423) log_opt_values /usr/local/lib/python2.7/dist-packages/oslo/config/cfg.py:1988
>>>>
>>>> But now we don't know how to continue. We don't any of our logs in q-vpn when we execute commands like:
>>>>
>>>> neutron vpn-ipsecpolicy-create test-ike-policy
>>>> neutron vpn-ikepolicy-list
>>>> neutron vpn-service-list
>>>>
>>>> We don't see any error anyway.
>>>>
>>>> How we could proceed?
>>>>
>>>> Thank you.
>>>>
>>>>
>>>> Julio C. Barrera Juez
>>>> Office phone: (+34) 93 357 99 27 (ext. 527)
>>>> Office mobile phone: (+34) 625 66 77 26
>>>> Distributed Applications and Networks Area (DANA)
>>>> i2CAT Foundation, Barcelona
>>>>
>>>>
>>>> On 17 July 2014 14:18, Paul Michali (pcm) <pcm at cisco.com> wrote:
>>>> See line @PCM
>>>>
>>>>
>>>> PCM (Paul Michali)
>>>>
>>>> MAIL …..…. pcm at cisco.com
>>>> IRC ……..… pcm_ (irc.freenode.com)
>>>> TW ………... @pmichali
>>>> GPG Key … 4525ECC253E31A83
>>>> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>>>>
>>>>
>>>>
>>>> On Jul 17, 2014, at 6:32 AM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
>>>>
>>>>> I have __init__.py in the directory. Sorry my code is not public, but I can show you some contents, anyway is an experiment with no functional code.
>>>>
>>>> @PCM Could you provide a patch with the files so we could patch it into a local repo and try things? I’m assuming since it is an experiment with no functional code that maybe there’s nothing proprietary? :)
>>>>
>>>>
>>>>
>>>>>
>>>>> My /etc/neutron/vpn_agent.ini:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> [DEFAULT]
>>>>>
>>>>> [vpnagent]
>>>>> # implementation location: /opt/stack/neutron/neutron/services/vpn/junos_vpnaas/device_drivers/fake_device_driver.py
>>>>> vpn_device_driver=neutron.services.vpn.junos_vpnaas.device_drivers.fake_device_driver.FakeDeviceDriver
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> @PCM Hmmm… Just a wild guess... I’m wondering if this is the issue. You class would need to inherit from the base device driver class. Does your fake_device_driver.py have the correct import paths? I say that, because your hierarchy is different. For example, the layout currently is…
>>>>
>>>> neutron/services/vpn/ - plugin
>>>> neutron/services/vpn/device_drivers/ - reference and Cisco device drivers
>>>> neutron/services/vpn/service_drivers/ - reference and Cisco service drivers
>>>>
>>>> Your hierarchy has another level…
>>>>
>>>> neutron/services/vpn/junos_vpnaas/device_drivers/
>>>>
>>>> I’m wondering if there is some import wrong. For example, the reference device driver has:
>>>>
>>>> from neutron.services.vpn import device_drivers
>>>> …
>>>> @six.add_metaclass(abc.ABCMeta)
>>>> class IPsecDriver(device_drivers.DeviceDriver):
>>>> """VPN Device Driver for IPSec.
>>>>
>>>> Where the import is used to access the base class DeviceDriver. If you’re doing the same, that file may be failing to load.
>>>>
>>>> Regards,
>>>>
>>>> PCM
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> FakeDeviceDriver is an empty class with a constructor located in file /opt/stack/neutron/neutron/services/vpn/junos_vpnaas/device_drivers/fake_device_driver.py.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I don't have access to my devstask instance, but the error was produced in q-vpn service:
>>>>> DeviceDriverImportError: Can not load driver :neutron.services.vpn.junos_vpnaas.device_drivers.fake_device_driver.FakeDeviceDriver
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> I can provide full stack this afternoon.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Thank you.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Julio C. Barrera Juez
>>>>> Office phone: (+34) 93 357 99 27 (ext. 527)
>>>>> Office mobile phone: (+34) 625 66 77 26
>>>>> Distributed Applications and Networks Area (DANA)
>>>>> i2CAT Foundation, Barcelona
>>>>>
>>>>>
>>>>> On 16 July 2014 20:59, Paul Michali (pcm) <pcm at cisco.com> wrote:
>>>>> Do you have a repo with the code that is visible to the public?
>>>>>
>>>>> What does the /etc/neutron/vpn_agent.ini look like?
>>>>>
>>>>> Can you put the log output of the actual error messages seen?
>>>>>
>>>>> Regards,
>>>>>
>>>>> PCM (Paul Michali)
>>>>>
>>>>> MAIL …..…. pcm at cisco.com
>>>>> IRC ……..… pcm_ (irc.freenode.com)
>>>>> TW ………... @pmichali
>>>>> GPG Key … 4525ECC253E31A83
>>>>> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>>>>>
>>>>>
>>>>>
>>>>> On Jul 16, 2014, at 2:43 PM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
>>>>>
>>>>>> I am fighting with this for months. I want to develop a VPN Neutron plugin, but it is almost impossible to realize how to achieve it. this is a thread I opened months ago and Paul Mchali helped me a lot: http://lists.openstack.org/pipermail/openstack-dev/2014-February/028389.html
>>>>>>
>>>>>> I want to know the minimum requirements to develop a device driver and a service driver for a VPN Neutron plugin. I tried adding an empty device driver and I got this error:
>>>>>>
>>>>>> DeviceDriverImportError: Can not load driver :neutron.services.vpn.junos_vpnaas.device_drivers.fake_device_driver.FakeDeviceDriver
>>>>>>
>>>>>> Both Python file and class exists, but the implementation is empty. What is the problem? What I need to include in this file/class to avoid this error?
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>>
>>>>>> Julio C. Barrera Juez
>>>>>> Office phone: (+34) 93 357 99 27 (ext. 527)
>>>>>> Office mobile phone: (+34) 625 66 77 26
>>>>>> Distributed Applications and Networks Area (DANA)
>>>>>> i2CAT Foundation, Barcelona
>>>>>> _______________________________________________
>>>>>> OpenStack-dev mailing list
>>>>>> OpenStack-dev at lists.openstack.org
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140728/5ec6c166/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140728/5ec6c166/attachment-0001.pgp>
More information about the OpenStack-dev
mailing list