[openstack-dev] How to implement and configure a new Neutron vpnaas driver from scratch?
Paul Michali (pcm)
pcm at cisco.com
Wed May 21 16:19:53 UTC 2014
You have to have your service driver specified (as default) in neutron.conf, and your device_driver specified in vpn_agent.ini. For the former, you can either modify it after the devstack run and then restart the server, or you can tweak the devstack scripts to modify neutron.conf and add in the service driver. For the device driver, you can modify it in /opt/stack/neutron/etc/vpn_agent.ini and it’ll get copied to /etc/neutron/ with the change.
The log below only shows the reference service driver:
2014-05-21 08:28:46.703 [00;32mDEBUG neutron.service [[00;36m-[00;32m] [01;35m[00;32mservice_providers.service_provider = ['LOADBALANCER:Haproxy:neutron.services.loadbalancer.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default', 'VPN:openswan:neutron.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default'][00m [00;33mfrom (pid=2199) log_opt_values /usr/local/lib/python2.7/dist-packages/oslo/config/cfg.py:1960[00m
PCM (Paul Michali)
MAIL …..…. pcm at cisco.com
IRC ……..… pcm_ (irc.freenode.com)
TW ………... @pmichali
GPG Key … 4525ECC253E31A83
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
On May 21, 2014, at 11:43 AM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
> First of all, excuse me for the silent days. I have made some changes in my devstack instance. I took current master branch of devstack repository and applied your patch to it with my plugin files defined (see it in GitHub https://github.com/logoff/devstack/tree/junos-vpnaas). I used a localrc file like this: http://pastebin.com/dE5bYkTE and when I execute this command, it works:
>
> neutron subnet-list
> +--------------------------------------+----------------+---------------+------------------------------------------------+
> | id | name | cidr | allocation_pools |
> +--------------------------------------+----------------+---------------+------------------------------------------------+
> | 7f8e23ef-be53-423e-8fb9-ea6b4f2b5d6b | private-subnet | 10.254.1.0/24 | {"start": "10.254.1.2", "end": "10.254.1.254"} |
> | f3b85fcc-1c01-4ac8-9950-1f45157c6887 | public-subnet | 172.24.4.0/24 | {"start": "172.24.4.2", "end": "172.24.4.254"} |
> +--------------------------------------+----------------+---------------+------------------------------------------------+
>
> but when I execute this command, it fails:
>
> neutron vpn-service-list
> 404 Not Found
>
> The resource could not be found.
>
>
> Here you can find my q-svc log file (it is big, only 10 minutes of execution: http://pastebin.com/raw.php?i=hqBzg8ED
> I can not see my plugin anywhere. What do I need to do to achieve adding my single provider VPNaaS plugin to a devstack instance?
>
>
> Julio C. Barrera Juez
> Office phone: +34 93 357 99 27
> Distributed Applications and Networks Area (DANA)
> i2CAT Foundation, Barcelona, Spain
> http://dana.i2cat.net
>
>
> On 9 May 2014 14:33, Paul Michali (pcm) <pcm at cisco.com> wrote:
> Not really sure from the log snippet. Can you make available the full log (not sure where you can post it)?
>
> The q-svc log should indicate that your service driver was found and loaded.
>
>
> PCM (Paul Michali)
>
> MAIL …..…. pcm at cisco.com
> IRC ……..… pcm_ (irc.freenode.com)
> TW ………... @pmichali
> GPG Key … 4525ECC253E31A83
> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>
>
>
> On May 6, 2014, at 3:13 PM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
>
>> Hi!
>>
>> First of all thnak you for your guidance.
>>
>> I have followed your instructions with Cisco CSR VPN plugin and my not finished plugin. I got same results on both cases
>> I invoked this command on CLI:
>>
>> neutron vpn-service-list
>>
>> and the result was:
>>
>> 404 Not Found
>>
>> The resource could not be found.
>>
>> Looking in the log of q-svc I see:
>>
>>
>> 02014-05-06 12:05:04.119 INFO neutron.wsgi [req-8e9b50d1-4f6a-4f32-aa3d-db792ba744d5 admin 1798cd11ee304907aa463e6fea7a09bb] (3743) accepted ('10.0.1.10', 52672)
>>
>> 2014-05-06 12:05:04.119 DEBUG keystoneclient.middleware.auth_token [-] Authenticating user token from (pid=3743) __call__ /opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py:603
>> 2014-05-06 12:05:04.120 DEBUG keystoneclient.middleware.auth_token [-] Removing headers from request environment: X-Identity-Status,X-Domain-Id,X-Domain-Name,X-Project-Id,X-Project-Name,X-Project-Domain-Id,X-Project-Domain-Name,X-User-Id,X-User-Name,X-User-Domain-Id,X-User-Domain-Name,X-Roles,X-Service-Catalog,X-User,X-Tenant-Id,X-Tenant-Name,X-Tenant,X-Role from (pid=3743) _remove_auth_headers /opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py:662
>> 2014-05-06 12:05:04.138 DEBUG keystoneclient.middleware.auth_token [-] Storing token in cache from (pid=3743) _cache_put /opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py:1121
>> 2014-05-06 12:05:04.139 DEBUG keystoneclient.middleware.auth_token [-] Received request from user: cbf59516f8c64a10966b0df843a58608 with project_id : 1798cd11ee304907aa463e6fea7a09bb and roles: admin from (pid=3743) _build_user_headers /opt/stack/python-keystoneclient/keystoneclient/middleware/auth_token.py:910
>> 2014-05-06 12:05:04.140 DEBUG routes.middleware [-] No route matched for GET /vpn/vpnservices.json from (pid=3743) __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:97
>> 2014-05-06 12:05:04.140 DEBUG routes.middleware [-] No route matched for GET /vpn/vpnservices.json from (pid=3743) __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:97
>> 2014-05-06 12:05:04.148 INFO neutron.wsgi [req-481d6f8b-c0f4-4862-965f-a0468c37a582 admin 1798cd11ee304907aa463e6fea7a09bb] 10.0.1.10 - - [06/May/2014 12:05:04] "GET /v2.0/vpn/vpnservices.json HTTP/1.1" 404 176 0.028534
>>
>> I don't know why the service was "Not found".
>>
>> May you give me some help to solve this problem?
>>
>> Thank you.
>>
>>
>>
>> Julio C. Barrera Juez
>> Office phone: +34 93 357 99 27
>> Distributed Applications and Networks Area (DANA)
>> i2CAT Foundation, Barcelona, Spain
>> http://dana.i2cat.net
>>
>>
>> On 28 April 2014 14:10, Paul Michali (pcm) <pcm at cisco.com> wrote:
>>
>> On Apr 26, 2014, at 7:39 AM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
>>
>>> I'm trying to configure any VPNaaS plugin in single-provider mode. I'm not able to achieve this goal. I'm using a devstack installation and I'm editing /etc/neutron/neutron.conf file, modifying this line:
>>>
>>> ...
>>> service_provider=VPN:cisco_csr:neutron.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default
>>> ...
>>>
>>> and /etc/neutron/vpn_agent.ini modifyin gthis line:
>>>
>>> ...
>>> vpn_device_driver=neutron.services.vpn.device_drivers.ipsec.IPsecDriver…
>>
>> PCM: So what are you modifying these lines to? Are they pointing to valid modules?
>>
>>
>>>
>>> I'm not sure if this configuration is OK. I have some doubts:
>>>
>>> - Is this configuration a valid one taking into account that plugin are available in Python modules path?
>>
>> PCM: Sorry, I don’t understand what you’re asking here.
>>
>>
>>> - Where are the log files located to check valid neutron configuration?
>>
>> PCM: There is a variable in DevStacks localrc to set where the logs are placed. For example:
>>
>> SCREEN_LOGDIR=/opt/stack/screen-logs
>>
>> I don’t know what the default is (disabled?).
>>
>>
>>> - What services should I restart each time I change this configuration?
>>
>> PCM: q-svc for the service_driver, and q-vpn for the device_driver (and maybe q-aft?).
>>
>>
>> What I do, is modify vpn_agent.ini in /opt/stack/neutron/etc/ and then, using a newer DevStack that has my commit for VPN (https://review.openstack.org/#/c/86567/), /etc/neutron/vpn_agent.ini will be set with the desired device driver and that will be loaded at start up.
>>
>> Also, I do a patch to DevStack’s lib/neutron and lib/neutron_plugins/services/vpn to setup neutron.conf as well, so that, again, /etc/neutron/neutron.conf is set up and stack.sh will do the right thing. The patch is:
>>
>> patch -p 1 << EOT
>> diff --git a/lib/neutron b/lib/neutron
>> index 02dcaf6..452281b 100644
>> --- a/lib/neutron
>> +++ b/lib/neutron
>> @@ -728,6 +728,7 @@ function _configure_neutron_fwaas {
>> function _configure_neutron_vpn {
>> neutron_vpn_install_agent_packages
>> neutron_vpn_configure_common
>> + neutron_vpnaas_configure_driver
>> }
>>
>> # _configure_neutron_plugin_agent() - Set config files for neutron plugin agent
>> diff --git a/lib/neutron_plugins/services/vpn b/lib/neutron_plugins/services/vpn
>> index d920ba6..a676fdc 100644
>> --- a/lib/neutron_plugins/services/vpn
>> +++ b/lib/neutron_plugins/services/vpn
>> @@ -18,6 +18,10 @@ function neutron_vpn_configure_common {
>> _neutron_service_plugin_class_add $VPN_PLUGIN
>> }
>>
>> +function neutron_vpnaas_configure_driver() {
>> + iniset_multiline $NEUTRON_CONF service_providers service_provider "VPN:cisco:neutron.services.vpn.service_drivers.cisco_ipsec.CiscoCsrIPsecVPNDriver:default"
>> +}
>> +
>> function neutron_vpn_stop {
>> local ipsec_data_dir=$DATA_DIR/neutron/ipsec
>> local pids
>> EOT
>>
>>
>> Regards,
>>
>>
>> PCM (Paul Michali)
>>
>> MAIL …..…. pcm at cisco.com
>> IRC ……..… pcm_ (irc.freenode.com)
>> TW ………... @pmichali
>> GPG Key … 4525ECC253E31A83
>> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>>
>>
>>>
>>> Thank you very much.
>>>
>>>
>>> Julio C. Barrera Juez
>>> Office phone: +34 93 357 99 27
>>> Distributed Applications and Networks Area (DANA)
>>> i2CAT Foundation, Barcelona, Spain
>>> http://dana.i2cat.net
>>>
>>>
>>> On 24 April 2014 16:14, Paul Michali (pcm) <pcm at cisco.com> wrote:
>>> Not sure I quite understand the question, but to configuring VPNaaS in single provider mode, from a user’s perspective is the same (see api.openstack.org).
>>>
>>> To bring up a cloud that uses a different vendor’s service and device driver, you need to modify neutron.conf to select the vendor’s service driver (as the default driver), instead of the reference driver, and in vpn_agent.ini you select the vendor’s device driver (instead of or in addition to the reference implementation, doesn’t matter, as it pairs with the service driver).
>>>
>>> HTHs,
>>>
>>>
>>> PCM (Paul Michali)
>>>
>>> MAIL …..…. pcm at cisco.com
>>> IRC ……..… pcm_ (irc.freenode.com)
>>> TW ………... @pmichali
>>> GPG Key … 4525ECC253E31A83
>>> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>>>
>>>
>>>
>>> On Apr 24, 2014, at 3:13 AM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
>>>
>>>> OK, thank you guys, I understood that it was not possible to configure and make work any VPNaaS plugin. I don't care, by now, because it works in single-provider mode. I knew about the Cisco implementation, but I don't know how to configure it, because I didn't find enough documentation about that topic. I need some help on the basics configuring a VPNaaS plugin in single provider mode, because I only found information about it in 3rd party blog posts, etc.
>>>>
>>>> What are the basic steps?
>>>>
>>>> Thank you again.
>>>>
>>>>
>>>> Julio C. Barrera Juez
>>>> Office phone: +34 93 357 99 27
>>>> Distributed Applications and Networks Area (DANA)
>>>> i2CAT Foundation, Barcelona, Spain
>>>> http://dana.i2cat.net
>>>>
>>>>
>>>> On 18 April 2014 10:50, Bo Lin <linb at vmware.com> wrote:
>>>> Hi Julio,
>>>> +1 for Paul's response. Multiple-provider VPNaaS support is delayed. But you can take https://review.openstack.org/#/c/74156/ and https://review.openstack.org/#/c/74144/ as examples to write your own vpnaas driver without multi-provider support. If any questions or problems in your codes leading to not work, just upload your codes onto the review board, we can find how to solve it :).
>>>>
>>>> Thanks!
>>>> ---Bo
>>>>
>>>>
>>>> From: "Paul Michali (pcm)" <pcm at cisco.com>
>>>>
>>>> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
>>>> Sent: Friday, April 11, 2014 2:15:18 AM
>>>>
>>>> Subject: Re: [openstack-dev] How to implement and configure a new Neutron vpnaas driver from scratch?
>>>>
>>>> By not “working” do you mean you cannot get the plugin to work in a multi-provider environment? Multi-provider solutions have been tabled until Juno, where more discussion is occurring on what is the best way to support different service providers.
>>>>
>>>> However, you should be able to get the plugin to work as the “sole” VPN service provider, which is what the Cisco solution does currently. You can look at how I’ve done that in the cisco_ipsec.py modules in the service_drivers and device_drivers directories, under neutron/services/vpn/.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> PCM (Paul Michali)
>>>>
>>>> MAIL …..…. pcm at cisco.com
>>>> IRC ……..… pcm_ (irc.freenode.com)
>>>> TW ………... @pmichali
>>>> GPG Key … 4525ECC253E31A83
>>>> Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83
>>>>
>>>>
>>>>
>>>> On Apr 10, 2014, at 1:51 PM, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
>>>>
>>>> Hi.
>>>>
>>>> After 8 months of the patch creation and being abandoned weeks ago (https://review.openstack.org/#/c/41827/) I still don't how can we develop a VPNaaS plugin following Bo Lin instructions. Is there any other patch trying to solve the problem? Is there any way to workaround the issue to get a VPNaaS plugin working?
>>>>
>>>> Thank you!
>>>>
>>>>
>>>> Julio C. Barrera Juez
>>>> Office phone: +34 93 357 99 27
>>>> Distributed Applications and Networks Area (DANA)
>>>> i2CAT Foundation, Barcelona, Spain
>>>> http://dana.i2cat.net
>>>>
>>>>
>>>> On 27 February 2014 10:51, Bo Lin <linb at vmware.com> wrote:
>>>> Hi Julio,
>>>> You can take https://review.openstack.org/#/c/74156/ and https://review.openstack.org/#/c/74144/ as examples to write your own vpnaas driver. More info about service type framework, you can also refer to neutron/services/loadbalancer codes.
>>>>
>>>> From: "Julio Carlos Barrera Juez" <juliocarlos.barrera at i2cat.net>
>>>> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org>
>>>> Sent: Thursday, February 27, 2014 5:26:32 PM
>>>> Subject: Re: [openstack-dev] How to implement and configure a new Neutron vpnaas driver from scratch?
>>>>
>>>>
>>>> I'm following the change you pointed a week ago. It seems that it is working now, and will be eventually approved soon. I will be happy when it is approved.
>>>>
>>>> Anyway, I need more information about how to develop a service driver and a device driver for VPN plugin. I realize doing reverse-engineering that I need and RPC agent and and RPC between them to communicate and use a kind of callbacks to answer. Where I can find documentation about it and some examples? Is there any best practise guide of the use of this architecture?
>>>>
>>>> Thank you again!
>>>>
>>>>
>>>> Julio C. Barrera Juez
>>>> Office phone: +34 93 357 99 27
>>>> Distributed Applications and Networks Area (DANA)
>>>> i2CAT Foundation, Barcelona, Spain
>>>> http://dana.i2cat.net
>>>>
>>>>
>>>> On 19 February 2014 09:13, Julio Carlos Barrera Juez <juliocarlos.barrera at i2cat.net> wrote:
>>>> Thank you very much Bo. I will try all your advices and check if it works!
>>>>
>>>>
>>>> Julio C. Barrera Juez
>>>> Office phone: +34 93 357 99 27
>>>> Distributed Applications and Networks Area (DANA)
>>>> i2CAT Foundation, Barcelona, Spain
>>>> http://dana.i2cat.net
>>>>
>>>>
>>>> On 18 February 2014 09:18, Bo Lin <linb at vmware.com> wrote:
>>>> I wonder whether your neutron server codes have added the " VPNaaS integration with service type framework" change on https://review.openstack.org/#/c/41827/21 , if not, the service_provider option is useless. You need to include the change before developing your own driver.
>>>>
>>>> QA (In my opinion and sth may be missing):
>>>> - What is the difference between service drivers and device drivers?
>>>> service drivers are driven by vpn service plugin and are responsible for casting rpc request (CRUD of vpnservices) to and do callbacks from vpn agent.
>>>> device drivers are driven by vpn agent and are responsible for implementing specific vpn operations and report vpn running status.
>>>>
>>>> - Could I implement only one of them?
>>>> device driver must be implemented based on your own device. Unless the default ipsec service driver is definitely appropriate, suggest you implement both of them. After including "VPNaaS integration with service type framework", the service driver work is simple.
>>>>
>>>> - Where I need to put my Python implementation in my OpenStack instance?
>>>> Do you mean let your instance runs your new codes? The default source codes dir is /opt/stack/neutron, you need to put your new changes into the dir and restart the neutron server.
>>>>
>>>> - How could I configure my OpenStack instance to use this implementation?
>>>> 1. Add your new codes into source dir
>>>> 2. Add appropriate vpnaas service_provider into neutron.conf and add appropriate "vpn_device_driver" option into vpn_agent.ini
>>>> 3. restart n-svc and q-vpn
>>>>
>>>> Hope help you.
>>>>
>>>> From: "Julio Carlos Barrera Juez" <juliocarlos.barrera at i2cat.net>
>>>> To: "OpenStack Development Mailing List" <openstack-dev at lists.openstack.org>
>>>> Sent: Monday, February 17, 2014 7:18:44 PM
>>>> Subject: [openstack-dev] How to implement and configure a new Neutron vpnaas driver from scratch?
>>>>
>>>>
>>>> Hi.
>>>>
>>>> I have asked in the Q&A website without success (https://ask.openstack.org/en/question/12072/how-to-implement-and-configure-a-new-vpnaas-driver-from-scratch/).
>>>>
>>>> I want to develop a vpnaas implementation. It seems that since Havana, there are plugins, services and device implementations. I like the plugin and his current API, then I don't need to reimplement it. Now I want yo implement a vpnaas driver, and I see I have two main parts to take into account: the service_drivers and the device_drivers. IPsec/OpenSwan implementation is the unique sample I've found.
>>>>
>>>> I'm using devstack to test my experiments.
>>>>
>>>> I tried to implement VpnDriver Python class extending the main API methods like IPsecVPNDriver does. I placed basic implementation files at the same level of IPsec/OpenSwan does and configured Neutron adding this line to /etc/neutron/neutron.conf file:
>>>>
>>>> service_provider = VPN:VPNaaS:neutron.services.vpn.service_drivers.our_python_filename.OurClassName:default
>>>>
>>>> I restarted Neutron related services in my devstack instance, but it seemed it didn't work.
>>>>
>>>>
>>>>
>>>> - What is the difference between service drivers and device drivers?
>>>> - Could I implement only one of them?
>>>> - Where I need to put my Python implementation in my OpenStack instance?
>>>> - How could I configure my OpenStack instance to use this implementation?
>>>>
>>>>
>>>>
>>>> I didn't find almost any documentation about these topics.
>>>>
>>>> Thank you very much.
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> https://urldefense.proofpoint.com/v1/url?u=http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=F5etm0B6kVJ9jleIhCvNyA%3D%3D%0A&m=9uhm%2F59JRfiZ3CXzuhBOpqcTqWk8APswRGJFZ8H2Tos%3D%0A&s=46fe06049efb1d29a85b63f7ce101cd69695a368c3da6ea3a91bcd7d2b71ce59
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> https://urldefense.proofpoint.com/v1/url?u=http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=F5etm0B6kVJ9jleIhCvNyA%3D%3D%0A&m=YmmNfPyv1TNDbHlwFZT9xRPhyBxsQW%2B2aJ3daQ8RC%2BI%3D%0A&s=638a7f219d00817d3d17746251a9b5090cce130fed11727be8a4cabd09754657
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> https://urldefense.proofpoint.com/v1/url?u=http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=F5etm0B6kVJ9jleIhCvNyA%3D%3D%0A&m=1%2FHmRV%2F3ce%2Bjpzxjfyhv6xjuBhiOBVrajFVFZjco9Zw%3D%0A&s=d81bebe644cccedf66fedc084cf34c54e82b5a62712e12d1b5e2c6c1c6ee2c81
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140521/a0d8b882/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140521/a0d8b882/attachment-0001.pgp>
More information about the OpenStack-dev
mailing list