Open Stack

The key point is how can we improve the 'maturity' of SDK of various languages (python, java, c#, etc) and stay up-to-date with the OpenStack API development across different projects (neutron, nova, cinder, etc). Today, OpenStack does not 'own' most of the SDK and has limited influence to those SDK's development roadmap, which is a 'risk' as far as promoting OpenStack App Development is concerned. How can we (as an OpenStack community) improve the collaboration and work closely with those SDK groups?

One reason that libcloud fall short in the cloud-benchmark report is due to nova-api and neutron-api. Libcloud uses nova-api to control security group, while Rackspace only expose neutron-api for security group. The OpenStack Compute API [1] is still showing nova-api as a valid way to control security group. This actually raises another concern on API compatibility between 'OpenStack-Powered' cloud today, which probably need to be addressed by Defcore.

Having said that, today, there are many companies working on 'multi-cloud' solution that run across different clouds (AWS, GCE, OpenStack, etc). I have been developing multi-cloud applications since 2008 for the media, finance and research industry. Multi-Cloud SDK (e.g. libcloud, jcloud) still remain a common choice for developers for many reasons. 

[1] http://developer.openstack.org/api-ref-compute-v2.1.html#listSecGroups


---
Yih Leong Sun, PhD 
Senior Software Cloud Architect
Open Source Technology Center
Intel Corporation
yih.leong.sun at intel.com
+1 503 264 0610



-----Original Message-----
From: Christopher Aedo [mailto:doc at aedo.net] 
Sent: Monday, March 21, 2016 3:53 PM
To: Everett Toews <everett.toews at rackspace.com>
Cc: user-committee at lists.openstack.org
Subject: Re: [User-committee] app dev on-boarding experience report

On Sun, Mar 20, 2016 at 2:25 PM, Everett Toews <everett.toews at rackspace.com> wrote:
> I've held my tongue on this topic for too long and, considering the results with OVH, I realize now that doing so has only been to the detriment of Python developers. The primary conclusion that we can draw from the report is this.
>
> 1. libcloud is the wrong choice of Python SDK for applications designed for OpenStack clouds.
>
> If the primary requirement of your application is to work with multiple clouds or to move between clouds, it's a fine choice. However, if your Python application is designed to run on OpenStack and take advantage of all of its features (such as the First App on OpenStack does), libcloud is the wrong choice.
>
> libcloud only supports cloud abstractions. Features which are common across all clouds. Such abstractions are extremely difficult to develop and are often leaky. If an abstraction isn't supported, the users have to look to another SDK or a CLI and try to cobble something together, as is evidenced in the Networking [1] and Orchestration [2] section of the First App guide. This is a horrible developer experience.
>
> 2. Recommending libcloud to users to write applications designed for OpenStack clouds is wrong.
>
> If users are encouraged to use libcloud, they will not be able to take full advantage of OpenStack's many features. They will be limited in what they can do in their applications and will forever be trying to piece together an incomplete puzzle. Adding in pieces from other SDKs or CLIs will only make their lives more difficult and hurt the perception of developer experience in OpenStack. There are better options to recommend like Shade [3] or the official Python OpenStack SDK [4].
>
> 3. libcloud should be removed from developer.openstack.org once Shade or the official Python OpenStack SDK have a First App guide.
>
> Before you say, "someone can just do the development to add the abstractions to libcloud" you need to answer these questions. What is the process for getting an abstraction added to libcloud? Does the libcloud project even want that abstraction? How much effort will it be? Is it even possible considering a particular OpenStack feature/project might not be abstractable? Exactly who is that someone who will do the development? Who will maintain it over time?
>
> I strongly believe that continuing to recommend libcloud for applications designed for OpenStack is the wrong direction for developer experience in OpenStack. You're effectively sending developers down a deadend.

I agree with most of your points here regarding libcloud being a poor choice for someone developing an app intended to be run on an OpenStack cloud, but we will gain more users and better OpenStack adoption if we aim to bring in users from other clouds as well.  It makes sense for the user committee to try to attract "cloud developers" rather than "OpenStack cloud developers".  In that regard there's room for multiple guides here.

The Shade version of the first-app tutorial[1]  is basically done.
I'm not sure what's left before it graduates from draft status but at this point that's what I would point someone at if they wanted an example of the best way to write something targeted for an OpenStack cloud.  For better or worse actually, Shade does a good job of hiding some of the little things that aren't exactly compatible from one cloud to the next (like clouds that have their own take on handling networking or identity).

There's a lot of value in having the libcloud version around as well, and I honestly think it benefits the community to offer more options (even if they're not perfect) rather than limiting what we present to the rest of the world.

[1]: http://developer.openstack.org/draft/firstapp-shade/

-Christopher

> Regards,
> Everett
>
> [1] http://developer.openstack.org/firstapp-libcloud/networking.html
> [2] 
> http://developer.openstack.org/firstapp-libcloud/orchestration.html
> [3] http://docs.openstack.org/infra/shade/
> [4] http://developer.openstack.org/sdks/python/openstacksdk/
>
>
>> On Mar 16, 2016, at 6:34 PM, Bonell Manjarrez, Marcela <marcela.bonell.manjarrez at intel.com> wrote:
>>
>> Hi all,
>>
>> I finally completed the developer experience report!
>> The updated version include OVH public cloud, so now for OpenStack we have RAX and OVH against AWS, Azure and Google Cloud.
>>
>> The FirstApp was deployed successfully over OVH using shade, libcloud also fails the same way it does in RAX (nova-network).
>> One of the most interesting things about using OVH public cloud is that it provides its own Cloud Control Panel (which is very interactive) and it also provides Horizon.
>>
>> Please take a look in the v3 version:
>> [Report] 
>> https://drive.google.com/open?id=19AgySEkzzYihlAh4MCNmRcqQEHvbTctBDfR
>> nMuITPds [Script to reproduce] 
>> https://drive.google.com/open?id=1vlwQaQyIcXwdJEaEGqbAvdGkilkiFmlHMCf
>> IMqULG9Y
>>
>> I will talk about this cloud comparison in Austin ;)
>> https://www.openstack.org/summit/austin-2016/summit-schedule/events/7
>> 057
>>
>> Thanks Christopher for the OVH account, it gave me the opportunity to 
>> complete the report :)
>>
>> Marcela Bonell.
>>
>> On 2/29/16, 5:04 PM, "Bonell Manjarrez, Marcela" <marcela.bonell.manjarrez at intel.com> wrote:
>>
>>> Hello,
>>>
>>> I have updated the developer experience report adding TryStack as cloud environment. The report has been completed, the OpenStack 1stApp was deployed successfully on TryStack (shade sdk)!
>>> Please take a look, specially in the “Experience satisfaction rate per provider” and “Conclusion” slides.
>>> [Report]https://drive.google.com/open?id=1eIXND7T3MvrBY-FGnCdJiqFeDZ
>>> ew1-M3Kzz2EXtvR34
>>>
>>> Important note:
>>> "TryStack is a testing environment by design. Instances are typically deleted after 24 hours, so for obvious reasons you should NOT put any data on this cloud you don't want to lose.”
>>>
>>> In the following threads you can find a detailed explanation about why libcloud/shade sdk did/didn’t work on RAX/TryStack:
>>> http://lists.openstack.org/pipermail/user-committee/2016-February/00
>>> 0662.html - TryStack 
>>> http://lists.openstack.org/pipermail/user-committee/2016-January/000
>>> 636.html - RAX
>>>
>>> Also, as part of this activity, I filed 4 bugs in the shade “getting 
>>> started” section. 1 was merged and I’m working on patches for the 
>>> rest ;)
>>>
>>> What’s next?
>>> - Devstack
>>> - OVH
>>>
>>> Marcela Bonell.
>>>
>>>
>>> On 2/8/16, 3:59 PM, "Bonell Manjarrez, Marcela" <marcela.bonell.manjarrez at intel.com> wrote:
>>>
>>>> Hi folks,
>>>>
>>>> Good news, I'm able to deploy the FirstApp (getting started script) in Trystack [1] with shade!
>>>>
>>>> First, I tried with libcloud without success, because libcloud has problems with networking (security groups).
>>>> Then I tried with shade and everything worked fine!
>>>>
>>>> The pre-work required to deploy the app is:
>>>>
>>>> * Generate an API password (Settings tab)
>>>> * Create an internal network
>>>> * Create a router
>>>> * Connect the internal and public networks with the router
>>>>
>>>> All these steps are well documented in a video[2] that is accessible from Trystack horizon login page [3].
>>>>
>>>> Despite the fact that Trystack is for testing purposes only (your instances are available just for 1-3 days), it can be used for training or to explore app development such as the FirstApp tutorial.
>>>>
>>>> [1] http://trystack.org/
>>>> [2] https://www.youtube.com/watch?v=z-M5Vt4-HYg
>>>> [3] https://x86.trystack.org/dashboard/auth/login/?next=/dashboard/
>>>>
>>>> Marcela Bonell.
>>>> _______________________________________________
>>>> User-committee mailing list
>>>> User-committee at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee
>>>>
>>>
>>>
>>> On 1/25/16, 1:27 PM, "Bonell Manjarrez, Marcela" <marcela.bonell.manjarrez at intel.com> wrote:
>>>
>>>> Hi folks,
>>>>
>>>> Continuing with the developer experience report, these are my findings using shade in RAX cloud:
>>>>
>>>> - Connect with OpenStack is easier, using  ~/.config/openstack/clouds.yml also the profiles for public clouds are very useful because is a guide for how to construct your connection and the services available for that provider [2].
>>>> - Shade has similar methods names that libcloud, but shade simplify 
>>>> some instructions such as: connection with the cloud, searches, 
>>>> floating IPs…
>>>> - I used the RAX profile and everything  was fine until the security group creation, now it’s not because the SDK (actually shade support nova-network and neutron) the problem is my provider (RAX):
>>>>
>>>> * When I try to create a security group I get this error message:
>>>>
>>>> Traceback (most recent call last):
>>>> File "/usr/local/lib/python2.7/dist-packages/shade/openstackcloud.py", line 1102, in list_security_groups
>>>>   "Unavailable feature: security groups"
>>>> shade.exc.OpenStackCloudUnavailableFeature: Unavailable feature: 
>>>> security groups
>>>>
>>>> As you can see in the rackspace.json[3] the “has_network” attribute is false, so shade disabled this feature in the RAX profile… Monty mentioned: "Rackspace puts neutron in the service catalog but it does not work (404s the endpoint)” [4]. I overwrote my RAX client config to force to use nova or neutron and as Monty said I got the 404 error...
>>>>
>>>>   my-rackspace:
>>>>     profile: rackspace
>>>>     auth:
>>>>                 username: XXX
>>>>                 password: XXX
>>>>           project_id: XXX
>>>>    region_name: 'DFW'
>>>>    secgroup_source: 'neutron'
>>>>
>>>> Error with neutron:
>>>> shade.exc.OpenStackCloudURINotFound: Error fetching security group 
>>>> list: 404 Not Found The resource could not be found.
>>>>
>>>> Error with nova:
>>>> shade.exc.OpenStackCloudException: Error fetching security group 
>>>> list (Inner Exception: Not found (HTTP 404) (Request-ID: 
>>>> req-239ab188-6c7c-45d1-9600-8567c26874e3))
>>>>
>>>> Also when Monty presented shade in the latest summit, he mentioned 
>>>> about it[5]
>>>>
>>>>
>>>> Well these are my findings for now.., I followed the FirstApp 
>>>> (Draft for shade) and as Stefano mentioned the draft have to be published at least the completed section. For me the “Getting started” section[6] is almost ready, I just need to fix the bug I found in the documentation[7] :) I’ll try TryStack to avoid the RAX problems but I haven’t receive my account authorization yet (manual approval step).
>>>>
>>>>
>>>> [1] http://git.openstack.org/cgit/openstack/os-client-config
>>>> [2] 
>>>> http://git.openstack.org/cgit/openstack/os-client-config/tree/os_cl
>>>> ient_config/vendors [3] 
>>>> http://git.openstack.org/cgit/openstack/os-client-config/tree/os_cl
>>>> ient_config/vendors/rackspace.json
>>>> [4] https://review.openstack.org/#/c/230033/
>>>> [5] 
>>>> https://www.youtube.com/watch?v=_guJmff89HI&feature=youtu.be&t=28m4
>>>> 7s [6] 
>>>> http://developer.openstack.org/draft/firstapp-shade/getting_started
>>>> .html [7] 
>>>> https://bugs.launchpad.net/openstack-api-site/+bug/1537564
>>>>
>>>> Marcela Bonell.
>>>> _______________________________________________
>>>> User-committee mailing list
>>>> User-committee at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee
>>> _______________________________________________
>>> User-committee mailing list
>>> User-committee at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee
>> _______________________________________________
>> User-committee mailing list
>> User-committee at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee
>
> _______________________________________________
> User-committee mailing list
> User-committee at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee

_______________________________________________
User-committee mailing list
User-committee at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee