Open Stack

Hi Gilles, folks,

Nice to read such answers, I’m really thrilled by what could goes out from
this discussion.

One last thing regarding the SDK and Broker part of the discussion.

GraphQL and SDK:

Obviously as you noticed it, I was focused on the python-openstacksdk part
of things even if it apply to the autonomous Openstack4j JAVA SDK or any
other SDK for your favorite language.

I do agree with you, GraphQL being a DSL it should in a long (or maybe not
so long depending the adoption rate ;-) ) run replace the REST part of the
SDK, however, I think the client libraries (at least for the python side of
think) should be enforced by the Openstack foundation/devs as it would
avoid having devs from one project/tool that will join the big tent to use
a different library of its own and so create fragmentation and pitfalls
already mentioned upper on my previous message.

For example, if I let our devs use their own client library for both
GraphQL and workers logic I will end up with at least a dozen of different
libraries per teams and it will be a nightmare to debug, investigate,
maintain etc.

For sure as this is a personal example some could argue that we should
enforce this choice at the company level and not at the solution level, but
if everyone talk the same language its easier to share information, make
consensus around a project, ease the development process by having a clear
and consistent path (Providing a common cookiecutter for all new projects)
and would give us the ability to manage a complete project with the
Openstack client tool such as:

```openstack brick init <project_name>```

Here I choose the “brick” term as a keyword in order to avoid namespace
collisions as projects and services are already used for ops side of things.

GraphQL broker:

Ok I see what you means and I honestly love the idea as it’s an elegant way
to split responsibility while being able to scale and efficiently
distribute requests.

I think that’s the implicit idea behind swift-proxy and how (most of)
companies achieve the horizontal scaling with HAProxy as a loadbalancer in
front of classic Openstack WSGI endpoints.

As this is a builtin feature of GraphQL that would allows a way better
service discovery and routing architecture.

Kind regards,
G.

Le mer. 2 mai 2018 à 09:41, Gilles Dubreuil <gdubreui at redhat.com> a écrit :

> I fixed the GraphQL typo (my mistake) in $subject to help with future ML
> searches.
>
> Please see inline too.
>
> On 02/05/18 07:37, Flint WALRUS wrote:
>
> Ok, here are my two cents regarding GraphQL integration within Openstack
> and some thoughts around this topic.
>
> 1°/- Openstack SDK should still exist and should be in my humble opinion a
> critical focus as it allow following benefits for large and medium
> companies :
>
> • It provide a common and clean structure for Openstack developments and
> should be used either by projects or tools willing to integrate Openstack
> as it will then create some sort of standard.
>
> For instance, here in my job we have A LOT (More than 10 000 peoples
> working within around 130 teams) of teams developing over Openstack using
> the SDK as a common shared base layout.
> That allow for teams to easily share and co-develop on projects. Those
> teams are spread around the world and so need to have clean guidelines as
> it avoid them reinventing the wheel, they’re not stuck with someone else
> obscure code created by another persons on the other side of the world or
> within a different timezone.
> Additionally it streamline our support and debug processes.
>
>
> I'm assuming you're talking about the Python SDK (Shade) which would make
> sense because it's the "lingua franca" of all projects.
>
> Nevertheless, for any SDKs/Languages, if adopted then GraphQL is likely to
> replace its REST SDK on the long run. GraphQL is a DSL bypassing a SDK need
> which get replaced with GraphQL client library. Basically the change, not a
> rewrite, is inevitable. But I insist on "the long run" part, initially both
> in parallel one wrapping the other, then progressively the REST content
> moving across to GraphQL.
>
>
> • We should get a common consensus before all projects start to implement
> it.
>
>
>
> This is going to be raised during the API SIG weekly meeting later this
> week.
> API developers (at least one) from every project are strongly welcomed to
> participate.
> I suppose it makes sense for the API SIG to be the place to discuss it, at
> least initially.
>
>
>
> This point is for me the most important one as it will fix flaws we get
> currently with the rest APIs development within Openstack.
>
> First it will avoid a fresh developer to be confused by too many options.
> Honestly, I know we are open etc, but this point really need to be
> addressed as it is the main issue that I face with Openstack advocacy since
> many years now.
>
> Having too many options even if explained within the documentation daunt a
> lot of people to quickly give a hand with projects.
>
> For instance I have a workmate that is currently working on an internal
> tool which ask me how should he implement its project REST interfaces.
>
> I told him TO NOT use WSME and to stick with what have been done by a
> major project. Unfortunately he choose to copy what have been done by
> Octavia which is actually using... WSME...
>
> GraphQL gives us the opportunity and ability to fix Openstack development
> inconsistencies by providing and enforcing a clean guideline regarding
> which library should be used and in which way.
>
> That would also have the side effect to easy the entry level for a new
> Openstack developer.
>
>
> I couldn't agree more!
>
>
> • New architecture opportunities.
>
> For sure that will bring new architecture opportunities, but the broker
> thing is not a good idea as each project should be able to be autonomous.
>
> I personally don’t like centralized services as it bring SPOF.
>
> Let’s take the AMQP example. For now most of Openstack deployments use a
> RabbitMQ or broker like system.
> Even if each (well at least major vanilla projects) services can (and
> should) use ZeroMQ.
> I do myself use RabbitMQ but my last weeks were so much
> debugging/investigation hell that we now plan to have a serious benchmark
> and test of ZMQ.
>
> One thing that I would love to see with GraphQL is a better distributed
> and traceable model.
>
>
> Exactly and the term broker I used is far from ideal,  I meant it in the
> context of a broker pattern providing distributed API service. GraphQL has
> "stiching" capabilities allowing to forward request to diverse GraphQL
> service, kind of a proxy, ideally such service to be distributed itself.
>
> The idea behind is a GraphQL proxy offering a single point of entry for
> OpenStack entire stack and of course leaving complete autonomy to the all
> services.
>
>
> https://blog.graph.cool/graphql-schema-stitching-explained-schema-delegation-4c6caf468405
>
> Anyway, I’m glad someone started this discussion as I feel it is a really
> important topic that would highly help Openstack on more than just
> interfacing topics.
> Le mar. 1 mai 2018 à 05:00, Gilles Dubreuil <gdubreui at redhat.com> a
> écrit :
>
>>
>>
>> On 01/05/18 11:31, Flint WALRUS wrote:
>>
>> Yes, that’s was indeed the sens of my point.
>>
>>
>> I was just enforcing it, no worries! ;)
>>
>>
>>
>> Openstack have to provide both endpoints type for a while for backward
>> compatibility in order to smooth the transition.
>>
>> For instance, that would be a good idea to contact postman devteam once
>> GraphQL will start to be integrated as it will allow a lot of ops to keep
>> their day to day tools by just having to convert their existing collections
>> of handful requests.
>>
>>
>> Shouldn't we have a common consensus before any project start pushing its
>> own GraphQL wheel?
>>
>> Also I wonder how GraphQL could open new architecture avenues for
>> OpenStack.
>> For example, would that make sense to also have a GraphQL broker linking
>> OpenStack services?
>>
>>
>>
>>
>> Or alternatively to provide a tool with similar features at least.
>> Le mar. 1 mai 2018 à 03:18, Gilles Dubreuil <gdubreui at redhat.com> a
>> écrit :
>>
>>>
>>>
>>> On 30/04/18 20:16, Flint WALRUS wrote:
>>>
>>> I would very much second that question! Indeed it have been one of my
>>> own wondering since many times.
>>>
>>> Of course GraphQL is not intended to replace REST as is and have to live
>>> in parallel
>>>
>>>
>>> Effectively a standard initial architecture is to have GraphQL sitting
>>> aside (in parallel) and wrapping REST and along the way develop GrapgQL
>>> Schema.
>>>
>>> It's seems too early to tell but GraphQL being the next step in API
>>> evolution it might ultimately replace REST.
>>>
>>>
>>> but it would likely and highly accelerate all requests within heavily
>>> loaded environments
>>>
>>>
>>> +1
>>>
>>>
>>> .
>>>
>>> So +1 for this question.
>>> Le lun. 30 avr. 2018 à 05:53, Gilles Dubreuil <gdubreui at redhat.com> a
>>> écrit :
>>>
>>>> Hi,
>>>>
>>>> Remember Boston's Summit presentation [1] about GraphQL [2] and how it
>>>> addresses REST limitations.
>>>> I wonder if any project has been thinking about using GraphQL. I
>>>> haven't
>>>> find any mention or pointers about it.
>>>>
>>>> GraphQL takes a complete different approach compared to REST. So we can
>>>> finally forget about REST API Description languages
>>>> (OpenAPI/Swagger/WSDL/WADL/JSON-API/ETC) and HATEOS (the hypermedia
>>>> approach which doesn't describe how to use it).
>>>>
>>>> So, once passed the point where 'REST vs GraphQL' is like comparing SQL
>>>> and no-SQL DBMS and therefore have different applications, there are no
>>>> doubt the complexity of most OpenStack projects are good candidates for
>>>> GraphQL.
>>>>
>>>> Besides topics such as efficiency, decoupling, no version management
>>>> need there many other powerful features such as API Schema out of the
>>>> box and better automation down that track.
>>>>
>>>> It looks like the dream of a conduit between API services and consumers
>>>> might have finally come true so we could move-on an worry about other
>>>> things.
>>>>
>>>> So has anyone already starting looking into it?
>>>>
>>>> [1]
>>>>
>>>> https://www.openstack.org/videos/boston-2017/building-modern-apis-with-graphql
>>>> [2] http://graphql.org
>>>>
>>>>
>>>>
>>>>
>>>> __________________________________________________________________________
>>>> OpenStack Development Mailing List (not for usage questions)
>>>> Unsubscribe:
>>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180502/32f8721e/attachment.html>