Open Stack

Those are some good questions and I pondered them last night even
before I read your email.

Right now, if no transport layer is passed in a default one is used
for them. The don't make me think implementation is already there. If
you want to use something other than the default one than you need to
inject one. I even had an idea how to tweak our use of the default one
to improve performance in the network layer. More on that after some
testing.

The idea that a class either holds state or performs functionality is
not agreed to. Some people like that. For example, Rich Hickey talked
about it at Strangeloop in his talk Simple Made Easy
(http://www.infoq.com/presentations/Simple-Made-Easy). Alternately,
there are definitions of OOP floating around like, "Rather than
structure programs as code and data, an object-oriented system
integrates the two using the concept of an “object”. An object is an
abstract data type that has state (data) and behavior (code)." This is
from http://spf13.com/post/is-go-object-oriented which talks about the
Go programming language.

While this has some debate to it, when I read about object oriented
programming on Wikipedia it starts out saying, "Object-oriented
programming (OOP) is a programming paradigm that represents the
concept of "objects" that have data fields (attributes that describe
the object) and associated procedures known as methods."

A transport layer in this case does have state and functionality. The
state is related to it's ability to transport things. Things like a
proxy server or timeout. Those are state. The functionality is moving
data.

There are other transport clients that don't have state about about
what they are transporting (like a base URL). For example, in the Go
standard library there is an HTTP client (with a lower level transport
mechanism) and neither of these layers has state about what they are
transporting. The state is around how they transport it.

The library as it was brought into Stackforge (before the author of
that component had ever seen Go) wrote one that worked this way. It
was a singleton but it's design handled transporting this way.

I was poking around the popular request library used in node.js/JS
(and used by pkgcloud for OpenStack) and it operates in the similar
manner to what we're talking about in a transport layer. Sure, you can
set default headers. That's useful for setting things like the user
agent. Looking at the example use of request and how pkgcloud uses it
works similar to how I've described using a transport layer.

Transport layers like this are not unheard of.

I'm aware of a number of codebases where Guzzle is used as a stateless
transport layer. Using curl with connection pooling across platforms
in PHP is hard. The API isn't easy to use. Guzzle gives it a good API
and makes it easy. Lots of people like that and use it that way. Since
the codebases that came to mind were not open source I poked around at
open source codebases. I found a lot of projects that use it as a
stateless client. I looked at Guzzle 4 which doesn't have much uptake
and Guzzle 3.

To answer the question, "Do we really feel confident building our
transport client like nobody else has before?"... I feel confident
doing it this way and it is like others before. It's just not the hot
talked about thing on the conference circuit right now.

Hope this helps clarify my thinking.

- Matt




On Mon, Jun 9, 2014 at 11:54 AM, Jamie Hannaford
<jamie.hannaford at rackspace.com> wrote:
> So what you’re proposing is effectively a stateless transport client that
> accepts input and then works on it? I didn’t realize you meant this until
> your previous e-mail - the more I think about your idea, the more I like it.
> I think it’s great for two reasons:
>
> - It really ties in with our idea of a pluggable transport layer. My idea of
> having transport state doesn’t actually go against this (nor does it break
> single responsibility/blur separation of concerns like you mention - but
> that’s another debate!) - but your idea enforces it in a stronger way.
>
> - In OOP, many people believe that a class should either hold state or
> perform functionality - but not both. If we remove state from our client,
> we’d have a smaller and more coherent scope of responsibility.
>
> However, as much as I like the idea, I do have some major reservations which
> we need to think about:
>
> - Nearly all HTTP clients out there hold state - whether it’s base URLs,
> default headers, whatever. Guzzle does this, ZF2 does this, Symfony does
> this, Python’s base HTTP client does this. Do we really feel confident
> building our transport client like nobody else has before? Maybe there are
> good reasons why they chose to maintain state in their clients for reasons
> we can’t immediately see now.
>
> - Do we really expect our end-users to create, manage and inject the
> transport clients? You yourself admitted that most of our users will not
> understand or use dependency injection - so why push a pattern that will
> hardly be used by the majority of our user base? I’ve worked on our current
> SDK for over a year - it has almost 40k downloads - and I’ve never been
> asked about custom transport configurations like proxy settings or whatever.
>
> - Related to the previous point: how are you going to lower the barriers for
> users who don’t want to inject their own HTTP clients? Will you offer a
> default transport client, for example?
>
> Jamie
>
> On June 7, 2014 at 9:58:07 PM, Matthew Farina (matt at mattfarina.com) wrote:
>
> My comments are inline below...
>
>
> On Fri, Jun 6, 2014 at 8:47 AM, Jamie Hannaford
> <jamie.hannaford at rackspace.com> wrote:
>>
>> Whether the same one is used for each service or a new one is used for
>> each service doesn't matter.
>>
>>
>> Yes, it does matter IMO - and here are the reasons why:
>>
>> 1. By sharing a global transport object you’re introducing the risk of
>> side effects. A transport object contains state that can be modified by its
>> service object. Somewhere along the line, a Swift service could introduce a
>> state modification that’s completely incompatible with a Nova service.
>> What’s worse is that it will be a nightmare to debug - you’d have to trawl
>> the entire service to see points where it interacts with the transport
>> client. This is why people don’t use singletons - it’s incredibly risky and
>> hard to debug. Object instantiations, on the other hand, are cheap and they
>> offer protection through isolation.
>
>
> There are two things here.
>
> First, if the transport client has no state for the service than it doesn't
> get mixed up on state. A Swift client would never introduce state for swift
> to the transport client because the transport client has no state for this.
> It's for transporting.
>
> Second, it's not a singleton. You could have the same transport client for
> all of them, a different transport client for each, or any permutation in
> between. If the transport client contains no state to a service than it
> doesn't matter.
>
> To quote wikipedia, "the singleton pattern is a design pattern that
> restricts the instantiation of a class to one object". A singleton is an
> intended restriction. This isn't a restriction. It's about options.
>
> If the service client is responsible for state for the service and the
> transport client is responsible for transporting information and the state
> of transport (e.g., is the info going through a proxy) than you don't run
> into issues where the transport client knows state of a service because
> that's the responsibility of the service client not the transport client.
>
>
>>
>>
>>  2. Certain services rely on custom transport configurations. Each
>> transport client has a base URL that is used for issuing HTTP requests -
>> every time you execute a request, you’re effectively adding relevant paths
>> for that API operation. A Swift service will have different URL endpoints
>> from a Nova one - so there’s no point sharing. Another example is custom
>> headers. Marconi requests custom headers to be sent, as does Glance. You
>> save these as default headers on the transport client, that are sent for all
>> requests that the service executes. These custom headers are not applicable
>> to any other service except Marconi/Glance.
>
>
> If a transport client know the base URL than it knows state about the
> service. The separation of concerns is broken. Why does it need to know the
> URL? Why does it need to know about custom headers? Customizations and state
> for a service are the responsibility of the service client and not the
> transport client.
>
> Why does a service client and transport client need to both know the state
> of the service? The responsibility become blurred here.
>
>
>>
>>
>> In the use-cases you mentioned, you’d easily handle that. You’d pass in
>> proxy settings through the OpenStack entry point (like you do with your
>> username and password), which would then percolate down into the transport
>> clients as they’re created. These settings would be injected into each
>> transport client. So if you require a different set-up for public clouds -
>> that’s fine - you define different settings and fire up another $openstack
>> object.
>
>
> How things get passed around isn't an issues. I don't think we need to
> debase how we pass settings around right now. The issue is separation of
> concerns between the service clients and the transport clients.
>
>
>>
>>
>> -OR- you could define different transport settings for different services
>> - by passing them into the $openstack->get(‘compute’, [‘custom_settings’ =>
>> true]); call. This is great because it gives users the ability to apply
>> custom transport options to certain services. So if I want to interact with
>> a private Compute instance, I’d pass in a custom transport configuration for
>> that service; if I wanted to use a proxy with my Swift service, I can pass
>> details into that service when creating it. You can only do this (provide
>> custom transport settings for 1 service) if each transport client is
>> isolated, i.e. if there’s a 1-to-1 relationship between service and
>> transport client. If you have a global one, you couldn’t introduce custom
>> settings per service because it’d affect ALL others, which is a bad user
>> experience.
>
>
> We're not talking about an application. We're talking about an SDK with a
> simple entry point for ease and building blocks you can do a lot with. This
> isn't about a 1-to-1 relationship between a service and transport client OR
> a global one. It's different than that.
>
> They should have different responsibilities. Entirely different. A transport
> client moves data. It doesn't know about a service. A service client knows
> about a service but not about moving data. They have their own scope.
>
> A transport client is used to move data. Since it's scope is about
> transporting the different configurations for it are about the different
> ways an application needs to transport things. If it's scoped at
> transporting it doesn't need or care to know anything else. It doesn't know
> state on a service (and state includes a URL to a service).
>
> A service client knows about a service. So, for each service you connect to
> you'd need one. You'd need as many service clients as services you'd connect
> to.
>
> A singleton (forcing just one) would be a bad thing. A 1-to-1 relationship
> between the two where the transport client knows about state of the service
> breaks the separation of concerns.
>
> They have separate jobs. The number of each you need depends on what's
> happening in the scope of each space.
>
> Does that make sense?
>
>>
>>
>> Jamie
>>
>> On June 5, 2014 at 6:33:34 PM, Matthew Farina (matt at mattfarina.com) wrote:
>>
>>
>> > My opinion is that we create a new transport client instance for every
>> > service client, not re-use existing instances. What’s your take on this?
>>
>> I'm not in agreement and here is why (with a use case).
>>
>> A transport client is concerned with transporting only. Whether the same
>> one is used for each service or a new one is used for each service doesn't
>> matter.
>>
>> An example of using two transport clients would be a case where an
>> application communicates with two different OpenStack clouds. One is a
>> public cloud and the application communicates through a proxy. A transport
>> client would know how to talk through the proxy to the public cloud. A
>> second OpenStack cloud is a private cloud that is on the same company
>> network. A second transport client would know how to talk to that without
>> communicating through the proxy.
>>
>> The service clients communicating with each cloud would use the
>> appropriate transport client.
>>
>> The mapping of transport client to service client doesn't need to be 1:1
>> if they operate in different spaces. Only having instances of a transport
>> client as needed decreases the use of resources or the time needed to manage
>> those.
>>
>> If a transport client is only concerned with transporting than what is the
>> need to have more than one per case to transport?
>>
>> - Matt
>>
>> On Thu, Jun 5, 2014 at 12:09 PM, Jamie Hannaford
>> <jamie.hannaford at rackspace.com> wrote:
>>>
>>> I completely agree with you regarding separation of concerns.
>>>
>>> I also agree with your definitions: a transport client is for managing
>>> HTTP transactions, a service client contains all the domain logic for an API
>>> service (Swift, Nova, etc.). A service knows nothing about HTTP, a transport
>>> client knows nothing about Swift. A transport client is injected into the
>>> service client, satisfying the type hint. So any transport client
>>> implementing our interface is fine.
>>>
>>> Up to this point I’m in 100% agreement. The area which I think I
>>> misunderstood was the creation process of service clients. My take was that
>>> you were advocating a shared transport client instance - i.e. a transport
>>> client instantiated once, and re-used for every service client. If we did
>>> that, there would be global state.
>>>
>>> My opinion is that we create a new transport client instance for every
>>> service client, not re-use existing instances. What’s your take on this?
>>>
>>> Jamie
>>>
>>> On June 5, 2014 at 5:17:57 PM, Matthew Farina (matt at mattfarina.com)
>>> wrote:
>>>
>>> We've started to talk about the interactions between transport
>>> clients, service clients, and state. I've noticed we're not on the
>>> same page so I wanted to start a dialog. Here's my starting point...
>>>
>>> A Transport Client is about transporting data. It sends and receives
>>> data.
>>>
>>> A Service Client handles the interactions with a service (e.g., swift,
>>> nova, keystone).
>>>
>>> A Service Client uses a Transport Client when it needs to transport
>>> data to and from a service.
>>>
>>> When it comes to state, a Transport Client knows about transporting
>>> things. That means it knows things like if there is a proxy and how to
>>> work with it. A Service Client knows about a service which includes
>>> and state for that service.
>>>
>>> In the realm of separation of concerns, a Service Client doesn't know
>>> about transport state and a Transport Client doesn't know about
>>> service state. They are separate.
>>>
>>> A Service Client doesn't care what Transport Client is used as long as
>>> the API (interface) is compliant. A Transport Client doesn't care what
>>> code calls it as long as it uses the public API defined by an
>>> interface.
>>>
>>> This is my take. If someone has a different take please share it with
>>> the reasoning.
>>>
>>> - Matt
>>>
>>>
>>>
>>> Jamie Hannaford
>>> Software Developer III - CH
>>> Tel: +41434303908
>>> Mob: +41791009767
>>>
>>>
>>>
>>> Rackspace International GmbH a company registered in the Canton of
>>> Zurich, Switzerland (company identification number CH-020.4.047.077-1) whose
>>> registered office is at Pfingstweidstrasse 60, 8005 Zurich, Switzerland.
>>> Rackspace International GmbH privacy policy can be viewed at
>>> www.rackspace.co.uk/legal/swiss-privacy-policy
>>> -
>>> Rackspace Hosting Australia PTY LTD a company registered in the state of
>>> Victoria, Australia (company registered number ACN 153 275 524) whose
>>> registered office is at Suite 3, Level 7, 210 George Street, Sydney, NSW
>>> 2000, Australia. Rackspace Hosting Australia PTY LTD privacy policy can be
>>> viewed at www.rackspace.com.au/company/legal-privacy-statement.php
>>> -
>>> Rackspace US, Inc, 5000 Walzem Road, San Antonio, Texas 78218, United
>>> States of America
>>> Rackspace US, Inc privacy policy can be viewed at
>>> www.rackspace.com/information/legal/privacystatement
>>> -
>>> Rackspace Limited is a company registered in England & Wales (company
>>> registered number 03897010) whose registered office is at 5 Millington Road,
>>> Hyde Park Hayes, Middlesex UB3 4AZ.
>>> Rackspace Limited privacy policy can be viewed at
>>> www.rackspace.co.uk/legal/privacy-policy
>>> -
>>> Rackspace Benelux B.V. is a company registered in the Netherlands
>>> (company KvK nummer 34276327) whose registered office is at
>>> Teleportboulevard 110, 1043 EJ Amsterdam.
>>> Rackspace Benelux B.V privacy policy can be viewed at
>>> www.rackspace.nl/juridisch/privacy-policy
>>> -
>>> Rackspace Asia Limited is a company registered in Hong Kong (Company no:
>>> 1211294) whose registered office is at 9/F, Cambridge House, Taikoo Place,
>>> 979 King's Road, Quarry Bay, Hong Kong.
>>> Rackspace Asia Limited privacy policy can be viewed at
>>> www.rackspace.com.hk/company/legal-privacy-statement.php
>>> -
>>> This e-mail message (including any attachments or embedded documents) is
>>> intended for the exclusive and confidential use of the individual or entity
>>> to which this message is addressed, and unless otherwise expressly
>>> indicated, is confidential and privileged information of Rackspace. Any
>>> dissemination, distribution or copying of the enclosed material is
>>> prohibited. If you receive this transmission in error, please notify us
>>> immediately by e-mail at abuse at rackspace.com and delete the original
>>> message. Your cooperation is appreciated.
>>
>>
>>
>>
>> Jamie Hannaford
>> Software Developer III - CH
>> Tel: +41434303908
>> Mob: +41791009767
>>
>>
>>
>> Rackspace International GmbH a company registered in the Canton of Zurich,
>> Switzerland (company identification number CH-020.4.047.077-1) whose
>> registered office is at Pfingstweidstrasse 60, 8005 Zurich, Switzerland.
>> Rackspace International GmbH privacy policy can be viewed at
>> www.rackspace.co.uk/legal/swiss-privacy-policy
>> -
>> Rackspace Hosting Australia PTY LTD a company registered in the state of
>> Victoria, Australia (company registered number ACN 153 275 524) whose
>> registered office is at Suite 3, Level 7, 210 George Street, Sydney, NSW
>> 2000, Australia. Rackspace Hosting Australia PTY LTD privacy policy can be
>> viewed at www.rackspace.com.au/company/legal-privacy-statement.php
>> -
>> Rackspace US, Inc, 5000 Walzem Road, San Antonio, Texas 78218, United
>> States of America
>> Rackspace US, Inc privacy policy can be viewed at
>> www.rackspace.com/information/legal/privacystatement
>> -
>> Rackspace Limited is a company registered in England & Wales (company
>> registered number 03897010) whose registered office is at 5 Millington Road,
>> Hyde Park Hayes, Middlesex UB3 4AZ.
>> Rackspace Limited privacy policy can be viewed at
>> www.rackspace.co.uk/legal/privacy-policy
>> -
>> Rackspace Benelux B.V. is a company registered in the Netherlands (company
>> KvK nummer 34276327) whose registered office is at Teleportboulevard 110,
>> 1043 EJ Amsterdam.
>> Rackspace Benelux B.V privacy policy can be viewed at
>> www.rackspace.nl/juridisch/privacy-policy
>> -
>> Rackspace Asia Limited is a company registered in Hong Kong (Company no:
>> 1211294) whose registered office is at 9/F, Cambridge House, Taikoo Place,
>> 979 King's Road, Quarry Bay, Hong Kong.
>> Rackspace Asia Limited privacy policy can be viewed at
>> www.rackspace.com.hk/company/legal-privacy-statement.php
>> -
>> This e-mail message (including any attachments or embedded documents) is
>> intended for the exclusive and confidential use of the individual or entity
>> to which this message is addressed, and unless otherwise expressly
>> indicated, is confidential and privileged information of Rackspace. Any
>> dissemination, distribution or copying of the enclosed material is
>> prohibited. If you receive this transmission in error, please notify us
>> immediately by e-mail at abuse at rackspace.com and delete the original
>> message. Your cooperation is appreciated.
>
>
>
>
> Jamie Hannaford
> Software Developer III - CH
> Tel: +41434303908
> Mob: +41791009767
>
>
>
>
> Rackspace International GmbH a company registered in the Canton of Zurich,
> Switzerland (company identification number CH-020.4.047.077-1) whose
> registered office is at Pfingstweidstrasse 60, 8005 Zurich, Switzerland.
> Rackspace International GmbH privacy policy can be viewed at
> www.rackspace.co.uk/legal/swiss-privacy-policy
> -
> Rackspace Hosting Australia PTY LTD a company registered in the state of
> Victoria, Australia (company registered number ACN 153 275 524) whose
> registered office is at Suite 3, Level 7, 210 George Street, Sydney, NSW
> 2000, Australia. Rackspace Hosting Australia PTY LTD privacy policy can be
> viewed at www.rackspace.com.au/company/legal-privacy-statement.php
> -
> Rackspace US, Inc, 5000 Walzem Road, San Antonio, Texas 78218, United States
> of America
> Rackspace US, Inc privacy policy can be viewed at
> www.rackspace.com/information/legal/privacystatement
> -
> Rackspace Limited is a company registered in England & Wales (company
> registered number 03897010) whose registered office is at 5 Millington Road,
> Hyde Park Hayes, Middlesex UB3 4AZ.
> Rackspace Limited privacy policy can be viewed at
> www.rackspace.co.uk/legal/privacy-policy
> -
> Rackspace Benelux B.V. is a company registered in the Netherlands (company
> KvK nummer 34276327) whose registered office is at Teleportboulevard 110,
> 1043 EJ Amsterdam.
> Rackspace Benelux B.V privacy policy can be viewed at
> www.rackspace.nl/juridisch/privacy-policy
> -
> Rackspace Asia Limited is a company registered in Hong Kong (Company no:
> 1211294) whose registered office is at 9/F, Cambridge House, Taikoo Place,
> 979 King's Road, Quarry Bay, Hong Kong.
> Rackspace Asia Limited privacy policy can be viewed at
> www.rackspace.com.hk/company/legal-privacy-statement.php
> -
> This e-mail message (including any attachments or embedded documents) is
> intended for the exclusive and confidential use of the individual or entity
> to which this message is addressed, and unless otherwise expressly
> indicated, is confidential and privileged information of Rackspace. Any
> dissemination, distribution or copying of the enclosed material is
> prohibited. If you receive this transmission in error, please notify us
> immediately by e-mail at abuse at rackspace.com and delete the original
> message. Your cooperation is appreciated.