Open Stack

Jamie, thanks for sharing those links. They are quite useful and led
me to a couple questions.

1. To quote the first link, "To do this, we need a way to describe the
requirement such that everyone – the business folks, the analyst, the
developer and the tester – have a common understanding of the scope of
the work." Where are the business folks, the analyst, and the tester?
behat does things in human readable language that's really useful for
the non-developer. Where do we have these in the development of this
SDK?

I ask because in that first post the idea of working with these
different types of people is a central point. If you're working on a
client project for non-technical clients, which is common in
consulting circles, and in enterprise apps where you have analysts,
product managers, and others this is definitely useful for them. Where
are they in the loop on developing this SDK?

2. Can you point to end users of the SDK who aren't developers or
engineers? That's not to say that someone developing an application
that uses the SDK isn't working with non-developers. If they have a
story about uploading a file to persistent storage the implementation
by the developer might use the SDK. But, us using BDD doesn't help
that process.

This is really about the people involved in this project and consuming
it. It's different from typical client consulting work or general
consumer facing production. The audience is different. Can you explain
how this technology is useful to this specific audience in a practical
way?

Thanks,
Matt


On Fri, Jun 6, 2014 at 12:55 PM, Jamie Hannaford
<jamie.hannaford at rackspace.com> wrote:
> Hey all,
>
> Sorry for the length of reply - but I want to provide as much information as
> possible about this topic.
>
> Instead of enumerating pros and cons, I want to give a bit of context first
> (about what “feature stories” actually are), and then respond to some common
> misconceptions about them. This way, the pros/cons of Behat are a bit more
> substantiated.
>
>
> Would Behat replace PHPUnit?
>
> No - they’re completely different. We’d still use phpunit for unit testing
> because it’s way better at xunit-like assertions. We’d use behat instead for
> functional testing - making sure that features work against a production
> API.
>
>
> Who’s using Behat and is it suitable for us?
>
> From what I’ve heard, we’re using it for some projects at Rackspace and
> possibly some OpenStack projects - but I need to double check that. I’ve
> reached out to some folks about their experiences with it - so I’ll post the
> findings when I hear back.
>
>
> What are BDD feature stories?
>
> Here’s a link to a fantastic article which explains the benefits of BDD
> feature stories: http://dannorth.net/whats-in-a-story/
>
> tl;dr:
>
> BDD takes the position that you can turn an idea for a requirement into
> implemented, tested, production-ready code simply and effectively, as long
> as the requirement is specific enough that everyone knows what’s going on.
> To do this, we need a way to describe the requirement such that everyone –
> end-user, contributor, manager, technical lead (in short, anyone interested
> in using our SDK in their business) – have a common understanding of the
> scope of the work. You are showing them, in human-readable language, the
> features of the SDK and what it offers them. The result is that everyone —
> regardless of proficiency, skill level and familiarity with the codebase —
> is on the same level of understanding. From this they can agree a common
> definition of “done”, and we escape the dual gumption traps of “that’s not
> what I asked for” or “I forgot to tell you about this other thing”.
>
> This, then, is the role of a Story. It is a description of a requirement and
> a set of criteria by which we all agree that it is “done”. It helps us
> understand and satisfy customer use-cases in a well expressed and clear way.
> It also helps us track project progress by having well-established
> acceptance criteria for feature sets.
>
>
> 3 misconceptions about BDD
>
> (Inspired by
> http://www.thoughtworks.com/insights/blog/3-misconceptions-about-bdd)
>
> 1. End-users don’t care about this! They want code
>
> This is actually a completely misdirected point. The purpose of behat is not
> to serve as a public-facing repository of sample code. Its actual purpose is
> twofold: to serve as a functional test suite (i.e. make sure our SDK works
> against an API), and secondly to serve as a communication device - to codify
> features in a human-readable way.
>
> It’s the role of documentation to explain the concepts of the SDK with
> detailed code samples. Another good idea is to provide a “samples” folder
> that contains standalone scripts for common use-cases - this is what we
> offer for our current SDK, and users appreciate it. Both of these will allow
> developers to copy and paste working code for their requirements.
>
> 2. Contributors don’t want to write these specifications!
>
> My response is this: how can you implement a piece of functionality if you
> have no idea what it’s supposed to do? You’re not communicating the scope of
> the feature, you’re not defining the criteria that makes the feature “done”
> - you’re just slinging code. Without feature stories, if somebody has a
> major problem with an approach you’re taking in your development of a
> feature, you’ll only find out afterwards in Gerrit. You’d end up wasting
> hours on stuff that does not deliver the feature as expected - it’s a
> wasteful and inefficient example of miscommunication that could easily be
> avoided.
>
> If you write the acceptance criteria beforehand - you know exactly what you
> have to contribute. Everybody knows when the feature will be “done” because
> you’ve settled all the defined customer use-cases. Writing feature stories
> is all about collaboration and communication.
>
> 3. You can achieve the same with normal code!
>
> Again, this is a misdirected point because comparing them is like comparing
> apples and oranges - they’re completely different things.
>
> Behat is for communication. The whole reason for using the Gherkin DSL is
> for its readability. Its aim is to improve communication and help people
> understand features.  PHP code does not offer this - you’d end up trying to
> communicate features to people who are not proficient with our codebase, who
> may not be strong coders, and who might not even know PHP. We can’t assume
> that people want to traverse a labyrinth of code just to understand how a
> feature works.
>
> Behat is for defining end-user expectations. Writing normal code doesn’t
> offer us a way to “test” functionality. With Behat we’re explicitly testing
> expectations: does this scenario return the right thing? Does it return this
> value? What about if I was a different user with different data? You’re
> testing scenarios. How easy would that be using normal PHP code? You’d have
> hundreds of lines and probably lots of messy exceptions. In short, it's
> messy and massively inefficient. Behat allows us to avoid this clumsiness
> and duplication by centralizing this type of expectation checking.
>
>
> Ugh, another dependency. What happens if we switch to another BDD framework?
>
> Absolutely nothing. The beauty of the Gherkin syntax is that it’s framework
> independent. This means we could even take the Feature files from another
> non-PHP project, such as an OpenStack one, and it’d be fine. We’d have to
> re-configure our context classes (the things which links human-readable DSL
> into SDK calls), but that’d be it. It’s completely interoperable :)
>
>
> Jamie
>
> On June 6, 2014 at 12:22:54 PM, Choi, Sam (sam.choi at hp.com) wrote:
>
> Hello all,
>
> During our most recent php sdk team meeting, a suggestion to use the PHP
> framework Behat (http://behat.org/) was brought up. I’d like to begin a
> discussion around the pros/cons of using Behat to create tests. For those
> not familiar with the php sdk or Behat, we are currently using PHPUnit for
> all of our tests and are considering Behat for writing human-readable
> stories that can potentially generate tests that can be run against the sdk.
>
>
>
> A couple general questions for all devs
>
> -        Is a similar framework being used for any OpenStack CLI/SDK
> project? I’d love to see an example.
>
> -        What kind of pros/cons have you all seen when using Behat or
> similar frameworks for testing?
>
>
>
> Jamie
>
> -        Would the suggested Behat framework be intended to supplement our
> existing PHPUnit tests? In effect, giving us a set of Behat tests and
> PHPUnit tests. Or was the suggestion to have all tests rely on the Behat
> framework?
>
>
>
> After reviewing Behat and some related technologies, I do have a few
> concerns at the moment. Any thoughts of the concerns would be appreciated.
>
>
>
> -        We are introducing another dependency for the PHP SDK. What happens
> if there is a reason for us to switch to another BDD framework or ditch it
> to go back to PHPUnit? It seems like we would likely be faced with
> significant test refactoring.
>
> -        Contributors will have to be proficient with Behat/BDD in order to
> write tests for the PHP SDK. This adds to what may already be a steep
> learning curve for a new contributor who has to learn our code base, Guzzle
> for the transport layer, how to use various/develop various cloud services
> etc.
>
> -        For more complicated tests, writing out features, scenarios, step
> definitions, and testing logic would almost certainly take longer than
> writing a traditional test in pure PHP.
>
>
>
> Thanks,
>
> --
>
> Sam Choi
>
> Hewlett-Packard Co.
>
> HP Cloud Services
>
> +1 650 316 1652 / Office
>
>
>
>
>
> Jamie Hannaford
> Software Developer III - CH
> Tel: +41434303908
> Mob: +41791009767
>
>
>
> Rackspace International GmbH a company registered in the Canton of Zurich,
> Switzerland (company identification number CH-020.4.047.077-1) whose
> registered office is at Pfingstweidstrasse 60, 8005 Zurich, Switzerland.
> Rackspace International GmbH privacy policy can be viewed at
> www.rackspace.co.uk/legal/swiss-privacy-policy
> -
> Rackspace Hosting Australia PTY LTD a company registered in the state of
> Victoria, Australia (company registered number ACN 153 275 524) whose
> registered office is at Suite 3, Level 7, 210 George Street, Sydney, NSW
> 2000, Australia. Rackspace Hosting Australia PTY LTD privacy policy can be
> viewed at www.rackspace.com.au/company/legal-privacy-statement.php
> -
> Rackspace US, Inc, 5000 Walzem Road, San Antonio, Texas 78218, United States
> of America
> Rackspace US, Inc privacy policy can be viewed at
> www.rackspace.com/information/legal/privacystatement
> -
> Rackspace Limited is a company registered in England & Wales (company
> registered number 03897010) whose registered office is at 5 Millington Road,
> Hyde Park Hayes, Middlesex UB3 4AZ.
> Rackspace Limited privacy policy can be viewed at
> www.rackspace.co.uk/legal/privacy-policy
> -
> Rackspace Benelux B.V. is a company registered in the Netherlands (company
> KvK nummer 34276327) whose registered office is at Teleportboulevard 110,
> 1043 EJ Amsterdam.
> Rackspace Benelux B.V privacy policy can be viewed at
> www.rackspace.nl/juridisch/privacy-policy
> -
> Rackspace Asia Limited is a company registered in Hong Kong (Company no:
> 1211294) whose registered office is at 9/F, Cambridge House, Taikoo Place,
> 979 King's Road, Quarry Bay, Hong Kong.
> Rackspace Asia Limited privacy policy can be viewed at
> www.rackspace.com.hk/company/legal-privacy-statement.php
> -
> This e-mail message (including any attachments or embedded documents) is
> intended for the exclusive and confidential use of the individual or entity
> to which this message is addressed, and unless otherwise expressly
> indicated, is confidential and privileged information of Rackspace. Any
> dissemination, distribution or copying of the enclosed material is
> prohibited. If you receive this transmission in error, please notify us
> immediately by e-mail at abuse at rackspace.com and delete the original
> message. Your cooperation is appreciated.
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>