Open Stack

Rich,

Thanks for the feedback. It is very welcome.

As regards the sample size, there are two variables:

A. The number of people who fill in the survey since the previous time
B. The window that we use to select the statistics (i.e. 6 months, 12 months, ...)

For A, I'd be very pleased if we can encourage more responses. We do ask for this at the various meetups and walk-up stations at summits. I am certainly aware of a significant number of deployments which have not been reported as part of the survey despite the NDA. Given the multiple deployment models of OpenStack, I suspect we may be getting a bias on those who follow mailing lists, twitter feeds etc. rather than those who use an OpenStack distribution.

For B, we can increase the coverage by accepting deployments that were filled out after a certain time (such as updated between 6 and 12 months ago). Doing so gives a significant increase in the numbers but at the expense of getting an out of date view. Statistics like the deployed version or number of nodes are time dependent and taking less current data risks giving an incorrect picture. Thus, we have preferred using percentages and a smaller window.

The last two surveys we have moved to percentage based deployments to allow analysis of trends and tried to keep the questions consistent. However, the answers to some questions have clearly shown the needs to add further options/clarifications. Thus, a 1-to-1 comparison is not always possible.

I'd be happy to have suggestions on how to best represent the community in these reports while reflecting how quickly the nature of the OpenStack deployments can change.

Tim
> -----Original Message-----
> From: Rich Bowen [mailto:rbowen at redhat.com]
> Sent: 11 June 2015 18:33
> To: user-committee at lists.openstack.org
> Subject: [User-committee] User survey feedback
> 
> I'm hearing a lot of negative feedback about the user survey, and I thought I
> should share it with this list. Please take this in the spirit in which it is intended - a
> desire to improve the survey and make it more valuable to all of our
> constituents, rather than criticism of the folks that have put their time and effort
> into it so far.
> 
> There are two big complaints that I hear consistently.
> 
> 1) The sample size.
> http://superuser.openstack.org/articles/openstack-users-share-how-their-
> deployments-stack-up
> is based on 270 deployments. This seems really small, given what we keep saying
> the global market for OpenStack is. This calls into question the validity of any
> conclusions we can draw from the numbers.
> 
> 2) Consistency from one survey to the next. It is very hard to determine any kind
> of trends when the survey, and the reporting mechanism, changes so much from
> one time to the next. Yes, adjustments will be made from one survey to the
> next, but the inability to determine trends is very frustrating.
> 
> There's one lesser complaint that I hear some, and that's that the wall around
> the survey is too high. That is, when I go to fill out the survey it asks me to
> become an individual member of the OpenStack Foundation, and that's quite a
> surprise to someone that's been asked to fill out a survey about their
> installation. I suspect we lose a lot of potential answers to this, based on booth
> conversations at events.
> 
> I am not a statistician - I used to be mathematician but I can't even claim that
> any more. I don't know what the solutions to these problems are. But I wanted
> to be sure that this committee is aware that these are the complaints that are
> being aired.
> 
> I'm sure that some of the complaints are from people who don't feel that the
> numbers reflect well on their particular project or company, and that's to be
> expected (and probably ignored). But if there are ways that we can improve the
> survey, and get better numbers from the broader community, that would seem
> to be a big benefit to everyone.
> 
> 
> --
> Rich Bowen - rbowen at redhat.com
> OpenStack Community Liaison
> http://rdoproject.org/
> 
> _______________________________________________
> User-committee mailing list
> User-committee at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee