Open Stack

Hi Tim,

 
Thanks for addressing some of the potential challenges around conducting
surveys on behalf of the community.

 
The research conducted by the OpenStack UX project, including surveys,
tend to focus on specific project needs rather than the overall direction
of the industry, which is captured by the bi-annual survey.  The project
surveys generally require a fair amount of detail in order to enable the
project teams to make decision around product direction. As a result,
adding a few questions to the foundation¹s survey doesn¹t generate the
level of detail typically needed by each project.  Also, any questions
added
to the bi-annual survey are at the discretion of the user committee, which
is a concern for me.

 
In addition, waiting for a survey every six months would not allow us to
be responsive to the project research needs.
 

I agree with your concern with population fatigue and we would prefer to
conduct studies with a more focused sample rather than the overall
OpenStack community.   For example, we may specifically focus on network
admins during one survey while focusing on other roles during another
study.  In those cases, user/operators
have generally been willing to participate because the results should have
a tangible impact on their daily activities.

 
One recommendation would be to distribute a screener to the overall
community to identify the specific skills and focus of its members.   The
user committee and project
teams could use the database created from the screener to identify
potential participants for the various research activities.   The goal
would be to be more focused on how
we recruit participants rather than rolling out to the entire community.
It would also allow us to track and limit how often members are being
invited to participate in studies to avoid population fatigue.  The
screener would also respondents to opt of
being recruited for research activities.
 

We¹ve asked in the past, but the user committee has not been unable to
provide anonymized data because of a policy within the foundation that
limits access to data to a handful of users.  I don¹t dispute the need for
the policy, but summary statistics aren¹t helpful for conducting
statistical analysis.  More recently, we¹ve asked for the raw data from
the operator job analysis survey because of its value in helping to drive
persona development, but have yet to hear back from the foundation.

 
Piet




Piet Kruithof
Sr UX Architect, HP Helion Cloud
PTL, OpenStack UX project


"For every complex problem, there is a solution that is simple, neat and
wrong.²

H L Menken





On 9/28/15, 1:14 PM, "user-committee-request at lists.openstack.org"
<user-committee-request at lists.openstack.org> wrote:

>Send User-committee mailing list submissions to
>	user-committee at lists.openstack.org
>
>To subscribe or unsubscribe via the World Wide Web, visit
>	http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee
>or, via email, send a message with subject or body 'help' to
>	user-committee-request at lists.openstack.org
>
>You can reach the person managing the list at
>	user-committee-owner at lists.openstack.org
>
>When replying, please edit your Subject line so it is more specific
>than "Re: Contents of User-committee digest..."
>
>
>Today's Topics:
>
>   1. [ceilometer] OpenStack Telemetry user survey (gord chung)
>   2. Per-project surveys (Tim Bell)
>   3. Re: Per-project surveys (Jonathan Proulx)
>   4. Re: Per-project surveys (Bruno Morel)
>
>
>----------------------------------------------------------------------
>
>Message: 1
>Date: Mon, 28 Sep 2015 11:20:31 -0400
>From: gord chung <gord at live.ca>
>To: user-committee at lists.openstack.org
>Subject: [User-committee] [ceilometer] OpenStack Telemetry user survey
>Message-ID: <BLU436-SMTP1679BBE89E1E497D27C2BDDDE4F0 at phx.gbl>
>Content-Type: text/plain; charset="utf-8"; format=flowed
>
>Hello,
>
>The OpenStack Telemetry (aka Ceilometer) team would like to collect
>feedback and information from its user base in order to drive future
>improvements to the project.  To do so, we have developed a survey. It
>should take about 15min to complete.
>Questions are fairly technical, so please ensure that you ask someone
>within your organization that is hands on using Ceilometer.
>
>https://goo.gl/rKNhM1
>
>On behalf of the Ceilometer community, we thank you for the time you
>will spend in helping us understand your needs.
>
>-- 
>Gordon Chung
>Ceilometer PTL
>
>
>
>------------------------------
>
>Message: 2
>Date: Mon, 28 Sep 2015 17:03:49 +0000
>From: Tim Bell <Tim.Bell at cern.ch>
>To: "user-committee at lists.openstack.org"
>	<user-committee at lists.openstack.org>
>Subject: [User-committee] Per-project surveys
>Message-ID:
>	<5D7F9996EA547448BC6C54C8C5AAF4E5010A478000 at CERNXCHG41.cern.ch>
>Content-Type: text/plain; charset="us-ascii"
>
> 
>
>There have been a couple of project surveys of OpenStack users in the past
>week (Neutron and Ceilometer).
>
> 
>
>They tend to duplicate the standard questions (industry, size of
>deployment,
>release levels, .) and also some of the questions we are currently asking
>in
>the survey (nova network, metering feedback). The commitment to anonymity
>is
>also not always clear.
>
> 
>
>The difficulty I see is that we reach survey limits, where people are not
>sure what is official and whether the questions/audience may lead to a
>bias.
>
> 
>
>On the other hand, the PTLs often want more detailed information and the
>user survey needs to be kept to a reasonable length (to avoid we lose
>everything because people bailed out before finishing).
>
> 
>
>Any suggestions on how to proceed ?
>
> 
>
>Tim
>
> 
>
>-------------- next part --------------
>An HTML attachment was scrubbed...
>URL: 
><http://lists.openstack.org/pipermail/user-committee/attachments/20150928/
>4ca53707/attachment-0001.html>
>-------------- next part --------------
>A non-text attachment was scrubbed...
>Name: smime.p7s
>Type: application/pkcs7-signature
>Size: 7349 bytes
>Desc: not available
>URL: 
><http://lists.openstack.org/pipermail/user-committee/attachments/20150928/
>4ca53707/attachment-0001.bin>
>
>------------------------------
>
>Message: 3
>Date: Mon, 28 Sep 2015 15:06:28 -0400
>From: Jonathan Proulx <jon at csail.mit.edu>
>To: Tim Bell <Tim.Bell at cern.ch>
>Cc: "user-committee at lists.openstack.org"
>	<user-committee at lists.openstack.org>
>Subject: Re: [User-committee] Per-project surveys
>Message-ID: <20150928190628.GL24467 at csail.mit.edu>
>Content-Type: text/plain; charset="us-ascii"
>
>
>I'm thinking we try (next round) to wrap this into the user survey.
>
>If we can organized the PTL surveys as optional and at the end
>hopefully we can retain the short general overview an relatively high
>response. 
>
>If the survey is arranged so PTLs can send deeper links to say 'update
>your Neuron usage survey' that go directly to that sub part we may not
>completely solve the survey over load but we can at least eliminate
>the duplication and bring all the data under the same privacy model.
>
>I'm thinking an "optional tab" at the end of the survey, after we
>thank you for your responses, with a list of links to project specific
>sections.
>
>-Jon 
>
>
>On Mon, Sep 28, 2015 at 05:03:49PM +0000, Tim Bell wrote:
>: 
>:
>:There have been a couple of project surveys of OpenStack users in the
>past
>:week (Neutron and Ceilometer).
>:
>: 
>:
>:They tend to duplicate the standard questions (industry, size of
>deployment,
>:release levels, .) and also some of the questions we are currently
>asking in
>:the survey (nova network, metering feedback). The commitment to
>anonymity is
>:also not always clear.
>:
>: 
>:
>:The difficulty I see is that we reach survey limits, where people are not
>:sure what is official and whether the questions/audience may lead to a
>bias.
>:
>: 
>:
>:On the other hand, the PTLs often want more detailed information and the
>:user survey needs to be kept to a reasonable length (to avoid we lose
>:everything because people bailed out before finishing).
>:
>: 
>:
>:Any suggestions on how to proceed ?
>:
>: 
>:
>:Tim
>:
>: 
>:
>
>
>
>:_______________________________________________
>:User-committee mailing list
>:User-committee at lists.openstack.org
>:http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee
>
>
>-- 
>-------------- next part --------------
>A non-text attachment was scrubbed...
>Name: smime.p7s
>Type: application/x-pkcs7-signature
>Size: 3659 bytes
>Desc: not available
>URL: 
><http://lists.openstack.org/pipermail/user-committee/attachments/20150928/
>5f7df03d/attachment-0001.bin>
>
>------------------------------
>
>Message: 4
>Date: Mon, 28 Sep 2015 19:14:14 +0000
>From: Bruno Morel <bmorel at internap.com>
>To: Jonathan Proulx <jon at csail.mit.edu>, Tim Bell <Tim.Bell at cern.ch>
>Cc: "user-committee at lists.openstack.org"
>	<user-committee at lists.openstack.org>
>Subject: Re: [User-committee] Per-project surveys
>Message-ID: <A21C8C21-16BA-47A2-8B53-0B813541125E at internap.com>
>Content-Type: text/plain; charset="utf-8"
>
>
>I agree with Jonathan, in detail :
>
>* making it a regular, scheduled thing : everybody in the community would
>know and expect every 6 month (or 3 month ?) to have the opportunity to
>give feedback, especially if it is pre / post cycle (ex. : January ->
>July -> January ?)
>* Making it a community wide effort and consolidated in one ?interaction?
>/ survey : having the ability to check/uncheck which project you want to
>give feedback about would be very useful and probably help focus each set
>of question to each project goals / needs
>* A main section would be, in my opinion, the right place to get all the
>general project-independant information we want to gather and any related
>question to the community efforts
>
>Bruno
>
>
>
>
>
>
>
>On 2015-09-28, 15:06, "Jonathan Proulx" <jon at csail.mit.edu> wrote:
>
>>
>>I'm thinking we try (next round) to wrap this into the user survey.
>>
>>If we can organized the PTL surveys as optional and at the end
>>hopefully we can retain the short general overview an relatively high
>>response. 
>>
>>If the survey is arranged so PTLs can send deeper links to say 'update
>>your Neuron usage survey' that go directly to that sub part we may not
>>completely solve the survey over load but we can at least eliminate
>>the duplication and bring all the data under the same privacy model.
>>
>>I'm thinking an "optional tab" at the end of the survey, after we
>>thank you for your responses, with a list of links to project specific
>>sections.
>>
>>-Jon 
>>
>>
>>On Mon, Sep 28, 2015 at 05:03:49PM +0000, Tim Bell wrote:
>>: 
>>:
>>:There have been a couple of project surveys of OpenStack users in the
>>past
>>:week (Neutron and Ceilometer).
>>:
>>: 
>>:
>>:They tend to duplicate the standard questions (industry, size of
>>deployment,
>>:release levels, .) and also some of the questions we are currently
>>asking in
>>:the survey (nova network, metering feedback). The commitment to
>>anonymity is
>>:also not always clear.
>>:
>>: 
>>:
>>:The difficulty I see is that we reach survey limits, where people are
>>not
>>:sure what is official and whether the questions/audience may lead to a
>>bias.
>>:
>>: 
>>:
>>:On the other hand, the PTLs often want more detailed information and the
>>:user survey needs to be kept to a reasonable length (to avoid we lose
>>:everything because people bailed out before finishing).
>>:
>>: 
>>:
>>:Any suggestions on how to proceed ?
>>:
>>: 
>>:
>>:Tim
>>:
>>: 
>>:
>>
>>
>>
>>:_______________________________________________
>>:User-committee mailing list
>>:User-committee at lists.openstack.org
>>:http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee
>>
>>
>>-- 
>
>------------------------------
>
>_______________________________________________
>User-committee mailing list
>User-committee at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/user-committee
>
>
>End of User-committee Digest, Vol 32, Issue 3
>*********************************************

-------------- next part --------------
A non-text attachment was scrubbed...
Name: default[1].xml
Type: application/xml
Size: 3222 bytes
Desc: default[1].xml
URL: <http://lists.openstack.org/pipermail/user-committee/attachments/20150928/c8b87b75/attachment-0001.xml>