<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<base href="x-msg://4638/">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<!--?xml version="1.0" encoding="UTF-8" standalone="no"?--><span class="Apple-style-span" style="font-family: Arial; ">Hi Rob, Mark and All,</span>
<div style="font-family: Arial; "><br>
</div>
<span class="Apple-style-span" style="font-family: Arial; ">I hear you. This will be good information to have. But, practically speaking, it's going to be difficult to collect. </span>
<div><font class="Apple-style-span" face="Arial"><br>
</font></div>
<div><font class="Apple-style-span" face="Arial">
<div style="font-family: Arial; ">Just because it's a challenge doesn't mean we shouldn't try. We can attack this thing on two fronts.</div>
<div style="font-family: Arial; "><br>
</div>
<div style="font-family: Arial; ">A. Add the question(s) of what API calls people are using to the survey. Just need to figure out how to resolve #1 and #2 below.</div>
<div>
<div><br>
</div>
<div>B. Get aggregate API call data from companies willing to share it. Is there any such effort already underway to collect this kind of data by the Foundation?</div>
<div><br>
</div>
<div>There are a number of issues that stand out to me but before getting into them some quick definitions to be clear what I'm talking about.</div>
<div style="font-family: Arial; "><br>
</div>
</div>
</font></div>
<div>
<div><font class="Apple-style-span" face="Arial">OpenStack developer = developer working on OpenStack itself</font></div>
<div><font class="Apple-style-span" face="Arial">OpenStack operator = operator deploying/maintaining an OpenStack cloud</font></div>
<div><font class="Apple-style-span" face="Arial">application developer = developer working on application being deployed on an OpenStack cloud</font></div>
<div><font class="Apple-style-span" face="Arial">application operator = operator deploying/maintaining an application on an OpenStack cloud</font></div>
<div><font class="Apple-style-span" face="Arial">users = any of OpenStack operator, application developer, application operator<br>
</font>
<div style="font-family: Arial; "><br>
</div>
<span class="Apple-style-span" style="font-family: Arial; ">1. The breadth and depth of API calls.</span></div>
<div><span class="Apple-style-span" style="font-family: Arial; "><br>
</span><span class="Apple-style-span" style="font-family: Arial; ">Displaying all of the API calls possible on screen for the survey would be a challenge in itself. It would probably have to be generated and presented to the user in a hierarchical layout. I
have trouble believing very many users would be wiling to drill down through the layers to check the methods they use. Maybe I'm being overly skeptical here.</span></div>
<div><font class="Apple-style-span" face="Arial"><br>
</font></div>
<div><font class="Apple-style-span" face="Arial">Presenting a subset of the API calls could work but I'm not sure how to pick the subset. This is a tricky one. Would love to hear suggestions.</font>
<div style="font-family: Arial; "><br>
</div>
<div style="font-family: Arial; ">2. Application developers actually taking the survey.</div>
<div style="font-family: Arial; "><br>
</div>
<div style="font-family: Arial; ">Currently the survey is targeted towards OpenStack operators. Not to say we can't switch the focus but it'll be a bit more difficult to get application developers/operators in front of it just because they're a bit more removed
from OpenStack. We could ask the OpenStack operators to ask their application developers/operators to take it and give them an email template for them to send it out or something along those lines.</div>
<div style="font-family: Arial; "><br>
</div>
<div style="font-family: Arial; ">Perhaps it's time to open the survey with a "What kind of user are you?" type question.</div>
<div style="font-family: Arial; "><br>
</div>
<span class="Apple-style-span" style="font-family: Arial; ">3. Application developers only knowing their single system</span></div>
<div><font class="Apple-style-span" face="Arial"><br>
</font></div>
<div><font class="Apple-style-span" face="Arial">The feedback from an app dev is useful but they really only know about their own system. What we really need is an aggregate of all of the API calls being made.<br>
</font>
<div style="font-family: Arial; "><br>
</div>
<span class="Apple-style-span" style="font-family: Arial; ">4. The difference between application developers and application operators</span></div>
<div><font class="Apple-style-span" face="Arial"><br>
</font></div>
<div><font class="Apple-style-span" face="Arial">They are going to have different needs and make different API calls. An app dev might rely heavily on Marconi and never call Nova and vice versa for an app operator. Again we really need aggregate numbers.<br>
</font>
<div style="font-family: Arial; "><br>
</div>
<div style="font-family: Arial; ">To get real aggregate numbers the best thing to do is to go the logs. To me this appears to be a daunting challenge. Hurdles include: getting companies to agree to allow this and share their aggregates, logging every API call
(is this already done?), logging configuration, aggregation (the logs could be spread over many systems or, hopefully, centralized), aggregation from archived logs, etc. This is where B from above comes in.</div>
<div style="font-family: Arial; "><br>
</div>
<div style="font-family: Arial; ">Let me know what you think.</div>
<div style="font-family: Arial; "><br>
</div>
<div style="font-family: Arial; ">Thanks,</div>
<div style="font-family: Arial; ">Everett</div>
<div style="font-family: Arial; "><br>
</div>
<div><font class="Apple-style-span" face="Arial"><br>
</font>
<div>
<div>On Jan 20, 2014, at 12:15 AM, <<a href="mailto:Rob_Hirschfeld@Dell.com">Rob_Hirschfeld@Dell.com</a>></div>
<div> <<a href="mailto:Rob_Hirschfeld@Dell.com">Rob_Hirschfeld@Dell.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; ">
<div lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1" style="page: WordSection1; ">
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Mark & Everett,<o:p></o:p></span></div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">I agree that there’s good potential alignment w/ the DefCore data needs. It would be very interesting to know which API calls are required by different client libraries.
That would help identify classes of tests for review. We’ve already captured that as a desired criteria for weighting but I don’t know how we’re doing to determine that on a test-by-test basis!<o:p></o:p></span></div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); ">Rob<o:p></o:p></span></div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125); "><o:p> </o:p></span></div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<a name="_____replyseparator"></a><b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; ">From:</span></b><span style="font-size: 10pt; font-family: Tahoma, sans-serif; "><span class="Apple-converted-space"> </span>Mark Collier [mailto:mark@collierclan.net]<span class="Apple-converted-space"> </span><br>
<b>Sent:</b><span class="Apple-converted-space"> </span>Friday, January 17, 2014 4:28 PM<br>
<b>To:</b><span class="Apple-converted-space"> </span>Everett Toews<br>
<b>Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a>;
<a href="mailto:user-committee@lists.openstack.org">user-committee@lists.openstack.org</a><br>
<b>Subject:</b><span class="Apple-converted-space"> </span>Re: [Openstack] Formulate application developer oriented questions for the user survey<br>
<b>Importance:</b><span class="Apple-converted-space"> </span>Low<o:p></o:p></span></div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<o:p> </o:p></div>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
Thanks for bringing up the opportunity to expand the survey to gather feedback from end-users (of the API...) Everett. <o:p></o:p></div>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<o:p> </o:p></div>
</div>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
I think this is a great idea, and I was also thinking that as we make progress on the interop efforts with things like DefCore[1][2][3] (and RefStack) that Rob Hirshfeld & others are leading, it's critical that we get input from this class of "user". For example,
it would be ideal IMHO to identify which APIs (and underlying functions) are most valued, to inform the list of tests included and ultimately the ones that "must pass". <o:p></o:p></div>
</div>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<o:p> </o:p></div>
</div>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
Today there is an optional component in the survey for "deployment" which obviously fits the operator class of "user", so perhaps there is a similar path with more detailed question for the devs targeting the APIs (i.e. deploying apps on openstack clouds, for
lack of a better phrase) that includes some kind of feedback mechanism for individual APIs (don't care /nice to have/ must have) as well as related issues like api discoverability.<o:p></o:p></div>
</div>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<o:p> </o:p></div>
</div>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
[1] <a href="http://robhirschfeld.com/category/clouds/openstack/defcore/" style="color: blue; text-decoration: underline; ">http://robhirschfeld.com/category/clouds/openstack/defcore/</a><o:p></o:p></div>
</div>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
[2]<span class="Apple-converted-space"> </span><a href="https://wiki.openstack.org/wiki/Governance/DefCoreCommittee" style="color: blue; text-decoration: underline; ">https://wiki.openstack.org/wiki/Governance/DefCoreCommittee</a><o:p></o:p></div>
</div>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
[3] <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee" style="color: blue; text-decoration: underline; ">http://lists.openstack.org/cgi-bin/mailman/listinfo/defcore-committee</a><o:p></o:p></div>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 12pt; ">
<o:p> </o:p></p>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
On Fri, Jan 17, 2014 at 2:09 PM, Everett Toews <<a href="mailto:everett.toews@rackspace.com" target="_blank" style="color: blue; text-decoration: underline; ">everett.toews@rackspace.com</a>> wrote:<o:p></o:p></div>
<div>
<p class="MsoNormal" style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 12pt; ">
On Jan 17, 2014, at 5:45 AM, Gregor von Laszewski wrote:<br>
<br>
> Everett:<br>
><br>
> you may want to add a question such as<br>
><br>
> * “Why did you use this library?”<o:p></o:p></p>
</div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
That's a good point. One of the things that concerns me about how the questions are worded right now is that they are very reactionary. That's great for finding out the current state of thing but does nothing to inform us on where users want to go in the future
and how we can help them get there.<br>
<br>
They're more "What are you using right now?" not "What will you need in the future?"<br>
<br>
Asking why will also help reveal requirements. Personally I need to think on this a bit more and take another crack at the questions later.<o:p></o:p></div>
<div>
<p class="MsoNormal" style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 12pt; ">
<br>
> This may give an additional insight and possibly motivation for further actions/development. Some examples<br>
><br>
> a) in Python the use of libcloud via the EC2 is popular. Why?: compatibility<br>
><br>
> b) in the cloudmesh project we developed our own compatibility library that makes use of the native openstack protocol instead of using lib cloud/EC2 to access openstack. Why: (1) libcloud/EC2 has limited functionality, (2) debugging of production clouds
with native protocols (starting thousands of vms), (3) easier integration into user interfaces while leveraging JSON. (4) Together this allows us to have an API that accesses and manages VMS on AWS, Azure, and Openstack the same way but uses in case of Openstack
the native protocol instead of lib cloud/EC2.<o:p></o:p></p>
</div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
Just out of curiosity, why didn't you contribute to the libcloud project to fill in the missing functionality rather than start your own?<br>
<br>
Thanks,<br>
Everett<o:p></o:p></div>
<div>
<div>
<div style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
<br>
<br>
_______________________________________________<br>
Mailing list:<span class="Apple-converted-space"> </span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank" style="color: blue; text-decoration: underline; ">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to :<span class="Apple-converted-space"> </span><a href="mailto:openstack@lists.openstack.org" style="color: blue; text-decoration: underline; ">openstack@lists.openstack.org</a><br>
Unsubscribe :<span class="Apple-converted-space"> </span><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank" style="color: blue; text-decoration: underline; ">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><o:p></o:p></div>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-right: 0in; margin-left: 0in; font-size: 12pt; font-family: 'Times New Roman', serif; margin-top: 0in; margin-bottom: 0.0001pt; ">
</p>
</div>
</div>
</div>
</span></blockquote>
</div>
<br>
</div>
</div>
</div>
</body>
</html>