<font size=2 face="sans-serif">Generally speaking as a track chair my
experience has been the same as described below, we've used spreadsheets
to try to balance opinion and shape things appropriately.</font>
<br>
<br><font size=2 face="sans-serif">When I was at HP we had a massive internal
conference, each submission is reviewed by four people, who score it according
to various requirements. Each reviewer is also required to write a paragraph
of feedback for each paper, there's regularly thousands of submissions
for this conference and it would basically take a week of effort from every
reviewer to deal with their load. Reviewers were broken into groups and
divisions, each with their own leaders who dedicated even more time to
the project. Abstracts are also typically 2-3 pages of varying quality
academic content.</font>
<br>
<br><font size=2 face="sans-serif">It's an awesome conference that I always
enjoyed being a part of, however I imagine there must have been at least
50 reviewers and 15 or so group chairs etc, each putting in at least a
working week of effort which makes a conservative estimate of the effort
involved around 2535 man hours (65*38) - I don't think that the OpenStack
community could sustain similar levels of effort every 6 months. At the
moment I'm guessing being a track chair on Security, a smaller track, takes
up maybe 10-15 hours of my time - read through all the submissions a couple
of times, score them, do background reading, stack-rank, discuss with other
track chairs, reach out to authors etc.</font>
<br>
<br><font size=2 face="sans-serif">The process of putting together a track
is additive, we are not rejecting talks because they're bad necessarily,
they just didn't fit into the overall delivery of a compelling, well balanced
track. I think the OpenStack community, TC, Summit organizers and Track
Chairs are actually doing a pretty good job. The tool is much enhanced
over previous years and I'm sure it'll continue to be improved.</font>
<br>
<br><font size=2 face="sans-serif">-Rob</font>
<br>
<br>
<br>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Nick Chase <nchase@mirantis.com></font>
<br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">Matt Fischer <matt@mattfischer.com></font>
<br><font size=1 color=#5f5f5f face="sans-serif">Cc:
</font><font size=1 face="sans-serif">"openstack-track-chairs@lists.openstack.org"
<openstack-track-chairs@lists.openstack.org></font>
<br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">25/08/2016 21:45</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">Re: [Openstack-track-chairs]
Track Chair Feedback</font>
<br>
<hr noshade>
<br>
<br>
<br>
<br><font size=3>On 8/25/2016 4:28 PM, Matt Fischer wrote:</font>
<br><font size=3>I hope that the track chairs were not only selected on
the basis of technical knowledge but on their integrity and of putting
the summit & community ahead of company goals. I also think that the
foundation has done a good job ensuring that different communities and
companies are chosen, which hopefully avoids the ability of one company
to push their talks over others. For me, I will never push a team talk
unless the other track chairs have it under consideration, and if they
do I'm happy to provide honest feedback on it. I hope fellow track chairs
will concur with that.</font>
<br><font size=3><br>
Exactly!!<br>
</font>
<br><font size=3>Opening up a requirement that track chairs have to justify
why they chose one talk over another will only foster further argument,
it won't resolve anything. It's also untenable when rejecting 200 talks
to justify the ranking of each one. And regardless of what you do and whether
you justify your reasons or not, many good talks were not picked, and some
feelings were probably hurt.</font>
<br><font size=3><br>
I'm not suggesting that we explain why talks were rejected, just why they
were SELECTED.<br>
<br>
Feelings are always going to be hurt. I've already gotten 3 phone
calls from dejected colleagues who got their "sorry, no thanks"
letter and I've had to tell them, "Look, you're in good company."
Hell, I didn't get in either. :)<br>
<br>
But while we've definitely been improving in the "transparency"
department, I think it wouldn't be a bad thing to go further.<br>
<br>
---- Nick<br>
</font>
<br>
<br><font size=3>On Thu, Aug 25, 2016 at 4:13 PM, Nick Chase <</font><a href=mailto:nchase@mirantis.com target=_blank><font size=3 color=blue><u>nchase@mirantis.com</u></font></a><font size=3>>
wrote:</font>
<br><font size=3>Well, when we were working on our track (both this time
and for Austin) what we did was break it down into sections (for example,
we might have 3 talks on "DevOps") and then try to choose a talk
from each section. In some cases, we had to choose between similar
talks, and discuss what our rationale was; it wouldn't be a bad thing to
provide a place for track chairs to explain their choices. It would
even provide a way for submitters to improve their submissions for next
time.</font>
<p><font size=3>---- Nick</font>
<p>
<br><font size=3>On 8/25/2016 3:50 PM, Shilla Saebi wrote:</font>
<br><font size=3>I think we should come up with a way for track chairs
who have to make selections on talks that were submitted by members of
their own company or team to be fair. I noticed a few talks go through
where the track chairs worked for the same company as the talks that were
selected, and other talks with more complete abstracts (in my opinion)
and a more diverse set of speakers, was not selected. </font>
<br>
<br><font size=3>On Thu, Aug 25, 2016 at 2:18 PM, Nathan C Ziemann <</font><a href=mailto:ziemann@us.ibm.com target=_blank><font size=3 color=blue><u>ziemann@us.ibm.com</u></font></a><font size=3>>
wrote:</font>
<br><font size=3>+1 on export. </font>
<br>
<br><font size=3>We wrapped up our final selections using a spreadsheet
which makes it easy to evaluate multiple criteria and chair votes/comments
in one view. <br>
<br>
Nate Ziemann </font>
<br><font size=3>Sent from my iPhone</font>
<br><font size=3><br>
On Aug 25, 2016, at 9:54 AM, Nick Chase <</font><a href=mailto:nchase@mirantis.com target=_blank><font size=3 color=blue><u>nchase@mirantis.com</u></font></a><font size=3>>
wrote:<br>
</font>
<br><font size=3>Actually the last two summits tracks I've worked with
have finished up in spreadsheets, where we've organized submissions by
topic and marked off who voted for each. It really made the final
processing easier. </font>
<br>
<br><font size=3>I'd love to see a simple "export" function so
we don't wind up doing it by hand.</font>
<br>
<br><font size=3>---- Nick<br>
<br>
On Thursday, August 25, 2016, Amrith Kumar <</font><a href=mailto:amrith@tesora.com target=_blank><font size=3 color=blue><u>amrith@tesora.com</u></font></a><font size=3>>
wrote:</font>
<br><font size=2 color=#004080>On the whole, I think the process and the
tool were fine, there were some hiccups that you have already identified.</font>
<p><font size=2 color=#004080> </font>
<p><font size=2 color=#004080>I found the user interface (specifically
the two pane format and the sizing requirement) to be a little troublesome.
Eventually I just grabbed the json off the interface and populated a spreadsheet
and that worked out much easier.</font>
<p><font size=2 color=#004080> </font>
<p><font size=2 color=#004080>Why was that?</font>
<p><font size=2 color=#004080> </font>
<p><font size=2 color=#004080>I wanted to tag each of the 93 talks with
a bunch of tags that I found to be useful, things like the names of the
projects covered, sales-pitch, customer-story, performance, … and so on,
and later filter talks based on those tags.</font>
<p><font size=2 color=#004080> </font>
<p><font size=2 color=#004080>So, if the next iteration of the tool had
a way in which each chair could tag talks using an arbitrary (private)
set of tags, that would be nifty.</font>
<p><font size=2 color=#004080> </font>
<p><font size=2 color=#004080>-amrith </font>
<p><font size=2 color=#004080> </font>
<p><font size=2 face="Calibri"><b>From:</b> Kunzmann, Gerald [</font><a href="mailto:kunzmann@docomolab-euro.com"><font size=2 face="Calibri">mailto:kunzmann@docomolab-euro.com</font></a><font size=2 face="Calibri">]
<b><br>
Sent:</b> Thursday, August 25, 2016 3:22 AM<b><br>
To:</b> openstack-track-chairs@lists.openstack.org<b><br>
Subject:</b> Re: [Openstack-track-chairs] Track Chair Feedback</font>
<p><font size=3> </font>
<p><font size=2 color=#004080 face="Calibri">+1 nice tool.</font>
<p><font size=2 color=#004080 face="Calibri"> </font>
<p><font size=2 color=#004080 face="Calibri">Would be great to have a sort
function e.g. by “community vote”.</font>
<p><font size=2 color=#004080 face="Calibri"> </font>
<p><font size=2 color=#004080 face="Calibri">Gerald</font>
<p><font size=2 color=#004080 face="Calibri"> </font>
<p><font size=2 face="Calibri"><b>From:</b> Travis McPeak [</font><a href=mailto:travis.mcpeak@gmail.com><font size=2 face="Calibri">mailto:travis.mcpeak@gmail.com</font></a><font size=2 face="Calibri">]
<b><br>
Sent:</b> Donnerstag, 25. August 2016 02:02<b><br>
To:</b> Christopher Aedo <doc@aedo.net>; openstack-track-chairs@lists.openstack.org<b><br>
Subject:</b> Re: [Openstack-track-chairs] Track Chair Feedback</font>
<p><font size=3> </font>
<p><font size=3>+1 great tool and easy process overall.</font>
<p><font size=3> </font>
<p><font size=3 face="Times New Roman">Thanks!</font>
<p><font size=3>On Wed, Aug 24, 2016, 4:47 PM Christopher Aedo <doc@aedo.net>
wrote:</font>
<br><font size=3>Thank you for all the hard work you've put into this tool
(and all the<br>
other OpenStack bits you're always improving.) This was my second<br>
time around and I thought the tool worked great - looking forward to<br>
seeing even more improvements for Boston!<br>
<br>
Thank you Jimmy!<br>
<br>
-Christopher<br>
<br>
On Wed, Aug 24, 2016 at 3:20 PM, Jimmy McArthur <jimmy@openstack.org>
wrote:<br>
> All -<br>
><br>
> Thanks again for your hard (and quick!) work on the OpenStack Barcelona<br>
> Summit Track selections. You all did an amazing job and we're really<br>
> grateful to each of you.<br>
><br>
> I took notes of feedback, but please feel free to add to the list
if you<br>
> feel I've missed something:<br>
><br>
> Better ability for Track Chairs to merge<br>
><br>
> Option to email all speakers from one or more presentations<br>
> Ability for Speakers to submit/propose a new Title / Abstract as an<br>
> attachment or within the thread<br>
> Track Chairs ability to deactivate merged talks, and save one with
new<br>
> abstract (or merge into a new record so we don't taint the originals)<br>
><br>
> Email issues<br>
><br>
> Track conversations with users within the Track Chair tool i/o just
via<br>
> email. Could also consider using ZenDesk API for this?<br>
> There was a problem with Sendgrid API this go round, so there were
just<br>
> generally problems. They were:<br>
><br>
> No CC of other track chairs<br>
> Reply address was noreply@openstack.org i/o the ZenDesk ticket<br>
> Adding name of Presentation and other details automatically within
the email<br>
> "Track chair XYZ has a question about your presentation: ABCD"<br>
> Email all speakers i/o just one<br>
><br>
> Need the option back of being able to compare everyone's list side
by side<br>
><br>
> The new method of using the dropdown makes it difficult for the team
to<br>
> compare selected talks<br>
> A way to auto-select common talks for placement in the team section
would be<br>
> cool, or at least a way to quickly view that list, along with the
# of team<br>
> members that selected each talk<br>
><br>
> An option for a Track to "Close" their selections<br>
><br>
> Send an email to track chairs list<br>
> Mark Track as "Closed" for non-admins<br>
><br>
> Another suggestion for the Boston CFP process might be to consider
separate<br>
> user and developer sub-tracks. For example. What might be considered
a How<br>
> To/Best Practice for developers might be completely irrelevant to
users.<br>
> We at Dev Tools track found out that there are a lot of drafts related
to<br>
> the CI/CD and Testing processes. Seems reasonable to add such a track<br>
> (Deployment and Testing) as an option for the next summit. What do
you<br>
> think?<br>
><br>
><br>
> Thank you again!<br>
> Jimmy<br>
><br>
> _______________________________________________<br>
> Openstack-track-chairs mailing list<br>
> Openstack-track-chairs@lists.openstack.org<br>
> </font><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-track-chairs" target=_blank><font size=3 color=blue><u>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-track-chairs</u></font></a><font size=3><br>
><br>
<br>
_______________________________________________<br>
Openstack-track-chairs mailing list<br>
Openstack-track-chairs@lists.openstack.org</font><font size=3 color=blue><u><br>
</u></font><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-track-chairs" target=_blank><font size=3 color=blue><u>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-track-chairs</u></font></a>
<p>
<br><font size=3><br>
_______________________________________________<br>
Openstack-track-chairs mailing list</font><font size=3 color=blue><u><br>
</u></font><a href="mailto:Openstack-track-chairs@lists.openstack.org" target=_blank><font size=3 color=blue><u>Openstack-track-chairs@lists.openstack.org</u></font></a><font size=3 color=blue><u><br>
</u></font><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-track-chairs" target=_blank><font size=3 color=blue><u>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-track-chairs</u></font></a><font size=3><br>
</font>
<br>
<br>
<br><font size=3 color=#8f8f8f>-- <br>
Nick Chase, Head of Technical and Marketing Content, Mirantis<br>
Editor in Chief, OpenStack:Unlocked</font>
<br><font size=3><br>
_______________________________________________<br>
Openstack-track-chairs mailing list</font><font size=3 color=blue><u><br>
</u></font><a href="mailto:Openstack-track-chairs@lists.openstack.org"><font size=3 color=blue><u>Openstack-track-chairs@lists.openstack.org</u></font></a><font size=3 color=blue><u><br>
</u></font><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-track-chairs" target=_blank><font size=3 color=blue><u>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-track-chairs</u></font></a><font size=3><br>
</font>
<br>
<br>
<br><font size=3>-- <br>
Nick Chase, Head of Technical and Marketing Content, Mirantis<br>
Editor in Chief, OpenStack:Unlocked</font><tt><font size=2>_______________________________________________<br>
Openstack-track-chairs mailing list<br>
Openstack-track-chairs@lists.openstack.org<br>
</font></tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-track-chairs"><tt><font size=2>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-track-chairs</font></tt></a><tt><font size=2><br>
</font></tt>
<br>
<br><font size=2 face="sans-serif"><br>
Unless stated otherwise above:<br>
IBM United Kingdom Limited - Registered in England and Wales with number
741598. <br>
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6
3AU<br>
</font>