<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Aug 23, 2014 at 11:02 AM, Anne Gentle <span dir="ltr"><<a href="mailto:anne@openstack.org" target="_blank">anne@openstack.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">
<div class="">On Fri, Aug 22, 2014 at 6:17 PM, Rochelle.RochelleGrober <span dir="ltr"><<a href="mailto:rochelle.grober@huawei.com" target="_blank">rochelle.grober@huawei.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">/flame-on<br>
Ok, this is funny to some of us in the community. The general populace of this community is so against the idea of management that they will use the term for a despotic dictator as a position name rather than "manager". Sorry, but this needed to be said.<br>
/flame-off<br>
<br>
Specific comments in line:<br>
<div><br>
Thierry Carrez wrote:<br>
><br>
> Hi everyone,<br>
><br>
> We all know being a project PTL is an extremely busy job. That's<br>
> because<br>
> in our structure the PTL is responsible for almost everything in a<br>
> project:<br>
><br>
> - Release management contact<br>
> - Work prioritization<br>
> - Keeping bugs under control<br>
> - Communicate about work being planned or done<br>
> - Make sure the gate is not broken<br>
> - Team logistics (run meetings, organize sprints)<br>
> - ...<br>
><br>
<br>
</div>Point of clarification: I've heard PTL=Project Technical Lead and PTL=Program Technical Lead. Which is it? It is kind of important as OpenStack grows, because the first is responsible for *a* project, and the second is responsible for all projects within a program.<br>
<br></blockquote><div><br></div></div><div>Now Program, formerly Project.</div></div></div></div></blockquote><div><br></div><div>I think this is worthy of more exploration. Our docs seem to be very inconsistent about what a PTL is - and more broadly, what the difference is between a Project and a Program.<br>
<br>Just a few examples:</div><div><br></div><div><a href="https://wiki.openstack.org/wiki/PTLguide">https://wiki.openstack.org/wiki/PTLguide</a> says "Program Technical Lead". <a href="https://wiki.openstack.org/wiki/PTL_Elections_March/April_2014">https://wiki.openstack.org/wiki/PTL_Elections_March/April_2014</a> simply says PTL - but does say that each PTL is elected by/for a Program. However, Thierry pointed to <a href="https://wiki.openstack.org/wiki/Governance/Foundation/Structure">https://wiki.openstack.org/wiki/Governance/Foundation/Structure</a> which still refers to Project Technical Leads and says explicitly that they lead individual projects, not programs. I actually have edit access to that page, so I could at least update that with a simple "s/Project/Program/", if I was sure that was the right thing to do.</div>
<div><br><a href="http://www.openstack.org/">http://www.openstack.org/</a> has a link in the bottom nav that says "Projects"; it points to <a href="http://www.openstack.org/projects/">http://www.openstack.org/projects/</a> which redirects to <a href="http://www.openstack.org/software/">http://www.openstack.org/software/</a> which has a list of things like "Compute" and "Storage" - which as far as I know are Programs, not Projects. I don't know how to update that link in the nav panel.<br>
<br>I wasn't around when the original Programs/Projects discussion was happening - which, I suspect, has a lot to do with why I'm confused today - it seems as though people who were around at the time understand the difference, but people who have joined since then are relying on multiple conflicting verbal definitions. I believe, though, that <a href="http://lists.openstack.org/pipermail/openstack-dev/2013-June/010821.html">http://lists.openstack.org/pipermail/openstack-dev/2013-June/010821.html</a> was one of the earliest starting points of the discussion. That page points at <a href="https://wiki.openstack.org/wiki/Projects">https://wiki.openstack.org/wiki/Projects</a>, which today contains a list of Programs. That page does have a definition of what a Program is, but doesn't explain what a Project is or how they relate to Programs. This page seems to be locked down, so I can't edit it.<br>
<br>That page does mention projects, once. The context makes it read, to me, as though a program can follow one process to "become part of OpenStack" and then another process to "become an Integrated project and part of the OpenStack coordinated release" - when my understanding of reality is that the second process applies to Projects, not Programs.<br>
<br>I've tried to find any other page that talks about what a Project is and how they relate to Programs, but I haven't been able to find anything. Perhaps there's some definition locked up in a mailing list thread or some TC minutes, but I haven't been able to find it.<br>
<br>During the previous megathread, I got the feeling that at least some of the differing viewpoints we saw were possibly down to some people thinking of a PTL as responsible for just one project, while others think of a PTL as being responsible for any projects that might fit within a Program's scope. As we approach the next PTL elections, I think it would be helpful for us to recap the discussions that led to the Program/Project split and make sure our docs are consistent, so that people who weren't following the discussion this time last year can still be clear what they're voting for.</div>
<div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra">
<div class="gmail_quote"><div><div class="h5"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
I'd also like to set out as an example of a Program that is growing to encompass multiple projects, the Neutron Program. Look at how it is expanding:<br>
<br>
Multiple sub-teams for: LBAAS, DNAAS, GBP, etc. This model could be extended such that:<br>
- the subteam is responsible for code reviews, including the first +2 for design, architecture and code of the sub-project, always also keeping an eye out that the sub-project code continues to both integrate well with the program, and that the program continues to provide the needed code bits, architecture modifications and improvements, etc. to support the sub-project.<br>
- the final +2/A would be from the Program reviewers to ensure that all integrate nicely together into a single, cohesive program.<br>
- This would allow sub-projects to have core reviewers, along with the program and be a good separation of duties. It would also help to increase the number of reviews moving to merged code.<br>
- Taken to a logical stepping stone, you would have project technical leads for each project, and they would make up a program council, with the program technical lead being the chair of the council.<br>
<br>
This is a way to offload a good chunk of PTL tactical responsibilities and help them focus more on the strategic.<br>
<div><div><br>
> They end up being completely drowned in those day-to-day operational<br>
> duties, miss the big picture, can't help in development that much<br>
> anymore, get burnt out. Since you're either "the PTL" or "not the PTL",<br>
> you're very alone and succession planning is not working that great<br>
> either.<br>
><br>
> There have been a number of experiments to solve that problem. John<br>
> Garbutt has done an incredible job at helping successive Nova PTLs<br>
> handling the release management aspect. Tracy Jones took over Nova bug<br>
> management. Doug Hellmann successfully introduced the concept of Oslo<br>
> liaisons to get clear point of contacts for Oslo library adoption in<br>
> projects. It may be time to generalize that solution.<br>
><br>
> The issue is one of responsibility: the PTL is ultimately responsible<br>
> for everything in a project. If we can more formally delegate that<br>
> responsibility, we can avoid getting up to the PTL for everything, we<br>
> can rely on a team of people rather than just one person.<br>
><br>
> Enter the Czar system: each project should have a number of liaisons /<br>
> official contacts / delegates that are fully responsible to cover one<br>
> aspect of the project. We need to have Bugs czars, which are<br>
> responsible<br>
> for getting bugs under control. We need to have Oslo czars, which serve<br>
> as liaisons for the Oslo program but also as active project-local oslo<br>
> advocates. We need Security czars, which the VMT can go to to progress<br>
> quickly on plugging vulnerabilities. We need release management czars,<br>
> to handle the communication and process with that painful OpenStack<br>
> release manager. We need Gate czars to serve as first-line-of-contact<br>
> getting gate issues fixed... You get the idea.<br>
><br>
</div></div>/flame-on<br>
Let's call spades, spades here. Czar is not only overkill, but the wrong metaphor.<br>
/flame-off<br></blockquote><div><br></div></div></div><div>I'm with Rocky on the anti-czar-as-a-word camp. We all like clever names to shed the "corporate" stigma but this word ain't it. Liaison or lead?<div>
<br></div>
<div>Also wanted to point to <a href="https://wiki.openstack.org/wiki/PTLguide" target="_blank">https://wiki.openstack.org/wiki/PTLguide</a> as it's quite nice.</div></div><div><br></div><div>I think PTLs tend to find help when they absolutely are ready to fall over, and I'm all for a plan that helps us not fall over. I've had people step up for bug triaging, gate work, tests, and oslo, sometimes one person did three or four at once. I certainly can't do it all for cross-project. Based on what I've seen, I doubt that we can add this much formality to this across 20+ programs. It's the "bigger more integrated project" vs. "smaller more focused project" difference that won't let us do a pattern here. We can certainly document the responsibilities and let programs know there are some best practices.</div>
<span class=""><font color="#888888">
<div><br></div><div>Anne </div></font></span><div><div class="h5">
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Each position suggested here exists in corporate development projects:<br>
- Bug czar == bug manager/administrator/QA engineer/whatever - someone in charge of making sure bugs get triages, assigned and completed<br>
- Oslo czar == systems engineers/project managers who make sure that the project is in line with the rest of the projects that together make an integrated release. This position needs to stretch beyond just Oslo to encompass all the cross-project requirements and will likely be its own "committee"<br>
- Gate Czar == integration engineer(manager)/QA engineer(manager)/build-release engineer. This position would also likely be a liaison to Infra.<br>
- Security Czar == security guru (that name takes me back ;-)<br>
- Release management Czar == Project release manager<br>
- Doc Czar == tech editor<br>
- Tempest Czar == QA engineer(manager)<br>
<br>
Yes, programs are now mostly big enough to require coordination and management. The roles are long defined, so let's just document the definitions and get volunteers. Each project would be able to decide which roles needed to be filled and whether individuals are capable of filling multiple roles.<br>
<br>
These roles need to be transparent in a governance sense and should be increasing communication between projects, community and contributors.<br>
<div><div><br>
> Some people can be czars of multiple areas. PTLs can retain some czar<br>
> activity if they wish. Czars can collaborate with their equivalents in<br>
> other projects to share best practices. We just need a clear list of<br>
> areas/duties and make sure each project has a name assigned to each.<br>
><br>
> Now, why czars ? Why not rely on informal activity ? Well, for that<br>
> system to work we'll need a lot of people to step up and sign up for<br>
> more responsibility. Making them "czars" makes sure that effort is<br>
> recognized and gives them something back. Also if we don't formally<br>
> designate people, we can't really delegate and the PTL will still be<br>
> directly held responsible. The Release management czar should be able<br>
> to<br>
> sign off release SHAs without asking the PTL. The czars and the PTL<br>
> should collectively be the new "project drivers".<br>
><br>
> At that point, why not also get rid of the PTL ? And replace him with a<br>
> team of czars ? If the czar system is successful, the PTL should be<br>
> freed from the day-to-day operational duties and will be able to focus<br>
> on the project health again. We still need someone to keep an eye on<br>
> the<br>
> project-wide picture and coordinate the work of the czars. We need<br>
> someone to pick czars, in the event multiple candidates sign up. We<br>
> also<br>
> still need someone to have the final say in case of deadlocked issues.<br>
><br>
> People say we don't have that many deadlocks in OpenStack for which the<br>
> PTL ultimate power is needed, so we could get rid of them. I'd argue<br>
> that the main reason we don't have that many deadlocks in OpenStack is<br>
> precisely *because* we have a system to break them if they arise. That<br>
> encourages everyone to find a lazy consensus. That part of the PTL job<br>
> works. Let's fix the part that doesn't work (scaling/burnout).<br>
><br>
<br>
</div></div>Perhaps, the crux of the issue with PTLs is that we still need PTLs, but they should be *Project* Technical Leads and the role currently defined as PTL should really become a Program Manager - someone who coordinates all the projects, their issues, needs, communications and leave the deep technical work to the project leads and individual developers. Consider the project technical leads seeing the trees and the program manager sees the forest. Let the program manager gather the technical requirements and wishes for the next release and track those, let the team of project technical leads determine the design, architecture, tools, etc.<br>
<div><br>
> --<br>
> Thierry Carrez (ttx)<br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</div>Great idea Thierry, just thought I'd propose a slight twist on your concept.<br>
And sorry for the flames, but I'm in Jay Pipe's camp when it comes to needing to ensure inclusiveness, openness and transparency for the community and I think even the name czars are a slippery slope to something less open.<br>
<br>
--Rocky Grober<br>
<div><div><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div></div></div><br></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>