Open Stack

On 04/03/2015 11:16 AM, Thierry Carrez wrote:
> Monty Taylor wrote:
>> As a follow up to my previous email about thinking about ceasing to use
>> storyboard, I have done some work towards investigating using phabricator.
>> [...]
> 
> For the record, I'll repost some of the analysis I did for mordred on this.
> 
> 
> Main blocker: Ability to track tasks across projects/branches
> -------------------------------------------------------------
> 
> One of the main features we need from a task tracker in OpenStack is the
> ability to track the completion of tasks across projects and branches.
> That affects vulnerability management, bug backports, and generally
> cross-project work. The fact that Launchpad has a limitation there (on
> the number of tasks it supports before timeouting) is issue #2 with LP.
> The problem is stated once, then a number of tasks affecting different
> groups are created, and the issue is closed when the last task is completed.
> 
> In Maniphest as it stands, that would translate into one of those:
> 
> 1- Create a single task that affects multiple "projects"
> 2- Create a main task and multiple subtasks, one for each "project"
> 
> The main issue with (1) is that you can't track partial completion (i.e.
> which groups have completed their task and which haven't), which is the
> main goal of using a common task tracker.
> 
> The main issue with (2) is that it's difficult to see what groups are
> affected from the "main" task (since it only lists task names), and the
> main task is not autoclosed when the last subtask is closed (requiring
> manual intervention).

I think 2 is the way phabricator expects this to be modeled. It's
certainly different, but it also gives us a view of a thing we've NEVER
had that I honestly think might be more useful to more people. If you
look at:

https://secure.phabricator.com/T1536

You can see quite quickly which tasks are done and which are still
blocking this main task from being done. That those aren't broken out by
project there is actually a little nice to me - we are "One Project"
after all.

Looking at how wikipedia are using the feature as well may be
enlightening. I believe they similarly have multiple projects as well as
multiple vertical efforts.

In fact, one of the things I particularly like is that it's NOT tied to
repos or branches ... because things like "product management" have
interests that are particularly clunky when we're tied to a thinking
everything is related to a git repository... as is thinking that having
multiple git repositories means multiple bug projects. The mapping is
actually potentially different given the project.

> In both cases, the UI allows multiple "projects" to apply to a given
> task, which is great for tagging, but not so great when you want to
> track tasks per affected repository and branches. Finally, there is no
> concept of branches -- we would have to create one project for every
> repo/branch combinations.

I'm not so sure about this. I actually think we could create a "nova"
project and a "stable/kilo" project. Or we may need a nova-kilo - I
don't know - but I think there are some iterations we should investigate.

> Finally, "projects" have no relationship between them. So even if you
> associated a task to project "openstack/nova--master" you would have to
> also manually associate it to project "nova" (as in openstack/nova +
> openstack/python-novaclient) so that Nova team members could also find
> it there. There is basically no projectgroup concept, which is issue #3
> we have with Launchpad.
> 
> In summary, I think Phabricator's "projects" concept is a (really) neat
> *tagging* system, with associated dashboard and all. 

Yes. It is, in fact, explicitly a tagging system.

> It's just not meant
> to cover multiple "projects" in the OpenStack sense. Our projects have a
> few properties which make them difficult to emulate with Phabricator's
> "projects": there should be only one per task, they have an affected
> branch (master, juno...), and they are grouped into higher-level
> structures that can be searched and browsed as well.
> 
> Phabricator seems to be sized for single separate projects, and it's
> "project" concept is just a way to organize work within that single
> project. It is pretty obvious when you see what happens if you click one
> of their "projects": you end up in a dashboard with one item per bug.
> Imagine clicking openstack/nova and landing on a dashboard with
> thousands of bugs: it's just not meant to be used for that. It is
> excellent for tracking a subset of tasks and organizing them, but not as
> a higher-level structure.
> 
> So in order to make it work, we'd need a whole new concept added, let's
> call it "repository". Each repository may have multiple branches
> associated with it. When you create a task, you select a repository and
> a branch (and only one). That field is clearly shown in the task name
> (no need to drill down to the subtask to find which repo/branch is
> actually affected by the task). Clicking on that will bring you to a
> list of bugs affecting that repository. Repositories can be grouped
> arbitrarily into repogroups which let you search and navigate issues in
> a set of repositories, rather than force everyone to rebuild queries to
> find, say, "all infra issues".

Although I pretty much defer to you in all things related to this and
always have - I'd like to repeat what I said in my other email ...
namely, I think this is very explicitly about not having a perfect
system but realizing that there are always tradeoffs.

We have, for 5 years, been using Launchpad Blueprints, which have almost
no value other than overhead. So I'm not comparing this to what
storyboard might be in 2 years, I'm comparing it to what we have right now.

> Then we could encode a rule so that when the last subtask of a repo-less
> task is closed, the parent task is auto-closed. Then that /could/ all
> work for task coordination for OpenStack, although it would be slightly
> less convenient than Launchpad or StoryBoard (which do not require you
> to drill down to update subtasks, and keep the discussion in one single
> place).

Yes - I think a thing that autocloses parent tasks when all blocking
tasks could be easy enough to accomplish. However, that would undercut
the value of having blocks/blocked-on relationships between issues,
which is a thing we've never had. I'd rather suggest that someone
probably should own the thing we think of as a top-level task instead.
If we auto-close, then as soon as the task I'm working on that is
blocked by another task which is not a sub-task in the launchpad sense
closes, my bug would get closed - which is not the case - I've simply
been waiting on it so that I could work.

> 
> Could do a lot better: CLI/API
> ------------------------------
> 
> One of the main pain points with Launchpad is its partial API.
> Phabricator comes with the "arc" tool, but it is designed to drive the
> DVCS git-like featureset. There is no CLI for Maniphest, only a shortcut
> to make raw Conduit (API) calls. The Conduit API itself looks
> mostly undocumented and (2) "making Conduit more robust isn't currently
> a major project priority because there isn't much demand for it outside
> of internal scripts".

Since they call out not-much-demand - I betcha that we could get
traction here. Also - there is a python library that I used to interact
with bugs in phabricator.

> We rely on a lot of scripting to interface with Launchpad, which is why
> a complete and clean API was such a significant part in StoryBoard
> design. It looks like the Maniphest story on APIs is actually worse than
> Launchpad.
> 
> 
> Could do better: Tracking feature development
> ---------------------------------------------
> 
> A feature is a complex task for which you want specific rules to apply
> (like tasks may only affect master branch). It's good to visually
> separate them too, so that feature-frozen period clearly excludes them.
> Communicating the % of feature completion is also valuable information.
> Being able to order subtasks is a nice add too.
> 
> Maniphest has no concept of task types, although it could be emulated
> with a "project" (we'd tag all features with a "feature" project), or we
> could use a custom field for that. It doesn't have a concept of feature
> completion status either. Ideally that would be derived from the number
> of completed subtasks, or set by the user on the top task. The absence
> of a story (or task group) might bite us here (since in Maniphest
> everything is a task, and completion only makes sense on a task group).
> Also you can't order subtasks in Maniphest at all.
> 
> Not sure how to solve this one, and improvements around feature tracking
> are the #1 reason for StoryBoard to exist. But one could argue that it
> won't be worse than using Launchpad anyway.

I would argue that we would be substantially improved over launchpad,
given that blueprints is so useless.

> Could do better: ACLs for Vulnerability management
> --------------------------------------------------
> 
> Vulnerability management is about the ability to file a report that is
> originally only seen by its reporter and a specific team. Then we
> gradually add people to the ACL as we pull them in to help with
> resolution, up to the point where the bug is open.
> 
> While Maniphest has task ACL support, it is pretty crude. There seem to
> be no way to encourage people to file security reports privately (they
> have to find out and make use of the "Visible To" and "Editable By"
> dropboxes by themselves). Then there is no way short of defining a
> "Custom policy" for a user to say ("me and the VMT"). Then there is no
> way (short of expanding the custom policy) for the VMT to add
> individuals to the ACL.
> 
> For this one to work, we'd need a way to encourage people to file
> vulnerabilities the right way (some sort of task creation template, or
> could be a separate website for vulnerability reports), then a way to
> make the bug visible/editable to "VMT and all subscribers". That way
> people we add to the CC end up being in the ACL without having to play
> with Custom Policies. A visual indicator of such bugs would also be nice
> (embargoed issues should not be discussed openly, and sometimes it's
> hard to remember the thing you are on is one of those restricted issues).

I'd love to learn how wikimedia is working with this.

> 
> We'd probably work around the limitation: Release reporting
> -----------------------------------------------------------
> 
> One of the things we use Launchpad for is to produce release pages
> listing what features and bugfixes landed in a given release tag. There
> is no support for that (at all) in Maniphest. So we'd have to switch to
> git-log generated changelogs, but that may not be a crazy thing to do
> anyway. Release reporting is quite separate from task tracking,
> Launchpad just accidentally did both.

Thanks for diving in! I realize that I'm explicitly requesting that you
think about tools that by definition mean giving up on elements of
design of the thing that you do, and that sucks. I hope that my push
back comes across as an argument that we could potentially rethink a few
things and not fall apart rather than a suggestion that the things you
want in life are somehow invalid.