Open Stack

Monty Taylor wrote:
> As a follow up to my previous email about thinking about ceasing to use
> storyboard, I have done some work towards investigating using phabricator.
> [...]

For the record, I'll repost some of the analysis I did for mordred on this.


Main blocker: Ability to track tasks across projects/branches
-------------------------------------------------------------

One of the main features we need from a task tracker in OpenStack is the
ability to track the completion of tasks across projects and branches.
That affects vulnerability management, bug backports, and generally
cross-project work. The fact that Launchpad has a limitation there (on
the number of tasks it supports before timeouting) is issue #2 with LP.
The problem is stated once, then a number of tasks affecting different
groups are created, and the issue is closed when the last task is completed.

In Maniphest as it stands, that would translate into one of those:

1- Create a single task that affects multiple "projects"
2- Create a main task and multiple subtasks, one for each "project"

The main issue with (1) is that you can't track partial completion (i.e.
which groups have completed their task and which haven't), which is the
main goal of using a common task tracker.

The main issue with (2) is that it's difficult to see what groups are
affected from the "main" task (since it only lists task names), and the
main task is not autoclosed when the last subtask is closed (requiring
manual intervention).

In both cases, the UI allows multiple "projects" to apply to a given
task, which is great for tagging, but not so great when you want to
track tasks per affected repository and branches. Finally, there is no
concept of branches -- we would have to create one project for every
repo/branch combinations.

Finally, "projects" have no relationship between them. So even if you
associated a task to project "openstack/nova--master" you would have to
also manually associate it to project "nova" (as in openstack/nova +
openstack/python-novaclient) so that Nova team members could also find
it there. There is basically no projectgroup concept, which is issue #3
we have with Launchpad.

In summary, I think Phabricator's "projects" concept is a (really) neat
*tagging* system, with associated dashboard and all. It's just not meant
to cover multiple "projects" in the OpenStack sense. Our projects have a
few properties which make them difficult to emulate with Phabricator's
"projects": there should be only one per task, they have an affected
branch (master, juno...), and they are grouped into higher-level
structures that can be searched and browsed as well.

Phabricator seems to be sized for single separate projects, and it's
"project" concept is just a way to organize work within that single
project. It is pretty obvious when you see what happens if you click one
of their "projects": you end up in a dashboard with one item per bug.
Imagine clicking openstack/nova and landing on a dashboard with
thousands of bugs: it's just not meant to be used for that. It is
excellent for tracking a subset of tasks and organizing them, but not as
a higher-level structure.

So in order to make it work, we'd need a whole new concept added, let's
call it "repository". Each repository may have multiple branches
associated with it. When you create a task, you select a repository and
a branch (and only one). That field is clearly shown in the task name
(no need to drill down to the subtask to find which repo/branch is
actually affected by the task). Clicking on that will bring you to a
list of bugs affecting that repository. Repositories can be grouped
arbitrarily into repogroups which let you search and navigate issues in
a set of repositories, rather than force everyone to rebuild queries to
find, say, "all infra issues".

Then we could encode a rule so that when the last subtask of a repo-less
task is closed, the parent task is auto-closed. Then that /could/ all
work for task coordination for OpenStack, although it would be slightly
less convenient than Launchpad or StoryBoard (which do not require you
to drill down to update subtasks, and keep the discussion in one single
place).


Could do a lot better: CLI/API
------------------------------

One of the main pain points with Launchpad is its partial API.
Phabricator comes with the "arc" tool, but it is designed to drive the
DVCS git-like featureset. There is no CLI for Maniphest, only a shortcut
to make raw Conduit (API) calls. The Conduit API itself looks
mostly undocumented and (2) "making Conduit more robust isn't currently
a major project priority because there isn't much demand for it outside
of internal scripts".

We rely on a lot of scripting to interface with Launchpad, which is why
a complete and clean API was such a significant part in StoryBoard
design. It looks like the Maniphest story on APIs is actually worse than
Launchpad.


Could do better: Tracking feature development
---------------------------------------------

A feature is a complex task for which you want specific rules to apply
(like tasks may only affect master branch). It's good to visually
separate them too, so that feature-frozen period clearly excludes them.
Communicating the % of feature completion is also valuable information.
Being able to order subtasks is a nice add too.

Maniphest has no concept of task types, although it could be emulated
with a "project" (we'd tag all features with a "feature" project), or we
could use a custom field for that. It doesn't have a concept of feature
completion status either. Ideally that would be derived from the number
of completed subtasks, or set by the user on the top task. The absence
of a story (or task group) might bite us here (since in Maniphest
everything is a task, and completion only makes sense on a task group).
Also you can't order subtasks in Maniphest at all.

Not sure how to solve this one, and improvements around feature tracking
are the #1 reason for StoryBoard to exist. But one could argue that it
won't be worse than using Launchpad anyway.


Could do better: ACLs for Vulnerability management
--------------------------------------------------

Vulnerability management is about the ability to file a report that is
originally only seen by its reporter and a specific team. Then we
gradually add people to the ACL as we pull them in to help with
resolution, up to the point where the bug is open.

While Maniphest has task ACL support, it is pretty crude. There seem to
be no way to encourage people to file security reports privately (they
have to find out and make use of the "Visible To" and "Editable By"
dropboxes by themselves). Then there is no way short of defining a
"Custom policy" for a user to say ("me and the VMT"). Then there is no
way (short of expanding the custom policy) for the VMT to add
individuals to the ACL.

For this one to work, we'd need a way to encourage people to file
vulnerabilities the right way (some sort of task creation template, or
could be a separate website for vulnerability reports), then a way to
make the bug visible/editable to "VMT and all subscribers". That way
people we add to the CC end up being in the ACL without having to play
with Custom Policies. A visual indicator of such bugs would also be nice
(embargoed issues should not be discussed openly, and sometimes it's
hard to remember the thing you are on is one of those restricted issues).


We'd probably work around the limitation: Release reporting
-----------------------------------------------------------

One of the things we use Launchpad for is to produce release pages
listing what features and bugfixes landed in a given release tag. There
is no support for that (at all) in Maniphest. So we'd have to switch to
git-log generated changelogs, but that may not be a crazy thing to do
anyway. Release reporting is quite separate from task tracking,
Launchpad just accidentally did both.


Hope this helps,

-- 
Thierry Carrez (ttx)