Open Stack

On Thu, Aug 21, 2014 at 7:28 AM, Kyle Mestery <mestery at mestery.com> wrote:
> On Thu, Aug 21, 2014 at 5:12 AM, Ihar Hrachyshka <ihrachys at redhat.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA512
>>
>> On 20/08/14 18:28, Salvatore Orlando wrote:
>>> Some comments inline.
>>>
>>> Salvatore
>>>
>>> On 20 August 2014 17:38, Ihar Hrachyshka <ihrachys at redhat.com
>>> <mailto:ihrachys at redhat.com>> wrote:
>>>
>>> Hi all,
>>>
>>> I've read the proposal for incubator as described at [1], and I
>>> have several comments/concerns/suggestions to this.
>>>
>>> Overall, the idea of giving some space for experimentation that
>>> does not alienate parts of community from Neutron is good. In that
>>> way, we may relax review rules and quicken turnaround for preview
>>> features without loosing control on those features too much.
>>>
>>> Though the way it's to be implemented leaves several concerns, as
>>> follows:
>>>
>>> 1. From packaging perspective, having a separate repository and
>>> tarballs seems not optimal. As a packager, I would better deal with
>>> a single tarball instead of two. Meaning, it would be better to
>>> keep the code in the same tree.
>>>
>>> I know that we're afraid of shipping the code for which some users
>>> may expect the usual level of support and stability and
>>> compatibility. This can be solved by making it explicit that the
>>> incubated code is unsupported and used on your user's risk. 1) The
>>> experimental code wouldn't probably be installed unless explicitly
>>> requested, and 2) it would be put in a separate namespace (like
>>> 'preview', 'experimental', or 'staging', as the call it in Linux
>>> kernel world [2]).
>>>
>>> This would facilitate keeping commit history instead of loosing it
>>> during graduation.
>>>
>>> Yes, I know that people don't like to be called experimental or
>>> preview or incubator... And maybe neutron-labs repo sounds more
>>> appealing than an 'experimental' subtree in the core project.
>>> Well, there are lots of EXPERIMENTAL features in Linux kernel that
>>> we actively use (for example, btrfs is still considered
>>> experimental by Linux kernel devs, while being exposed as a
>>> supported option to RHEL7 users), so I don't see how that naming
>>> concern is significant.
>>>
>>>
>>>> I think this is the whole point of the discussion around the
>>>> incubator and the reason for which, to the best of my knowledge,
>>>> no proposal has been accepted yet.
>>>
>>
>> I wonder where discussion around the proposal is running. Is it public?
>>
> The discussion started out privately as the incubation proposal was
> put together, but it's now on the mailing list, in person, and in IRC
> meetings. Lets keep the discussion going on list now.
>

In the spirit of keeping the discussion going, I think we probably
need to iterate in practice on this idea a little bit before we can
crystallize on the policy and process for this new repo. Here are few
ideas on how we can start this iteration:

* Namespace for the new repo:
Should this be in the neutron namespace, or a completely different
namespace like "neutron labs"? Perhaps creating a separate namespace
will help the packagers to avoid issues of conflicting package owners
of the namespace.

* Dependency on Neutron (core) repository:
We would need to sort this out so that we can get UTs to run and pass
in the new repo. Can we set the dependency on Neutron milestone
releases? We already publish tar balls for the milestone releases, but
I am not sure we publish these as packages to pypi. If not could we
start doing that? With this in place, the incubator would always lag
the Neutron core by at the most one milestone release.

* Modules overlapping with the Neutron (core) repository:
We could initially start with the features that required very little
or no changes to the Neutron core, to avoid getting into the issue of
blocking on changes to the Neutron (core) repository before progress
can be made in the incubator.

* Packaging of ancillary code (CLI, Horizon, Heat):
We start by adding these as subdirectories inside each feature. The
packaging folks are going to find it difficult to package this.
However, can we start with this approach, and have a parallel
discussion on how we can evolved this strategy? Perhaps the individual
projects might decide to allow support for the Neutron incubator
features once they can actually see what goes into the incubator,
and/or other projects might also follow the incubator approach.

If we have loose consensus on the above, some of us folks who are
involved with features that are candidates for the incubator (e.g.
GBP, LBaaS), can immediately start iterating on this plan, and report
back our progress in a specified time frame.

Thanks,
~Sumit.

>>>
>>> 2. If those 'extras' are really moved into a separate repository
>>> and tarballs, this will raise questions on whether packagers even
>>> want to cope with it before graduation. When it comes to supporting
>>> another build manifest for a piece of code of unknown quality, this
>>> is not the same as just cutting part of the code into a separate
>>> experimental/labs package. So unless I'm explicitly asked to
>>> package the incubator, I wouldn't probably touch it myself. This is
>>> just too much effort (btw the same applies to moving plugins out of
>>> the tree - once it's done, distros will probably need to reconsider
>>> which plugins they really want to package; at the moment, those
>>> plugins do not require lots of time to ship them, but having ~20
>>> separate build manifests for each of them is just too hard to
>>> handle without clear incentive).
>>>
>>>
>>>> One reason instead for moving plugins out of the main tree is
>>>> allowing their maintainers to have full control over them. If
>>>> there was a way with gerrit or similars to give somebody rights
>>>> to merge code only on a subtree I probably would not even
>>>> consider the option of moving plugin and drivers away. From my
>>>> perspective it's not that I don't want them in the main tree,
>>>> it's that I don't think it's fair for core team reviewers to take
>>>> responsibility of approving code that they can't fully tests (3rd
>>>> partt CI helps, but is still far from having a decent level of
>>>> coverage).
>>>
>>
>> I agree with that. I actually think that moving vendor plugins outside
>> the main tree AND rearranging review permissions and obligations
>> should be extremely beneficial to the community. I'm totally for that
>> as quick as possible (Kilo please!) Reviewers waste their time
>> reviewing plugins that are in most cases interesting for a tiny
>> fraction of operators. Let the ones that are primarily interested in
>> good quality of that code (vendors) to drive development. And if some
>> plugins become garbage, it's bad news for specific vendors; if neutron
>> screws because of lack of concentration on core features and open
>> source plugins, everyone is doomed.
>>
>> Of course, splitting vendor plugins into separate repositories will
>> make life of packagers a bit harder, but the expected benefits from
>> such move are huge, so - screw packagers on this one. :)
>>
>> Though the way incubator is currently described in that proposal on
>> the wiki doesn't clearly imply similar benefits for the project, hence
>> concerns.
>>
> Lets not confuse the incubator with moving drivers out of tree. The
> two proposals are separate, and they are satisfying different
> requirements. I am hugely in favor of moving drivers out of the tree
> for all the reasons stated here, but that needs to be a different
> thread than the incubator thread here.
>
>>>
>>>
>>> 3. The fact that neutron-incubator is not going to maintain any
>>> stable branches for security fixes and major failures concerns me
>>> too. In downstream, we don't generally ship the latest and greatest
>>> from PyPI. Meaning, we'll need to maintain our own downstream
>>> stable branches for major fixes. [BTW we already do that for python
>>> clients.]
>>>
>>>
>>>> This is a valid point. We need to find an appropriate trade off.
>>>> My thinking was that incubated projects could be treated just
>>>> like client libraries from a branch perspective.
>>>
>>
>> Clients assume some level of backwards compatibility, so upgrades are
>> generally assumed to be smooth; and the whole point of incubator is
>> NOT providing any such guarantees, which makes updating packages to
>> the newly released version from PyPI too scary for distributions. All
>> in all, we already maintain stable client branches for RDO:
>> http://openstack.redhat.com/Clients I guess we'll need the same for
>> incubator, if we decide to package it.
>>
> The point of the incubator is NOT to maintain backwards compatibility,
> so I'm confused here. These are optional packages for which we're
> trying to get some initial feedback from operators. The packaging
> aspect makes this easier for them. But we're not guaranteeing API
> backwards compatibility, we're not releasing security fixes, etc.
> Operators running these need to be aware of this. The packaging aspect
> just makes this easier for them to consume.
>
>>>
>>>
>>> 4. Another unclear part of the proposal is that notion of keeping
>>> Horizon and client changes required for incubator features in
>>> neutron-incubator. AFAIK the repo will be governed by Neutron Core
>>> team, and I doubt the team is ready to review Horizon changes (?).
>>> I think I don't understand how we're going to handle that. Can we
>>> just postpone Horizon work till graduation?
>>>
>>>
>>>> I too do not think it's a great idea, mostly because there will
>>>> be horizon bits not shipped with horizon, and not verified by
>>>> horizon core team. I think it would be ok to have horizon support
>>>> for neutron incubator. It won't be the first time that support
>>>> for experimental features is added in horizon.
>>
>> You mean, in the main horizon tree?
>>
> I would think we'd need the horizon team to comment on any decisions
> here before we make rash decisions around what they would or would not
> take in their tree.
>
>>>
>>>
>>> 5. The wiki page says that graduation will require full test
>>> coverage. Does it mean 100% coverage in 'coverage' report? I don't
>>> think our existing code is even near that point, so maybe it's not
>>> fair to require that from graduated code.
>>>
>>>
>>>> I agree that by these standards we should take the whole neutron
>>>> and return it to incubation, or probably just chuck it in the
>>>> bin. It's not a mystery that Neutron quality is well below
>>>> integrated level but let's not diverge.
>>>
>>>> On the other hand, the fact that Neutron's code is rubbish does
>>>> not authorise the addition of further rubbish. I see this
>>>> requirement for graduation as a measure to ensure new additions
>>>> to neutron have proper quality. In the meanwhile it will be
>>>> mandatory for the neutron community to keep working on quality,
>>>> scalability and improve testing coverage. Otherwise there will be
>>>> no talk about neutron-incubator, mostly because probably there
>>>> will be no neutron.
>>>
>>
>> Of course, we should raise the bar for all the code - already merged,
>> in review, and in incubator. I just think there is no reason to make
>> those requirements different from general acceptance requirements (do
>> we have those formally defined?).
>>
>>>
>>> A separate tree would probably be reasonable if it would be
>>> governed by a separate team. But as it looks now, it's still
>>> Neutron Cores who will do the review heavy-lifting. So I wonder why
>>> not just applying different review rules for patches for core and
>>> the staging subtree.
>>>
>>>
>>>> This is a good point. As a neutron core I don't want to do the
>>>> heavy lifting there. I think we should define rules which allow
>>>> teams to iterate quickly while enabling the neutron core team to
>>>> retain some form of control on what goes in the incubator.
>>>
>>>
>>> [1]: https://wiki.openstack.org/wiki/Network/Incubator [2]:
>>> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/staging
>>>
>>>  /Ihar
>>>
>>> _______________________________________________ OpenStack-dev
>>> mailing list OpenStack-dev at lists.openstack.org
>>> <mailto:OpenStack-dev at lists.openstack.org>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>>
>>> _______________________________________________ OpenStack-dev
>>> mailing list OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
>>
>> iQEcBAEBCgAGBQJT9cYWAAoJEC5aWaUY1u57+NoH/3vjIB2e4gDzB2I1ycobIdVm
>> 08+dEpaDCFf27tcAYxe0Hcn8Xbc302vheGZf8c5kGEI89UvhtNbsfI9XXivmFPuq
>> aeuLum860XKFTD1BU/1VznjBXGp5yguzuobifOpjoS0YkDgOECkTvR2diqsj0I8K
>> N+RD25oJ0rzbXyWqidYyuziAPtTUUeeIt/3R9mBrTzaYzYxDaLqB6o1U2xlA3A7P
>> CU+tV2kj3A9pHZzlxpygq7M4kLb0iWE2jeaGPgGI67y5IheNjLRApVMLcHTsPnC9
>> ClfBHNbi4Ve5n4HEBN016aSFr9IXfN2Q+5rfaOlZ+UDtq7mc6gYTNNT4OTifd88=
>> =2j5b
>> -----END PGP SIGNATURE-----
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev