Open Stack

On 04/18/2016 01:40 PM, Doug Hellmann wrote:
> Excerpts from Matthew Thode's message of 2016-04-18 13:22:38 -0500:
>> On 04/18/2016 12:33 PM, Doug Hellmann wrote:
>>> Excerpts from Matthew Thode's message of 2016-04-18 10:23:37 -0500:
>>>> On 04/18/2016 08:24 AM, Hayes, Graham wrote:
>>>>> On 18/04/2016 13:51, Sean Dague wrote:
>>>>>> On 04/18/2016 08:22 AM, Chris Dent wrote:
>>>>>>> On Mon, 18 Apr 2016, Sean Dague wrote:
>>>>>>>
>>>>>>>> So if you have strong feelings and ideas, why not get them out in email
>>>>>>>> now? That will help in the framing of the conversation.
>>>>>>>
>>>>>>> I won't be at summit and I feel pretty strongly about this topic, so
>>>>>>> I'll throw out my comments:
>>>>>>>
>>>>>>> I agree with the basic premise: In the big tent universe co-
>>>>>>> installability is holding us back and is a huge cost in terms of spent
>>>>>>> energy. In a world where service isolation is desirable and common
>>>>>>> (whether by virtualenv, containers, different hosts, etc) targeting an
>>>>>>> all-in-one install seems only to serve the purposes of all-in-one rpm-
>>>>>>> or deb-based installations.
>>>>>>>
>>>>>>> Many (most?) people won't be doing those kinds of installations. If all-in-
>>>>>>> one installations are important to the rpm- and deb- based distributions
>>>>>>> then _they_ should be resolving the dependency issues local to their own
>>>>>>> infrastructure (or realizing that it is too painful and start
>>>>>>> containerizing or otherwise as well).
>>>>>>>
>>>>>>> I think making these changes will help to improve and strengthen the
>>>>>>> boundaries and contracts between services. If not technically then
>>>>>>> at least socially, in the sense that the negotiations that people
>>>>>>> make to get things to work are about what actually matters in their
>>>>>>> services, not unwinding python dependencies and the like.
>>>>>>>
>>>>>>> A lot of the basics of getting this to work are already in place in
>>>>>>> devstack. One challenge I've run into the past is when devstack
>>>>>>> plugin A has made an assumption about having access to a python
>>>>>>> script provided by devstack plugin B, but it's not on $PATH or its
>>>>>>> dependencies are not in the site-packages visible to the current
>>>>>>> context. The solution here is to use full paths _into_ virtenvs.
>>>>>>
>>>>>> As Chris said, doing virtualenvs on the Devstack side for services is
>>>>>> pretty much there. The team looked at doing this last year, then stopped
>>>>>> due to operator feedback.
>>>>>>
>>>>>> One of the things that gets a little weird (when using devstack for
>>>>>> development) is if you actually want to see the impact of library
>>>>>> changes on the environment. As you'll need to make sure you loop and
>>>>>> install those libraries into every venv where they are used. This
>>>>>> forward reference doesn't really exist. So some tooling there will be
>>>>>> needed.
>>>>>>
>>>>>> Middleware that's pushed from one project into another (like Ceilometer
>>>>>> -> Swift) is also a funny edge case that I think get funnier here.
>>>>>>
>>>>>> Those are mostly implementation details, that probably have work
>>>>>> arounds, but would need people on them.
>>>>>>
>>>>>>
>>>>>>  From a strategic perspective this would basically make traditional Linux
>>>>>> Packaging of OpenStack a lot harder. That might be the right call,
>>>>>> because traditional Linux Packaging definitely suffers from the fact
>>>>>> that everything on a host needs to be upgraded at the same time. For
>>>>>> large installs of OpenStack (especially public cloud cases) traditional
>>>>>> packages are definitely less used.
>>>>>>
>>>>>> However Linux Packaging is how a lot of people get exposed to software.
>>>>>> The power of onboarding with apt-get / yum install is a big one.
>>>>>>
>>>>>> I've been through the ups and downs of both approaches so many times now
>>>>>> in my own head, I no longer have a strong preference beyond the fact
>>>>>> that we do one approach today, and doing a different one is effort to
>>>>>> make the transition.
>>>>>>
>>>>>>     -Sean
>>>>>>
>>>>>
>>>>> It is also worth noting that according to the OpenStack User Survey [0]
>>>>> 56% of deployments use "Unmodifed packages from the operating system".
>>>>>
>>>>> Granted it was a small sample size (302 responses to that question)
>>>>> but it is worth keeping this in mind as we talk about moving the burden
>>>>> to packagers.
>>>>>
>>>>> 0 - 
>>>>> https://www.openstack.org/assets/survey/April-2016-User-Survey-Report.pdf (page 
>>>>> 36)
>>>>>
>>>>> __________________________________________________________________________
>>>>> OpenStack Development Mailing List (not for usage questions)
>>>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>>
>>>> To add to this, I'd also note that I as a packager would likely stop
>>>> packaging Openstack at whatever release this goes into.  While the
>>>> option to package and ship a virtualenv installed to /usr/local or /opt
>>>> exists bundling is not something that should be supported given the
>>>> issues it can have (update cadence and security issues mainly).
>>>
>>> That's a useful data point, but it comes across as a threat and I'm
>>> having trouble taking it as a constructive comment.
>>>
>>> Can you truly not imagine any other useful way to package OpenStack
>>> other than individual packages with shared dependencies that would
>>> be acceptable?
>>>
>>> Doug
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>> As Sean stated, I didn't mean to come across as a threat, but it is the
>> most likely outcome of doing this.  Yes, I could technically package a
>> venv but then I'd be concerned about security issues that come with
>> bundling.  Also, at least specifically to gentoo, while we can bundle
>> software and install that it's highly looked down upon.
>>
>> Some of this is specific to shared system libraries (c and the like) but
>> here's a couple of links.
>>
>> https://wiki.gentoo.org/wiki/Why_not_bundle_dependencies
>> https://blog.flameeyes.eu/2009/01/bundling-libraries-for-despair-and-insecurity
> 
> Some folks seem to be conflating "upstream wants to stop worrying
> about co-installability" with "upstream wants us to bundle
> dependencies". That's not what I'm proposing.
> 
> I think I've said that one outcome I would be happy with is to have
> more people helping us with the dependency management.  That doesn't
> solve all of our issues upstream, but it does improve the situation.
> Another alternate that might work is for downstream folks to do
> their own dependency management. That's less optimal, and may not
> change the outcome for Gentoo or Debian, where the downstream teams
> are small (1 person each, I think?).
> 
> I started the discussion to solicit ideas, but I would prefer to
> avoid proposing what downstream should do because (a) I'm sure
> folks want options and (b) I'm sure I'm not the person to identify
> those options.
> 
> Doug
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
Ya, I'd be happy to work more with upstream.  I already review the
stable-reqs updates and watch them for the stable branches I package
for.  Not sure what else is needed.

-- 
-- Matthew Thode (prometheanfire)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160418/a30d3d8a/attachment.pgp>