Open Stack

On 04/18/2016 10:57 AM, Michał Jastrzębski wrote:
> So I also want to stress out that shared libraries are huge pain
> during upgrades. While I'm not in favor of packages with embedded
> virtualenvs (as Matt pointed out, this has a lot of issues), having
> shared dependency pool pretty much means that you need to upgrade
> *everything* that is openstack at single run, and that is prone to
> errors, volatile and nearly impossible to rollback if something goes
> wrong. One way to address this issue is putting services in
> containers, but that is not an solution to problem at hand (56% use
> apt-get install as Graham says). Packagers have hard time keeping up
> already, if we add fairly complex logic to this (virtualenvs) we will
> probably end up with cross-compatibility hell of people not keeping up
> with changes.
> 
> That being said, in my opinion, this percentage is this high because
> that's exactly what we suggest in install docs, once we came out with
> a solution we should fix it there as well.
> 
> 
> On 18 April 2016 at 10:23, Matthew Thode <prometheanfire at gentoo.org> wrote:
>> On 04/18/2016 08:24 AM, Hayes, Graham wrote:
>>> On 18/04/2016 13:51, Sean Dague wrote:
>>>> On 04/18/2016 08:22 AM, Chris Dent wrote:
>>>>> On Mon, 18 Apr 2016, Sean Dague wrote:
>>>>>
>>>>>> So if you have strong feelings and ideas, why not get them out in email
>>>>>> now? That will help in the framing of the conversation.
>>>>>
>>>>> I won't be at summit and I feel pretty strongly about this topic, so
>>>>> I'll throw out my comments:
>>>>>
>>>>> I agree with the basic premise: In the big tent universe co-
>>>>> installability is holding us back and is a huge cost in terms of spent
>>>>> energy. In a world where service isolation is desirable and common
>>>>> (whether by virtualenv, containers, different hosts, etc) targeting an
>>>>> all-in-one install seems only to serve the purposes of all-in-one rpm-
>>>>> or deb-based installations.
>>>>>
>>>>> Many (most?) people won't be doing those kinds of installations. If all-in-
>>>>> one installations are important to the rpm- and deb- based distributions
>>>>> then _they_ should be resolving the dependency issues local to their own
>>>>> infrastructure (or realizing that it is too painful and start
>>>>> containerizing or otherwise as well).
>>>>>
>>>>> I think making these changes will help to improve and strengthen the
>>>>> boundaries and contracts between services. If not technically then
>>>>> at least socially, in the sense that the negotiations that people
>>>>> make to get things to work are about what actually matters in their
>>>>> services, not unwinding python dependencies and the like.
>>>>>
>>>>> A lot of the basics of getting this to work are already in place in
>>>>> devstack. One challenge I've run into the past is when devstack
>>>>> plugin A has made an assumption about having access to a python
>>>>> script provided by devstack plugin B, but it's not on $PATH or its
>>>>> dependencies are not in the site-packages visible to the current
>>>>> context. The solution here is to use full paths _into_ virtenvs.
>>>>
>>>> As Chris said, doing virtualenvs on the Devstack side for services is
>>>> pretty much there. The team looked at doing this last year, then stopped
>>>> due to operator feedback.
>>>>
>>>> One of the things that gets a little weird (when using devstack for
>>>> development) is if you actually want to see the impact of library
>>>> changes on the environment. As you'll need to make sure you loop and
>>>> install those libraries into every venv where they are used. This
>>>> forward reference doesn't really exist. So some tooling there will be
>>>> needed.
>>>>
>>>> Middleware that's pushed from one project into another (like Ceilometer
>>>> -> Swift) is also a funny edge case that I think get funnier here.
>>>>
>>>> Those are mostly implementation details, that probably have work
>>>> arounds, but would need people on them.
>>>>
>>>>
>>>>  From a strategic perspective this would basically make traditional Linux
>>>> Packaging of OpenStack a lot harder. That might be the right call,
>>>> because traditional Linux Packaging definitely suffers from the fact
>>>> that everything on a host needs to be upgraded at the same time. For
>>>> large installs of OpenStack (especially public cloud cases) traditional
>>>> packages are definitely less used.
>>>>
>>>> However Linux Packaging is how a lot of people get exposed to software.
>>>> The power of onboarding with apt-get / yum install is a big one.
>>>>
>>>> I've been through the ups and downs of both approaches so many times now
>>>> in my own head, I no longer have a strong preference beyond the fact
>>>> that we do one approach today, and doing a different one is effort to
>>>> make the transition.
>>>>
>>>>      -Sean
>>>>
>>>
>>> It is also worth noting that according to the OpenStack User Survey [0]
>>> 56% of deployments use "Unmodifed packages from the operating system".
>>>
>>> Granted it was a small sample size (302 responses to that question)
>>> but it is worth keeping this in mind as we talk about moving the burden
>>> to packagers.
>>>
>>> 0 -
>>> https://www.openstack.org/assets/survey/April-2016-User-Survey-Report.pdf (page
>>> 36)
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>> To add to this, I'd also note that I as a packager would likely stop
>> packaging Openstack at whatever release this goes into.  While the
>> option to package and ship a virtualenv installed to /usr/local or /opt
>> exists bundling is not something that should be supported given the
>> issues it can have (update cadence and security issues mainly).
>>
>> --
>> -- Matthew Thode (prometheanfire)
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
Well, the main issue with upgrading it all is probably doing a
db-upgrade along the way before you revert.  At least on Gentoo (where I
package) you can select the version you install for.  At the moment I
have liberty and mitaka both packaged and you can switch between.

-- 
-- Matthew Thode (prometheanfire)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160418/994c5497/attachment.pgp>