[openstack-dev] [requirements][lbaas] gunicorn to g-r

Davanum Srinivas davanum at gmail.com
Wed Oct 19 15:38:26 UTC 2016


Adam,

One distinction i heard was that this is just a runtime dependency,
hence my suggestion.

Thanks,
Dims

On Wed, Oct 19, 2016 at 11:00 AM, Adam Harwell <flux.adam at gmail.com> wrote:
> Dims: that wasn't meant as hostile to you, though re-reading it kind of
> sounds that way.
> You were not the first in this thread to suggest bindep, and while your
> links are useful, I don't think it makes a lot of sense for our use case. I
> legitimately can't understand why this *one* dependency (not anything a
> deployer will need to install on their control-plane instances) is suggested
> as a binary dependency, when it is a python module that we include in our
> code just like *everything else* in our requirements file.
>
> On Wed, Oct 19, 2016 at 11:02 PM Adam Harwell <flux.adam at gmail.com> wrote:
>>
>> We literally install every other dependency from pypi with
>> requirements.txt, so I'm struggling understand why all the sudden we need to
>> install this one as a binary, for our devstack specific script, when we are
>> planning a move to a distro that doesn't even support binary packages?
>> Should we switch our entire requirements file to bindep? If not, what makes
>> this different?
>>
>>
>> On Wed, Oct 19, 2016, 22:56 Davanum Srinivas <davanum at gmail.com> wrote:
>>>
>>> Adam,
>>>
>>> Have you see this yet?
>>>
>>>
>>> http://docs.openstack.org/infra/bindep/readme.html#writing-requirements-files
>>>
>>> http://codesearch.openstack.org/?q=platform&i=nope&files=bindep.txt&repos=
>>>
>>> Thanks,
>>> Dims
>>>
>>> On Wed, Oct 19, 2016 at 9:40 AM, Adam Harwell <flux.adam at gmail.com>
>>> wrote:
>>> > Yes, but we need to use SOMETHING for our own devstack gate tests --
>>> > maybe
>>> > it is easier to think of our devstack code as a "third party setup",
>>> > and
>>> > that it uses gunicorn for its DIB images (but not every deployer needs
>>> > to).
>>> > In this case, how do we include it? Devstack needs it to run our gate
>>> > jobs,
>>> > which means it has to be in our main codebase, but deployers don't
>>> > necessarily need it for their deployments (though it is the default
>>> > option).
>>> > Do we include it in global-requirements or not? How do we use it in
>>> > devstack
>>> > if it is not in global-requirements? We don't install it as a binary
>>> > because
>>> > the plan is to stay completely distro-independant (or target a distro
>>> > that
>>> > doesn't even HAVE binary packages like cirros). Originally I just put
>>> > the
>>> > line "pip install gunicorn>=19.0" directly in our DIB script, but was
>>> > told
>>> > that was a dirty hack, and that it should be in requirements.txt like
>>> > everything else. I'm not sure I agree, and it seems like maybe others
>>> > are
>>> > suggesting I go back to that method?
>>> >
>>> >  --Adam
>>> >
>>> > On Wed, Oct 19, 2016 at 10:19 PM Hayes, Graham <graham.hayes at hpe.com>
>>> > wrote:
>>> >>
>>> >> On 18/10/2016 19:57, Doug Wiegley wrote:
>>> >> >
>>> >> >> On Oct 18, 2016, at 12:42 PM, Doug Hellmann <doug at doughellmann.com>
>>> >> >> wrote:
>>> >> >>
>>> >> >> Excerpts from Doug Wiegley's message of 2016-10-18 12:21:20 -0600:
>>> >> >>>
>>> >> >>>> On Oct 18, 2016, at 12:10 PM, Doug Hellmann
>>> >> >>>> <doug at doughellmann.com>
>>> >> >>>> wrote:
>>> >> >>>>
>>> >> >>>> Excerpts from Doug Wiegley's message of 2016-10-18 12:00:35
>>> >> >>>> -0600:
>>> >> >>>>>
>>> >> >>>>>> On Oct 18, 2016, at 11:30 AM, Doug Hellmann
>>> >> >>>>>> <doug at doughellmann.com
>>> >> >>>>>> <mailto:doug at doughellmann.com>> wrote:
>>> >> >>>>>>
>>> >> >>>>>> Excerpts from Doug Wiegley's message of 2016-10-18 09:59:54
>>> >> >>>>>> -0600:
>>> >> >>>>>>>
>>> >> >>>>>>>> On Oct 18, 2016, at 5:14 AM, Ian Cordasco
>>> >> >>>>>>>> <sigmavirus24 at gmail.com
>>> >> >>>>>>>> <mailto:sigmavirus24 at gmail.com>
>>> >> >>>>>>>> <mailto:sigmavirus24 at gmail.com
>>> >> >>>>>>>> <mailto:sigmavirus24 at gmail.com>>> wrote:
>>> >> >>>>>>>>
>>> >> >>>>>>>>
>>> >> >>>>>>>>
>>> >> >>>>>>>> -----Original Message-----
>>> >> >>>>>>>> From: Thierry Carrez <thierry at openstack.org
>>> >> >>>>>>>> <mailto:thierry at openstack.org> <mailto:thierry at openstack.org
>>> >> >>>>>>>> <mailto:thierry at openstack.org>> <mailto:thierry at openstack.org
>>> >> >>>>>>>> <mailto:thierry at openstack.org> <mailto:thierry at openstack.org
>>> >> >>>>>>>> <mailto:thierry at openstack.org>>>>
>>> >> >>>>>>>> Reply: OpenStack Development Mailing List (not for usage
>>> >> >>>>>>>> questions) <openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>>>
>>> >> >>>>>>>> Date: October 18, 2016 at 03:55:41
>>> >> >>>>>>>> To: openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>>
>>> >> >>>>>>>> <openstack-dev at lists.openstack.org
>>> >> >>>>>>>>
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org><mailto:openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>>> >> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>>>
>>> >> >>>>>>>> Subject:  Re: [openstack-dev] [requirements][lbaas] gunicorn
>>> >> >>>>>>>> to
>>> >> >>>>>>>> g-r
>>> >> >>>>>>>>
>>> >> >>>>>>>>> Doug Wiegley wrote:
>>> >> >>>>>>>>>> [...] Paths forward:
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> 1. Add gunicorn to global requirements.
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> 2. Create a project specific “amphora-requirements.txt”
>>> >> >>>>>>>>>> file
>>> >> >>>>>>>>>> for the
>>> >> >>>>>>>>>> service VM packages (this is actually my preference.) It
>>> >> >>>>>>>>>> has
>>> >> >>>>>>>>>> been
>>> >> >>>>>>>>>> pointed out that this wouldn’t be kept up-to-date by the
>>> >> >>>>>>>>>> bot.
>>> >> >>>>>>>>>> We could
>>> >> >>>>>>>>>> modify the bot to include it in some way, or do it
>>> >> >>>>>>>>>> manually, or
>>> >> >>>>>>>>>> with a
>>> >> >>>>>>>>>> project specific job.
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> 3. Split our service VM builds into another repo, to keep a
>>> >> >>>>>>>>>> clean
>>> >> >>>>>>>>>> separation between API services and the backend. But, even
>>> >> >>>>>>>>>> this
>>> >> >>>>>>>>>> new
>>> >> >>>>>>>>>> repo’s standlone requirements.txt file will have the g-r
>>> >> >>>>>>>>>> issue
>>> >> >>>>>>>>>> from #1.
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> 4. Boot the backend out of OpenStack entirely.
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> All those options sound valid to me, so the requirements
>>> >> >>>>>>>>> team
>>> >> >>>>>>>>> should
>>> >> >>>>>>>>> pick what they are the most comfortable with.
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> My 2c: yes g-r is mostly about runtime dependencies and
>>> >> >>>>>>>>> ensuring
>>> >> >>>>>>>>> co-installability. However it also includes test/build-time
>>> >> >>>>>>>>> deps, and
>>> >> >>>>>>>>> generally converging dependencies overall sounds like a
>>> >> >>>>>>>>> valid
>>> >> >>>>>>>>> goal. Is
>>> >> >>>>>>>>> there any drawback in adding gunicorn to g-r (option 1) ?
>>> >> >>>>>>>>
>>> >> >>>>>>>> The drawback (in my mind) is that new projects might start
>>> >> >>>>>>>> using
>>> >> >>>>>>>> it giving operators yet another thing to learn about when
>>> >> >>>>>>>> deploying a new
>>> >> >>>>>>>> component (eventlet, gevent, gunicorn, ...).
>>> >> >>>>>>>>
>>> >> >>>>>>>> On the flip, what's the benefit of adding it to g-r?
>>> >> >>>>>>>
>>> >> >>>>>>> The positive benefit is the same as Octavia’s use case: it
>>> >> >>>>>>> provides an alternative for any non-frontline-api service to
>>> >> >>>>>>> run a
>>> >> >>>>>>> lightweight http/wsgi service as needed (service VMs, health
>>> >> >>>>>>> monitor agents,
>>> >> >>>>>>> etc). And something better than the built-in debug servers in
>>> >> >>>>>>> most of the
>>> >> >>>>>>> frameworks.
>>> >> >>>>>>>
>>> >> >>>>>>> On the proliferation point, it is certainly a risk, though
>>> >> >>>>>>> I’ve
>>> >> >>>>>>> personally heard pretty strong guidance that all main API
>>> >> >>>>>>> services in our
>>> >> >>>>>>> community should be trending towards pecan.
>>> >> >>>>>>
>>> >> >>>>>> Pecan is a way to build WSGI applications. Gunicorn is a way to
>>> >> >>>>>> deploy
>>> >> >>>>>> them. So they're not mutually exclusive.
>>> >> >>>>>
>>> >> >>>>> Right, agreed.
>>> >> >>>>>
>>> >> >>>>> What we’re trying to convey here is:
>>> >> >>>>>
>>> >> >>>>> - The normal way of making a REST endpoint in OpenStack is to
>>> >> >>>>> use
>>> >> >>>>> pecan (or flask or falcon), and let the deployer or packager
>>> >> >>>>> worry about the
>>> >> >>>>> runtime wsgi and/or reverse proxy.
>>> >> >>>>>
>>> >> >>>>> - This isn't a “normal” OpenStack endpoint, because it’s a
>>> >> >>>>> microservice inside a service VM, and thus has different needs,
>>> >> >>>>> and is much
>>> >> >>>>> less likely to be customized by a non-dev, to boot. And it needs
>>> >> >>>>> to be
>>> >> >>>>> “deploy ready” as a simple matter of it being a service VM black
>>> >> >>>>> box. It’s
>>> >> >>>>> part of the data plane, not the control plane.
>>> >> >>>>>
>>> >> >>>>> Thanks,
>>> >> >>>>> doug
>>> >> >>>>
>>> >> >>>> That all seems reasonable.
>>> >> >>>>
>>> >> >>>> We have a proliferation of these service VMs. It would be good to
>>> >> >>>> get some of the folks involved together to start a working group
>>> >> >>>> to see if there are some commonalities that can lead to shared
>>> >> >>>> processes or tools.
>>> >> >>>
>>> >> >>> That’s a good idea. I wonder if we can organize something in time
>>> >> >>> for
>>> >> >>> next week. I don’t think we should wait to make forward progress
>>> >> >>> for that,
>>> >> >>> but there is definitely some commonality we should be defining and
>>> >> >>> striving
>>> >> >>> towards.
>>> >> >>>
>>> >> >>> doug
>>> >> >>
>>> >> >> Sure, and I hope I didn't come across as implying that this work
>>> >> >> should
>>> >> >> wait.
>>> >> >>
>>> >> >> I expect you could take over a corner of the dev lounge or some
>>> >> >> other space to hold a BoF to at least start the discussion and get
>>> >> >> some interested folks lined up to lead the WG.
>>> >> >
>>> >> > Sounds good. In prep to sending an ML invite, what projects use
>>> >> > service
>>> >> > VMs?   There’s Octavia, Trove, and Tacker.  What else?
>>> >>
>>> >> It is in our (long term) road map for Designate as well.
>>> >>
>>> >> > Thanks,
>>> >> > doug
>>> >> >
>>> >> >>
>>> >> >> Doug
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> __________________________________________________________________________
>>> >> >> OpenStack Development Mailing List (not for usage questions)
>>> >> >> Unsubscribe:
>>> >> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> >> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > __________________________________________________________________________
>>> >> > OpenStack Development Mailing List (not for usage questions)
>>> >> > Unsubscribe:
>>> >> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> __________________________________________________________________________
>>> >> OpenStack Development Mailing List (not for usage questions)
>>> >> Unsubscribe:
>>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>> >
>>> >
>>> > __________________________________________________________________________
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe:
>>> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> >
>>>
>>>
>>>
>>> --
>>> Davanum Srinivas :: https://twitter.com/dims
>>>
>>>
>>> __________________________________________________________________________
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Davanum Srinivas :: https://twitter.com/dims



More information about the OpenStack-dev mailing list