[openstack-dev] [requirements][lbaas] gunicorn to g-r

Davanum Srinivas davanum at gmail.com
Wed Oct 19 13:49:51 UTC 2016


Adam,

Have you see this yet?

http://docs.openstack.org/infra/bindep/readme.html#writing-requirements-files
http://codesearch.openstack.org/?q=platform&i=nope&files=bindep.txt&repos=

Thanks,
Dims

On Wed, Oct 19, 2016 at 9:40 AM, Adam Harwell <flux.adam at gmail.com> wrote:
> Yes, but we need to use SOMETHING for our own devstack gate tests -- maybe
> it is easier to think of our devstack code as a "third party setup", and
> that it uses gunicorn for its DIB images (but not every deployer needs to).
> In this case, how do we include it? Devstack needs it to run our gate jobs,
> which means it has to be in our main codebase, but deployers don't
> necessarily need it for their deployments (though it is the default option).
> Do we include it in global-requirements or not? How do we use it in devstack
> if it is not in global-requirements? We don't install it as a binary because
> the plan is to stay completely distro-independant (or target a distro that
> doesn't even HAVE binary packages like cirros). Originally I just put the
> line "pip install gunicorn>=19.0" directly in our DIB script, but was told
> that was a dirty hack, and that it should be in requirements.txt like
> everything else. I'm not sure I agree, and it seems like maybe others are
> suggesting I go back to that method?
>
>  --Adam
>
> On Wed, Oct 19, 2016 at 10:19 PM Hayes, Graham <graham.hayes at hpe.com> wrote:
>>
>> On 18/10/2016 19:57, Doug Wiegley wrote:
>> >
>> >> On Oct 18, 2016, at 12:42 PM, Doug Hellmann <doug at doughellmann.com>
>> >> wrote:
>> >>
>> >> Excerpts from Doug Wiegley's message of 2016-10-18 12:21:20 -0600:
>> >>>
>> >>>> On Oct 18, 2016, at 12:10 PM, Doug Hellmann <doug at doughellmann.com>
>> >>>> wrote:
>> >>>>
>> >>>> Excerpts from Doug Wiegley's message of 2016-10-18 12:00:35 -0600:
>> >>>>>
>> >>>>>> On Oct 18, 2016, at 11:30 AM, Doug Hellmann <doug at doughellmann.com
>> >>>>>> <mailto:doug at doughellmann.com>> wrote:
>> >>>>>>
>> >>>>>> Excerpts from Doug Wiegley's message of 2016-10-18 09:59:54 -0600:
>> >>>>>>>
>> >>>>>>>> On Oct 18, 2016, at 5:14 AM, Ian Cordasco <sigmavirus24 at gmail.com
>> >>>>>>>> <mailto:sigmavirus24 at gmail.com> <mailto:sigmavirus24 at gmail.com
>> >>>>>>>> <mailto:sigmavirus24 at gmail.com>>> wrote:
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> -----Original Message-----
>> >>>>>>>> From: Thierry Carrez <thierry at openstack.org
>> >>>>>>>> <mailto:thierry at openstack.org> <mailto:thierry at openstack.org
>> >>>>>>>> <mailto:thierry at openstack.org>> <mailto:thierry at openstack.org
>> >>>>>>>> <mailto:thierry at openstack.org> <mailto:thierry at openstack.org
>> >>>>>>>> <mailto:thierry at openstack.org>>>>
>> >>>>>>>> Reply: OpenStack Development Mailing List (not for usage
>> >>>>>>>> questions) <openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>>>
>> >>>>>>>> Date: October 18, 2016 at 03:55:41
>> >>>>>>>> To: openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>>
>> >>>>>>>> <openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org><mailto:openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org
>> >>>>>>>> <mailto:openstack-dev at lists.openstack.org>>>>
>> >>>>>>>> Subject:  Re: [openstack-dev] [requirements][lbaas] gunicorn to
>> >>>>>>>> g-r
>> >>>>>>>>
>> >>>>>>>>> Doug Wiegley wrote:
>> >>>>>>>>>> [...] Paths forward:
>> >>>>>>>>>>
>> >>>>>>>>>> 1. Add gunicorn to global requirements.
>> >>>>>>>>>>
>> >>>>>>>>>> 2. Create a project specific “amphora-requirements.txt” file
>> >>>>>>>>>> for the
>> >>>>>>>>>> service VM packages (this is actually my preference.) It has
>> >>>>>>>>>> been
>> >>>>>>>>>> pointed out that this wouldn’t be kept up-to-date by the bot.
>> >>>>>>>>>> We could
>> >>>>>>>>>> modify the bot to include it in some way, or do it manually, or
>> >>>>>>>>>> with a
>> >>>>>>>>>> project specific job.
>> >>>>>>>>>>
>> >>>>>>>>>> 3. Split our service VM builds into another repo, to keep a
>> >>>>>>>>>> clean
>> >>>>>>>>>> separation between API services and the backend. But, even this
>> >>>>>>>>>> new
>> >>>>>>>>>> repo’s standlone requirements.txt file will have the g-r issue
>> >>>>>>>>>> from #1.
>> >>>>>>>>>>
>> >>>>>>>>>> 4. Boot the backend out of OpenStack entirely.
>> >>>>>>>>>
>> >>>>>>>>> All those options sound valid to me, so the requirements team
>> >>>>>>>>> should
>> >>>>>>>>> pick what they are the most comfortable with.
>> >>>>>>>>>
>> >>>>>>>>> My 2c: yes g-r is mostly about runtime dependencies and ensuring
>> >>>>>>>>> co-installability. However it also includes test/build-time
>> >>>>>>>>> deps, and
>> >>>>>>>>> generally converging dependencies overall sounds like a valid
>> >>>>>>>>> goal. Is
>> >>>>>>>>> there any drawback in adding gunicorn to g-r (option 1) ?
>> >>>>>>>>
>> >>>>>>>> The drawback (in my mind) is that new projects might start using
>> >>>>>>>> it giving operators yet another thing to learn about when deploying a new
>> >>>>>>>> component (eventlet, gevent, gunicorn, ...).
>> >>>>>>>>
>> >>>>>>>> On the flip, what's the benefit of adding it to g-r?
>> >>>>>>>
>> >>>>>>> The positive benefit is the same as Octavia’s use case: it
>> >>>>>>> provides an alternative for any non-frontline-api service to run a
>> >>>>>>> lightweight http/wsgi service as needed (service VMs, health monitor agents,
>> >>>>>>> etc). And something better than the built-in debug servers in most of the
>> >>>>>>> frameworks.
>> >>>>>>>
>> >>>>>>> On the proliferation point, it is certainly a risk, though I’ve
>> >>>>>>> personally heard pretty strong guidance that all main API services in our
>> >>>>>>> community should be trending towards pecan.
>> >>>>>>
>> >>>>>> Pecan is a way to build WSGI applications. Gunicorn is a way to
>> >>>>>> deploy
>> >>>>>> them. So they're not mutually exclusive.
>> >>>>>
>> >>>>> Right, agreed.
>> >>>>>
>> >>>>> What we’re trying to convey here is:
>> >>>>>
>> >>>>> - The normal way of making a REST endpoint in OpenStack is to use
>> >>>>> pecan (or flask or falcon), and let the deployer or packager worry about the
>> >>>>> runtime wsgi and/or reverse proxy.
>> >>>>>
>> >>>>> - This isn't a “normal” OpenStack endpoint, because it’s a
>> >>>>> microservice inside a service VM, and thus has different needs, and is much
>> >>>>> less likely to be customized by a non-dev, to boot. And it needs to be
>> >>>>> “deploy ready” as a simple matter of it being a service VM black box. It’s
>> >>>>> part of the data plane, not the control plane.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> doug
>> >>>>
>> >>>> That all seems reasonable.
>> >>>>
>> >>>> We have a proliferation of these service VMs. It would be good to
>> >>>> get some of the folks involved together to start a working group
>> >>>> to see if there are some commonalities that can lead to shared
>> >>>> processes or tools.
>> >>>
>> >>> That’s a good idea. I wonder if we can organize something in time for
>> >>> next week. I don’t think we should wait to make forward progress for that,
>> >>> but there is definitely some commonality we should be defining and striving
>> >>> towards.
>> >>>
>> >>> doug
>> >>
>> >> Sure, and I hope I didn't come across as implying that this work should
>> >> wait.
>> >>
>> >> I expect you could take over a corner of the dev lounge or some
>> >> other space to hold a BoF to at least start the discussion and get
>> >> some interested folks lined up to lead the WG.
>> >
>> > Sounds good. In prep to sending an ML invite, what projects use service
>> > VMs?   There’s Octavia, Trove, and Tacker.  What else?
>>
>> It is in our (long term) road map for Designate as well.
>>
>> > Thanks,
>> > doug
>> >
>> >>
>> >> Doug
>> >>
>> >>
>> >> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> >
>> >
>> > __________________________________________________________________________
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Davanum Srinivas :: https://twitter.com/dims



More information about the OpenStack-dev mailing list