Hi, Belmiro's point of linking operators and developers is hugely important because the developers have the tough job of catering for both large and small deployments. What can look like a safety net for small systems (e.g. per-container file descriptor limits) turns into a huge pitfall when deploying at scale. I'm really interested to be involved in this SIG. Cheers, Dave -- ** Dave Holland ** Systems Support -- Informatics Systems Group ** ** 01223 496923 ** Wellcome Sanger Institute, Hinxton, UK ** On Thu, Nov 14, 2019 at 10:31:10AM +0100, Belmiro Moreira wrote:
Hi, first of all thanks to Thierry for driving this SIG creation. Having a SIG to discuss how to deploy/operate a large deployment will be incredibly useful. In my opinion we shouldn't restrict ourselves to a specific project or deployment size (or number of cells) but discuss the limits of each project architecture, the projects dependencies, limitations at scale (functionality vs simplicity), operational difficulties... Sharing experiences and understand the different challenges and actions that we are using to mitigate them will be extremely valuable. I think that we already have a lot of examples of companies/organizations that are deploying OpenStack at large scale. Compiling all this information (Summit presentations, blogs, superuser articles, ...) will be a good starting point for all operators and discussions. Every deployment is different. I also would like this SIG to be the bridge between the operators of large deployments and developers. Bringing specific pain points to discussion with developers. cheers, Belmiro CERN
On Thu, Nov 14, 2019 at 8:25 AM Arnaud MORIN <[1]arnaud.morin@gmail.com> wrote:
Hi all, +1 for me and my employer (OVH). We are mostly interested in sharing good practices when deploying a region at scale, and operating it. For the deployment part, my main pain point is about the configuration parameters I should use on different software (e.g. nova behind wsgi). The current doc is designed to deploy a small pod, but when we are going large, usually some of those params needs tuning. I'd like to identify them and eventually tag them to help other being aware that they are useful at large scale. About operating, I am pretty sure we can share some good advices as well. E.g., avoid restarting neutron agents in a single shot. So definitely interested in that group. Thanks for bringing that up. Cheers.
Le mer. 13 nov. 2019 à 19:00, Stig Telfer <[2]stig.openstack@telfer.org> a écrit :
On 13 Nov 2019, at 11:18, Thierry Carrez <[3]thierry@openstack.org> wrote:
Hi everyone,
In Shanghai we held a forum session to gauge interest in a new SIG to specifically address cluster scaling issues. In the past we had several groups ("Large deployments", "Performance", LCOO...) but those efforts were arguably a bit too wide and those groups are now abandoned.
My main goal here is to get large users directly involved in a domain where their expertise can best translate into improvements in the software. It's easy for such a group to go nowhere while
Hi Thierry & all - Thanks for your mail. I’m interested in joining this SIG. Among others, I’m interested in participating in discussions around these common problems: - golden signals for scaling bottlenecks (and what to do about them) - using Ansible at scale - strategies for simplifying OpenStack functionality in order to scale Cheers, Stig trying to boil the ocean. To maximize its chances of success and make it sustainable, the group should have a narrow focus, and reasonable objectives.
My personal idea for the group focus was to specifically address
scaling issues within a single cluster: basically identify and address issues that prevent scaling a single cluster (or cell) past a number of nodes. By sharing analysis and experience, the group could identify common pain points that, once solved, would help raising that number.
There was a lot of interest in that session[1], and it
predictably exploded in lots of different directions, including some that are definitely past a single cluster (like making Neutron better support cells). I think it's fine: my initial proposal was more of a strawman. Active members of the group should really define what they collectively want to work on. And the SIG name should be picked to match that.
I'd like to help getting that group off the ground and to a
place where it can fly by itself, without needing external coordination. The first step would be to identify interested members and discuss group scope and objectives. Given the nature of the group (with interested members in Japan, Europe, Australia and the US) it will be hard to come up with a synchronous meeting time that will work for everyone, so let's try to hold that discussion over email.
So to kick this off: if you are interested in that group, please
reply to this email, introduce yourself and tell us what you would like the group scope and objectives to be, and what you can contribute to the group.
Thanks!
[etherpad.openstack.org]
-- Thierry Carrez (ttx)
References
1. mailto:arnaud.morin@gmail.com 2. mailto:stig.openstack@telfer.org 3. mailto:thierry@openstack.org 4. https://urldefense.proofpoint.com/v2/url?u=https-3A__etherpad.openstack.org_p_PVG-2Dlarge-2Dscale-2DSIG&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=64bKjxgut4Pa0xs5b84yPg&m=DdEhOLy_myry74y3z2LhDWbl3ztokcSVufGIqfDSCaM&s=L7GyQqoSsD_56ROhOkKxfMtbER6jrPjcNSZrjNsQrMg&e=
-- The Wellcome Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.