[openstack-dev] [Openstack-operators] Performance Team summit session results

Mark Wagner mwagner at redhat.com
Mon Nov 9 17:51:28 UTC 2015


For clarification, this is 3-4 PM (15:00 - 16:00) UTC, correct ?.

-mark

----- Original Message -----
> From: "Dina Belova" <dbelova at mirantis.com>
> To: "Matt Riedemann" <mriedem at linux.vnet.ibm.com>
> Cc: "OpenStack Development Mailing List" <openstack-dev at lists.openstack.org>, openstack-operators at lists.openstack.org
> Sent: Monday, November 9, 2015 4:30:55 AM
> Subject: Re: [Openstack-operators] [openstack-dev] Performance Team summit session results
> 
> Folks,
> 
> due to the doodle <http://doodle.com/poll/wv6qt8eqtc3mdkuz#table> 3:00 -
> 4:00 UTC Tuesdays (starting from tomorrow) is ok for all voted people.
> Although for the US folks with PST time zone it'll be very early due to the
> time zone change happened for US on November, 1st. Still hope seeing you
> there on *#openstack-performance* channel :)
> 
> I've created primary wiki pages for the team
> <https://wiki.openstack.org/wiki/Performance_Team> and its meetings
> <https://wiki.openstack.org/wiki/Meetings/Performance> - please feel free
> to add more items to the agenda.
> 
> See you tomorrow :)
> 
> Cheers,
> Dina
> 
> 
> On Mon, Nov 9, 2015 at 5:38 PM, Dina Belova <dbelova at mirantis.com> wrote:
> 
> > Matt,
> >
> > thank you so much for covering [1], [2] and [3] points - I'll ping folks
> > who've written these lines directly and will try to find out the answers.
> >
> > Cheers,
> > Dina
> >
> > On Fri, Oct 30, 2015 at 1:42 AM, Matt Riedemann <
> > mriedem at linux.vnet.ibm.com> wrote:
> >
> >>
> >>
> >> On 10/29/2015 10:55 AM, Matt Riedemann wrote:
> >>
> >>>
> >>>
> >>> On 10/29/2015 9:30 AM, Dina Belova wrote:
> >>>
> >>>> Hey folks!
> >>>>
> >>>> On Tuesday we had great summit session about performance team kick-off
> >>>> and yesterday it was a great LDT session as well and I’m really glad to
> >>>> see how much does the OpenStack performance topic is important for all
> >>>> of us. 40 minutes session surely was not enough to analyse everyone’s
> >>>> feedback and bottlenecks people usually see, so I’ll try to finalise
> >>>> what have been discussed and the next steps in this email.
> >>>>
> >>>> Performance team kick-off session
> >>>> (
> >>>> https://etherpad.openstack.org/p/mitaka-cross-project-performance-team-kick-off
> >>>> )
> >>>>
> >>>> can be shortly described with the following points:
> >>>>
> >>>>   * IBM, Intel, HP, Mirantis, Rackspace, Red Hat, Yahoo! and others were
> >>>>     taking part in the session
> >>>>   * Various tools are used right now for OpenStack benchmarking and
> >>>>     profiling right now:
> >>>>       o Rally (IBM, HP, Mirantis, Yahoo!)
> >>>>       o Shaker (Mirantis, merging its functionality to Rally right now)
> >>>>       o Gatling (Rackspace)
> >>>>       o Zipkin (Yahoo!)
> >>>>       o JMeter (Yandex)
> >>>>       o and others…
> >>>>   * Various issues have been seen during the OpenStack cloud operating
> >>>>     (full list can be found here -
> >>>>     https://etherpad.openstack.org/p/openstack-performance-issues).
> >>>> Most
> >>>>     mentioned issues were the following:
> >>>>       o performance of DB-related layers (DB itself and oslo.db) - it is
> >>>>         about 7 abstraction DB layers in Nova; performance of Nova
> >>>>         conductor was mentioned several times
> >>>>       o performance of MQ-related layers (MQ itself and oslo.messaging)
> >>>>   * Different companies are using different standards for performance
> >>>>     benchmarking (both control plane and data plane testing)
> >>>>   * The most wished output from the team due to the comments will be:
> >>>>       o agree on the “performance testing standard”, including answers
> >>>>         on the following questions:
> >>>>           + what tools need to be used for OpenStack performance
> >>>>             benchmarking?
> >>>>           + what benchmarking meters need to be covered? what we would
> >>>>             like to compare?
> >>>>           + what scenarios need to be covered?
> >>>>           + how can we compare performance of different cloud
> >>>> deployments?
> >>>>           + what performance deployment patterns can be used for various
> >>>>             workloads?
> >>>>       o share test plans and perform benchmarking tests
> >>>>       o create methodologies and documentation about best OpenStack
> >>>>         deployment and performance testing practices
> >>>>
> >>>>
> >>>> We’re going to cover all these topics further. First of all IRC channel
> >>>> for the discussions was created: *#openstack-performance*. We’re going
> >>>> to have weekly meeting related to current progress on that channel,
> >>>> doodle with the voting can be found here:
> >>>> http://doodle.com/poll/wv6qt8eqtc3mdkuz#table
> >>>>   (I was brave enough not to include timeslots that were overlapping
> >>>> with some of mine really hard-to-move activities :))
> >>>>
> >>>> Let’s have next week as a voting time, and have first IRC meeting in our
> >>>> channel the week after next. We can start our further discussions with
> >>>> “performance” and “performance testing” terms definition and
> >>>> benchmarking tools analysis.
> >>>>
> >>>> Cheers,
> >>>> Dina
> >>>>
> >>>>
> >>>>
> >>>> __________________________________________________________________________
> >>>>
> >>>> OpenStack Development Mailing List (not for usage questions)
> >>>> Unsubscribe:
> >>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>>
> >>> Thanks for writing this up, it's great to see people getting together
> >>> and sharing info on performance issues and trying to pinpoint the big
> >>> ones.
> >>>
> >>> I poked through the performance issues etherpad and was wondering how
> >>> many people with DB issues, particularly for nova-conductor, are using a
> >>> level of oslo.db that's new enough to be using pymysql rather than
> >>> mysql-python because from what I remember there were eventlet issues
> >>> without pymysql. That was added to oslo.db 1.12.0 [1].
> >>>
> >>> The nova-conductor workers / CPU usage is also a known issue in the
> >>> large ops gate job [2] but I'm not aware of anyone spending the time
> >>> drilling into what exactly is causing a lot of that overhead and if any
> >>> of it is abnormal.
> >>>
> >>> Finally, wrt DB, I'd also be interested to know if Rackspace, or anyone
> >>> else, is still running with the direct-to-sql stuff that comstud wrote
> >>> for nova [3] and if that still shows significant performance
> >>> improvements over using sqlalchemy ORM. Not to open that can of worms in
> >>> the -dev list here again, but it'd be an interesting data point.
> >>>
> >>> [1] https://review.openstack.org/#/c/184392/
> >>> [2] https://review.openstack.org/#/c/228636/
> >>> [3] https://blueprints.launchpad.net/nova/+spec/db-mysqldb-impl
> >>>
> >>>
> >> Oops, forgot to copy the ops list on the last reply.
> >>
> >> --
> >>
> >> Thanks,
> >>
> >> Matt Riedemann
> >>
> >>
> >> _______________________________________________
> >> OpenStack-operators mailing list
> >> OpenStack-operators at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> >>
> >
> >
> >
> > --
> >
> > Best regards,
> >
> > Dina Belova
> >
> > Software Engineer
> >
> > Mirantis Inc.
> >
> 
> 
> 
> --
> 
> Best regards,
> 
> Dina Belova
> 
> Software Engineer
> 
> Mirantis Inc.
> 
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> 



More information about the OpenStack-dev mailing list