Open Stack

On Mon, Oct 28, 2013 at 5:46 PM, Qing He <Qing.He at radisys.com> wrote:

> In my hard drive-less use case, I need an in-core-db/cache that can be in
> the same db cluster with real db (with hard drive) with the same sql api so
> that the current openstack code do not need to be changed, instead, just a
> pluggin with some configurations.
>

This is pretty much the original use case that dogpile.cache grew out of,
see:

  http://docs.sqlalchemy.org/en/rel_0_9/orm/examples.html#dogpile-caching


>
> -----Original Message-----
> From: Morgan Fainberg [mailto:m at metacloud.com]
> Sent: Monday, October 28, 2013 10:22 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] distibuted caching system in front of mysql
> server for openstack transactions
>
> In light of what Dolph said with regards to Keystone, we are using
> dogpile.cache to implement memoization in front of our driver calls.
> It it has the ability to cache directly as well, but it has been effective
> (so far) for our use-case.
>
> That being said, I am unsure if caching in front of MySQL is really what
> we want.  I believe that we should be caching after processing work (hence
> memoization mechanism) instead of at the SQL layer.  This also means we can
> be measured in what we cache (oh hey, it makes no sense to cache X because
> it needs to be "real time" or there isn't a performance issue with that
> query / call, but Y does a ton of processing and is an expensive
> join/temptable query).  In my experience, unless the whole application is
> designed with caching in mind, caching something as broad as MySQL calls
> (or any SQL store) is likely going to net exactly what Shawn Hartsock
> stated, adding a second performance issue.
>
> --Morgan
>
> On Mon, Oct 28, 2013 at 10:05 AM, Shawn Hartsock <hartsocks at vmware.com>
> wrote:
> >>
> >> I once heard a quote.. "I had a performance problem, so I added caching.
> >> now I have two performance problems."
> >>
> > this. 1,000 times this.
> >
> > Just to float this thought ... make sure it's considered...
> >
> > I've seen a *lot* of people misuse caching when what the really want is
> memoization.
> >
> > *
> > http://stackoverflow.com/questions/1988804/what-is-memoization-and-how
> > -can-i-use-it-in-python
> > *
> > http://stackoverflow.com/questions/10879137/how-can-i-memoize-a-class-
> > instantiation-in-python
> >
> > ... I'm not sure what you're trying to do. So YMMV, TTFN, BBQ.
> >
> > # Shawn Hartsock
> >
> > ----- Original Message -----
> >> From: "Clint Byrum" <clint at fewbar.com>
> >> To: "openstack-dev" <openstack-dev at lists.openstack.org>
> >> Sent: Monday, October 28, 2013 12:12:49 PM
> >> Subject: Re: [openstack-dev] distibuted caching system in front of
> mysql      server for openstack transactions
> >>
> >> Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700:
> >> > It's not specific to mysql (or sql at all), but keystone is using
> >> > dogpile.cache around driver calls to a similar effect.
> >> >
> >> >   http://dogpilecache.readthedocs.org/en/latest/
> >> >
> >> > It can persist to memcache, redis, etc.
> >> >
> >>
> >> I once heard a quote.. "I had a performance problem, so I added caching.
> >> now I have two performance problems."
> >>
> >> Caching is unbelievably awesome in the jobs it can do well. When the
> >> problem is straight forward and the requirements are few, it is just
> >> the right thing to relieve engineering pressure to make an
> >> application more scalable.
> >>
> >> However, IMO, more than narrow, well defined cache usage is a sign
> >> that the application needs some reworking to scale.
> >>
> >> I like the principle of "let's use dogpile so we don't have to
> >> reinvent multi-level caching". However, let's make sure we look at
> >> each performance issue individually, rather than just throwing them
> >> all in a cache box and waving the memcache wand.
> >>
> >> >
> >> > https://github.com/openstack/keystone/blob/master/keystone/common/c
> >> > ache/core.py
> >> >
> >> > On Fri, Oct 25, 2013 at 6:53 PM, Qing He <Qing.He at radisys.com> wrote:
> >> >
> >> > >  All,****
> >> > >
> >> > > Has anyone looked at the options of putting a distributed caching
> >> > > system in front of mysql server to improve performance? This
> >> > > should be similar to Oracle Coherence, or VMware VFabric
> >> > > SQLFire.****
> >> > >
> >> > > ** **
> >> > >
> >> > > Thanks,****
> >> > >
> >> > > ** **
> >> > >
> >> > > Qing****
> >> > >
> >> > > _______________________________________________
> >> > > OpenStack-dev mailing list
> >> > > OpenStack-dev at lists.openstack.org
> >> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >> > >
> >> > >
> >> >
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 

-Dolph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131031/a9e68798/attachment.html>