Open Stack

In light of what Dolph said with regards to Keystone, we are using
dogpile.cache to implement memoization in front of our driver calls.
It it has the ability to cache directly as well, but it has been
effective (so far) for our use-case.

That being said, I am unsure if caching in front of MySQL is really
what we want.  I believe that we should be caching after processing
work (hence memoization mechanism) instead of at the SQL layer.  This
also means we can be measured in what we cache (oh hey, it makes no
sense to cache X because it needs to be "real time" or there isn't a
performance issue with that query / call, but Y does a ton of
processing and is an expensive join/temptable query).  In my
experience, unless the whole application is designed with caching in
mind, caching something as broad as MySQL calls (or any SQL store) is
likely going to net exactly what Shawn Hartsock stated, adding a
second performance issue.

--Morgan

On Mon, Oct 28, 2013 at 10:05 AM, Shawn Hartsock <hartsocks at vmware.com> wrote:
>>
>> I once heard a quote.. "I had a performance problem, so I added caching.
>> now I have two performance problems."
>>
> this. 1,000 times this.
>
> Just to float this thought ... make sure it's considered...
>
> I've seen a *lot* of people misuse caching when what the really want is memoization.
>
> * http://stackoverflow.com/questions/1988804/what-is-memoization-and-how-can-i-use-it-in-python
> * http://stackoverflow.com/questions/10879137/how-can-i-memoize-a-class-instantiation-in-python
>
> ... I'm not sure what you're trying to do. So YMMV, TTFN, BBQ.
>
> # Shawn Hartsock
>
> ----- Original Message -----
>> From: "Clint Byrum" <clint at fewbar.com>
>> To: "openstack-dev" <openstack-dev at lists.openstack.org>
>> Sent: Monday, October 28, 2013 12:12:49 PM
>> Subject: Re: [openstack-dev] distibuted caching system in front of mysql      server for openstack transactions
>>
>> Excerpts from Dolph Mathews's message of 2013-10-28 08:40:19 -0700:
>> > It's not specific to mysql (or sql at all), but keystone is using
>> > dogpile.cache around driver calls to a similar effect.
>> >
>> >   http://dogpilecache.readthedocs.org/en/latest/
>> >
>> > It can persist to memcache, redis, etc.
>> >
>>
>> I once heard a quote.. "I had a performance problem, so I added caching.
>> now I have two performance problems."
>>
>> Caching is unbelievably awesome in the jobs it can do well. When the
>> problem is straight forward and the requirements are few, it is just the
>> right thing to relieve engineering pressure to make an application more
>> scalable.
>>
>> However, IMO, more than narrow, well defined cache usage is a sign
>> that the application needs some reworking to scale.
>>
>> I like the principle of "let's use dogpile so we don't have to reinvent
>> multi-level caching". However, let's make sure we look at each
>> performance issue individually, rather than just throwing them all in
>> a cache box and waving the memcache wand.
>>
>> >
>> > https://github.com/openstack/keystone/blob/master/keystone/common/cache/core.py
>> >
>> > On Fri, Oct 25, 2013 at 6:53 PM, Qing He <Qing.He at radisys.com> wrote:
>> >
>> > >  All,****
>> > >
>> > > Has anyone looked at the options of putting a distributed caching system
>> > > in front of mysql server to improve performance? This should be similar
>> > > to
>> > > Oracle Coherence, or VMware VFabric SQLFire.****
>> > >
>> > > ** **
>> > >
>> > > Thanks,****
>> > >
>> > > ** **
>> > >
>> > > Qing****
>> > >
>> > > _______________________________________________
>> > > OpenStack-dev mailing list
>> > > OpenStack-dev at lists.openstack.org
>> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> > >
>> > >
>> >
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev