[openstack-dev] [glance] Why no DB index on sort parameters

Nikhil Komawar nikhil.komawar at RACKSPACE.COM
Thu Apr 23 15:10:18 UTC 2015


Messing with indices is not a good idea to do iteratively.  Indexing large data sets is a really expensive operation and should be done carefully and consistently. Changing around indices is only going to make things unstable.

Thanks,
-Nikhil

________________________________________
From: Flavio Percoco <flavio at redhat.com>
Sent: Thursday, April 23, 2015 7:52 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [glance] Why no DB index on sort parameters

On 21/04/15 14:55 +0000, Nikhil Komawar wrote:
>Rally is great overall however, we need good EXPLAIN examples on real world data. Smaller deployments might benefit from a simple sample performance analysis however, larger data sets can have impacts on areas that you never expect.
>
>A spec means that we document the indices proposed in the code base, based on all of the use cases. The way I look at it, a patch is needed anyways and it (rally gate job) would get attention from reviewers when the patch is proposed.

Yes, I believe we need both. However, I'd probably just start with
something smaller and see how it behaves before going with big data
sets.

I'm not saying we don't need tests with proper data sets, I'm saying
that I'd probably start with smaller ones. As Mike already mentioned
in his email, there's an impact in writes and we can see that from
Rally tests, AFAICT.

The spec can come later, IMHO.

Cheers,
Flavio

>
>________________________________________
>From: Flavio Percoco <flavio at redhat.com>
>Sent: Tuesday, April 21, 2015 10:48 AM
>To: OpenStack Development Mailing List (not for usage questions)
>Subject: Re: [openstack-dev] [glance] Why no DB index on sort parameters
>
>On 21/04/15 14:39 +0000, Nikhil Komawar wrote:
>>This is a good idea. We recently removed a unique constraint that may result
>>into some queries being very slow especially those that involve "name"
>>property. I would recommend sketching out a spec that identifies potential full
>>table scans especially for queries that join over image_properties table.
>>
>>
>>We should discuss there what other use cases look like rather than smaller
>>feedback on the ML.
>
>More thatn a spec, I'd be interested in seeing the patch with the
>change up and the results reported in Rally.
>
>I guess we'll need a spec anyway, although I'd probably be ok with a
>good bug report here.
>
>/me *shrugs*
>Flavio
>
>>
>>
>>Thanks,
>>-Nikhil
>>━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
>>From: Mike Bayer <mbayer at redhat.com>
>>Sent: Tuesday, April 21, 2015 9:45 AM
>>To: openstack-dev at lists.openstack.org
>>Subject: Re: [openstack-dev] [glance] Why no DB index on sort parameters
>>
>>
>>
>>On 4/21/15 2:47 AM, Ajaya Agrawal wrote:
>>
>>    Hi All,
>>
>>    I see that glance supports arbitrary sort parameters and the default is
>>    "created_at" while listing images. Is there any reason why we don't have
>>    index over these fields? If we have an index over these fields then we
>>    would avoid a full table scan to do sorting. IMO at least the created_at
>>    field should have an index on it.
>>
>>just keep in mind that more indexes will place a performance penalty on INSERT
>>statements, particularly at larger volumes.  I have no idea if that is
>>important here but something to keep in mind.
>>
>>
>>
>>
>>
>>
>>    Cheers,
>>    Ajaya
>>
>>
>>
>>    __________________________________________________________________________
>>    OpenStack Development Mailing List (not for usage questions)
>>    Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>    http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>>__________________________________________________________________________
>>OpenStack Development Mailing List (not for usage questions)
>>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>--
>@flaper87
>Flavio Percoco
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
@flaper87
Flavio Percoco



More information about the OpenStack-dev mailing list