Open Stack

As VW mentioned, we're running our ELK clusters on a nova undercloud.

My elasticsearch cluster looks like this:

* 3 master nodes
* 6 client nodes (these nodes are running elasticsearch as clients only -
no data storage. they handle all api requests. they're also running kibana)
* 40-ish data nodes

The data nodes have 30gb memory with 15gb allocated to java heap. We're
using local storage on ssd.

I don't have the elasticsearch configs available to me at the moment, but
we're using fairly standard master/client/data node config settings. We've
tuned some of the threadpool settings. Some of our indices are also using
longer refresh times. We've increased the cluster and shard recovery
settings as the defaults were fairly low for the size of our data and
number of indices. I've also increased max file descriptors.

Most of our data in elasticsearch is from from logs (mostly openstack
service logs via logstash), and most of the default config settings have
worked well for us so far.

I think the biggest performance improvements we saw came from using ssd
drives, using the latest versions of elasticsearch, and splitting all
client/api request to dedicated elasticsearch client nodes.

Depending on your data and how you're using it, you may need or want to
visit other elasticsearch settings, increase number of replicas, increase
(or disable) index refresh time, increase number of shards, etc.

I use kopf [1] for checking the cluster status, and rally [2] for
performance testing.

-Nick

[1] https://github.com/lmenezes/elasticsearch-kopf
[2] https://github.com/elastic/rally

On Fri, Sep 2, 2016 at 7:59 AM, Matt Van Winkle <mvanwink at rackspace.com>
wrote:

> We are running entire ELK clusters on our undercloud.  Let me see if I can
> point our resident metrics expert at this as well.
>
> VW
>
> Sent from my iPhone
>
> On Sep 2, 2016, at 7:50 AM, David Medberry <openstack at medberry.net> wrote:
>
> Hey Tim,
> We've just started this effort. I'll see if the guy running the service
> can comment today.
>
> On Fri, Sep 2, 2016 at 6:36 AM, Tim Bell <Tim.Bell at cern.ch> wrote:
>
>>
>>
>> Has anyone had experience running ElasticSearch on top of OpenStack VMs ?
>>
>>
>>
>> Are there any tuning recommendations ?
>>
>>
>>
>> Thanks
>>
>> Tim
>>
>> _______________________________________________
>> OpenStack-operators mailing list
>> OpenStack-operators at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>>
>>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20160904/77822ab2/attachment.html>