Open Stack

On Mon, Jan 11, 2016 at 5:01 PM, Michał Jastrzębski <inc007 at gmail.com> wrote:
> On 11 January 2016 at 09:16, Eric LEMOINE <elemoine at mirantis.com> wrote:

>> * Logstash was designed for logs processing.  Heka is a "unified data
>> processing" software, designed to process streams of any type of data.
>> So Heka is about running one service on each box instead of many.
>> Using a single service for processing different types of data also
>> makes it possible to do correlations, and derive metrics from logs and
>> events.  See Rob Miller's presentation [4] for more details.
>
> Right now we use rsyslog for that.


Currently the services running in containers send their logs to
rsyslog. And rsyslog stores the logs in local files, located in the
host's /var/log directory.


> As I understand Heka right now
> would be actually alternative to rsyslog, and that is already
> implemented. Also with Heka case, we might run into same problem we've
> seen with rsyslog - transporting logs from service to heka. Keep in
> mind we're in dockers and heka will be in different container than
> service it's supposed to listen to. We do that by sharing faked
> /dev/log across containers and rsyslog can handle that.


I know. Our plan is to rely on Docker. Basically: containers write
their logs to stdout. The logs are collected by Docker Engine, which
makes them available through the unix:///var/run/docker.sock socket.
The socket is mounted into the Heka container, which uses the Docker
Log Input plugin [*] to reads the logs from that that socket.

[*] <http://hekad.readthedocs.org/en/latest/config/inputs/docker_log.html>


> Also Heka seems to be just processing mechanism while logstash is
> well...stash, so it saves and persists logs, so it seems to me they're
> different layers of log processing.

No. Logstash typically stores the logs in Elasticsearch. And we'd do
the same with Heka.


>
> Seems to me we need additional comparason of heka vs rsyslog;) Also
> this would have to be hands down better because rsyslog is already
> implemented, working and most of operators knows how to use it.


We don't need to remove Rsyslog. Services running in containers can
write their logs to both Rsyslog and stdout, which even is what they
do today (at least for the OpenStack services).


Hope that makes sense!