Open Stack

On 2/1/19 8:44 AM, Cédric Jeanneret wrote:
> On 1/30/19 6:26 PM, Fox, Kevin M wrote:
>> k8s's offical way of dealing with logs is to ensure use of the docker json logger, not the journald one. then all the k8s log shippers have a standard way to gather the logs. Docker supports log rotation and other options too. seems to work out pretty well in practice.
> sending directly to a file looks a good option indeed. Journald and
> (r)syslog have both some throttle issue, and it might create some issues
> in case of service restarting and the like.
>
> Pushing logs directly from the container engine (podman does actually
> support the same options) might be the way to go.
>
> As long as we have a common, easy way to output the logs, it's all for
> the best.
> The only concern I have with the "not-journald" path is the possible
> lack of "journalctl -f CONTAINER_NAME=foo". But, compared to the risks
> exposed in this thread about the possible crash if journald isn't
> available, and throttling, I think it's fine.
>
> Also, a small note regarding "log re-shipping": some people might want
> to push their logs to some elk/kelk/others - pushing the logs directly
> as json in plain files might help a log for that, as (r)syslog can then
> read them (and there, no bottleneck with throttle) and send it in the
> proper format to the remote logging infra.
>
> Soooo... yeah. imho the "direct writing as json" might be the way to go :).
That is just fine IMO. the runtime engine usually allows you to
configure the logging driver (docker in CentOS defaults... or used to
default, to journald); but if we find out that file is a better choice;
that's entirely fine. The whole point is to let the runtime engine do
its job, and handle the logging with the driver.
>
>> log shipping with other cri drivers such as containerd seems to work well too.
> Not tested yet, but at least podman has the option (as a work for this
> engine integration is done).
>
> Cheers,
>
> C.
>
>> Thanks,
>> Kevin
>> ________________________________________
>> From: Sean Mooney [smooney at redhat.com]
>> Sent: Wednesday, January 30, 2019 8:23 AM
>> To: Emilien Macchi; Juan Antonio Osorio Robles
>> Cc: openstack-discuss at lists.openstack.org
>> Subject: Re: [TripleO] containers logging to stdout
>>
>> On Wed, 2019-01-30 at 07:37 -0500, Emilien Macchi wrote:
>>>
>>> On Wed, Jan 30, 2019 at 5:53 AM Juan Antonio Osorio Robles <jaosorior at redhat.com> wrote:
>>>> Hello!
>>>>
>>>>
>>>> In Queens, the a spec to provide the option to make containers log to
>>>> standard output was proposed [1] [2]. Some work was done on that side,
>>>> but due to the lack of traction, it wasn't completed. With the Train
>>>> release coming, I think it would be a good idea to revive this effort,
>>>> but make logging to stdout the default in that release.
>>>>
>>>> This would allow several benefits:
>>>>
>>>> * All logging from the containers would en up in journald; this would
>>>> make it easier for us to forward the logs, instead of having to keep
>>>> track of the different directories in /var/log/containers
>>>>
>>>> * The journald driver would add metadata to the logs about the container
>>>> (we would automatically get what container ID issued the logs).
>>>>
>>>> * This wouldo also simplify the stacks (removing the Logging nested
>>>> stack which is present in several templates).
>>>>
>>>> * Finally... if at some point we move towards kubernetes (or something
>>>> in between), managing our containers, it would work with their logging
>>>> tooling as well
>>> Also, I would add that it'll be aligned with what we did for Paunch-managed containers (with Podman backend) where
>>> each ("long life") container has its own SystemD service (+ SystemD timer sometimes); so using journald makes total
>>> sense to me.
>> one thing to keep in mind is that journald apparently has rate limiting so if you contaiern are very verbose journald
>> will actully slowdown the execution of the contaienr application as it slows down the rate at wich it can log.
>> this came form a downstream conversation on irc were they were recommending that such applciation bypass journald and
>> log to a file for best performacne.
>>> --
>>> Emilien Macchi
>>
>>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190201/04198a5d/attachment-0001.sig>