[OpenStack-Infra] Log storage/serving

Joshua Hesketh joshua.hesketh at rackspace.com
Fri Sep 27 00:34:30 UTC 2013


On 9/25/13 2:47 AM, James E. Blair wrote:
> Joshua Hesketh <joshua.hesketh at rackspace.com> writes:
>
>> On 9/17/13 11:00 PM, Monty Taylor wrote:
>>> On 09/16/2013 07:22 PM, Joshua Hesketh wrote:
>>>> So if zuul dictates where a log goes and we place the objects in swift
>>>> with that path (change / patchset / pipeline / job / run) then zuul
>>>> could also handle placing indexes as it should know which objects to
>>>> expect.
>>>>
>>>> That said, if the path is deterministic (such as that) and the workers
>>>> provide the index for a run then I'm not sure how useful an index for
>>>> patchsets would be. I'd be interested to know if anybody uses the link
>>>> http://logs.openstack.org/34/45334/ without having come from gerrit or
>>>> another source where it is published.
>>> https://pypi.python.org/pypi/git-os-job
>> Right, but that calculates the path (as far as I can see) so we
>> therefore still don't necessarily need indexes generated.
> The final portion of the URL, signifying the run, is effectively random.
> So that tool actually relies on a one-level-up index page.  (That tool
> works on post jobs rather than check or gate, but the issues are
> similar).

So two questions,
1) Do we need a random job run? is it for debugging or something? And if 
so, can we provide it another way.
2) What if the tool provided the index for its runs?

>
> Other than that, most end users do not use indexes outside of the
> particular job run, and that's by design.  We try to put the most useful
> URL in the message that is left in Gerrit.
>
> However, those of us working on the infrastructure itself, or those
> engaged in special projects (such as mining old test logs), or even the
> occasional person curious about whether the problem they are seeing was
> encountered in _all_ runs of a test find the ability to locate logs from
> any run _very_ useful.  If we lost that ability, we would literally have
> no way to locate any logs other than the 'final' logs of a run, and
> those only through the comment left in Gerrit, due to the issue
> mentioned above.
>
> We can discuss doing that, but it would be a huge change from our
> current practice.
Yep, I'm convinced that the logs need to be accessible.

Cheers,
Josh

--
Rackspace Australia

>
> -Jim




More information about the OpenStack-Infra mailing list