[OpenStack-Infra] Log storage/serving

James E. Blair jeblair at openstack.org
Tue Sep 24 16:47:26 UTC 2013


Joshua Hesketh <joshua.hesketh at rackspace.com> writes:

> On 9/17/13 11:00 PM, Monty Taylor wrote:
>>
>> On 09/16/2013 07:22 PM, Joshua Hesketh wrote:
>>> So if zuul dictates where a log goes and we place the objects in swift
>>> with that path (change / patchset / pipeline / job / run) then zuul
>>> could also handle placing indexes as it should know which objects to
>>> expect.
>>>
>>> That said, if the path is deterministic (such as that) and the workers
>>> provide the index for a run then I'm not sure how useful an index for
>>> patchsets would be. I'd be interested to know if anybody uses the link
>>> http://logs.openstack.org/34/45334/ without having come from gerrit or
>>> another source where it is published.
>> https://pypi.python.org/pypi/git-os-job
> Right, but that calculates the path (as far as I can see) so we
> therefore still don't necessarily need indexes generated.

The final portion of the URL, signifying the run, is effectively random.
So that tool actually relies on a one-level-up index page.  (That tool
works on post jobs rather than check or gate, but the issues are
similar).

Other than that, most end users do not use indexes outside of the
particular job run, and that's by design.  We try to put the most useful
URL in the message that is left in Gerrit.

However, those of us working on the infrastructure itself, or those
engaged in special projects (such as mining old test logs), or even the
occasional person curious about whether the problem they are seeing was
encountered in _all_ runs of a test find the ability to locate logs from
any run _very_ useful.  If we lost that ability, we would literally have
no way to locate any logs other than the 'final' logs of a run, and
those only through the comment left in Gerrit, due to the issue
mentioned above.

We can discuss doing that, but it would be a huge change from our
current practice.

-Jim



More information about the OpenStack-Infra mailing list