Open Stack

Joshua Hesketh wrote:
> Great overview and plan James, thanks for that :-).
> 
> So it seems to me that we're duplicating the job of swift a little bit
> by writing a program to accept an object over http and store it on disk.
> If our end-game is logs stored in swift then why not make jenkins (and
> other workers) push the logs straight to swift?

I suspect the additional benefit is the token-based security which means
the slaves can have limited access to storage (they basically get a
token that can only be used for one very limited - and one-time -  task).

That said I agree that reinventing HTTP POST storage sounds overkill and
storing directly to swift would be a lot simpler. Does Swift include
complex access rules that we could leverage to implement the same kind
of security that James described in his original post ? If not, would it
make sense to actually add what we need to Swift itself ? (after all,
it's, you know, an OpenStack project)

> [...]
> *For example, depending on the size of the logs (and therefore the
> job/worker) we could actually use javascript to serve up the swift
> object with CORS further reducing the infrastructure requirement and
> utilising the powerful CPU's and javascript engines we all have. I
> already started doing that as a crude way to serve and add formatting to
> my logs[1]. Basically javascript grabs a file and runs some regex for
> highlighting. So my worker only reports
> http://laughing-spice/logviewer/?q=http://worker/job/index.html where
> the index.html contains links to logs like
> http://laughing-spice/logviewer/?q=http://worker/job/mysqllog.txt etc.
> [...]

Sidenote: I like that idea too, but I'm not sure how well that would
work with our huge logs.

-- 
Thierry Carrez (ttx)