Open Stack

Clark Boylan <cboylan at sapwetik.org> writes:

> Couple of thoughts about this and Ara specifically. Ara static
> generation easily produces tens of thousands of files. Copying many
> small files to the log server with rsync was often quite slow (on the
> order of 10 minutes for some jobs (that is my fuzzy memory though)). I
> am concerned that HTTP to $swift service will have similar problems
> with many small files. This is something we should test.

Yes.  If we want to get out of the business of running a log proxy (I
*very* much do), static generation is the only currently supported
option with ara.  Despite the downsides, we were able to use ara in
static generation mode before and it worked.  I'm hopeful that by
uploading to swift in parallel, we can mitigate the upload cost.

> Also, while swift doesn't have inode problems the end user needs to
> worry about, it does apparently have limits on practical number of
> objects per container. One of the issues we had in the past,
> particularly with the swift we had access to, was that each container
> was not directly accessible by default and you had to configure CDN
> distribution of each container to be publicly visible. This made
> creating many containers to shard the objects more complicated than we
> had hoped. All this to say we may still have to solve the "inode"
> problem just within the context of swift containers, creating
> containers, making them visible.
>
> We should do our best to test both of these items and/or follow up
> with whichever cloud hosts the containers to make sure we aren't
> missing anything else (possible object creation rate limits for
> example).

Yes, the object limit concern is why I think our swift role should
create containers as necessary and shard storage.

The CDN behavior you describe where public access to swift is not
possible except by CDN, and where the CDN uses unpredictable hostnames
per container which must be determined via a non-standard API call is,
not swift's standard behavior, it is a cloud-specific variant of swift.

I believe we can write upload roles for the swift-variant you describe.
We can also write upload roles for standard swift.  I think it will be
difficult to try to use both at the same time, so if we're serious about
distributing logs to cloud-local swifts in the future, we may want to
start by focusing on standard swift (and accept that clouds that either
don't run swift, or don't run standard swift, will export their logs to
another cloud.  That's not so bad.  Most of our clouds do something
similar today).

-Jim