Open Stack

hi Sahara folks,

This serves as a detailed status update for the Swift trust authentication spec[1], and to bring up concerns about integration for the Juno cycle.

So far I have pushed a few reviews that start to lay the groundwork for the infrastructure needed to complete this blueprint. I have tried to keep the changes as low impact as possible so as not to create incompatible commits. I will continue this for as long as makes sense, but we are approaching the point at which disruptive changes will be introduced.

Currently, I am working on delegating and revoking trusts for job executions. The next steps will be to finish the periodic updater that will distribute authentication tokens to cluster instances. After this I plan to start integrating the job binaries to use the authentication tokens as this will all be contained within Sahara.

Once these pieces are done I will focus on the Swift-Hadoop component and finalize the workflow creation to support the new Swift references. I will hold these changes until we are ready to switch to this new style of authentication as this will be disruptive to our current deployments. I would like to get some assistance understanding the Swift-Hadoop component, any guidance would be greatly appreciated.

That's the status update, I'm confident that over the next few weeks much of this will be implemented and getting ready for review.

I do have concerns around how we will integrate and release this update. Once the trust authentication is in place we will be changing the way Swift information is distributed to the cluster instances. This means that existing vm images will need to be updated with the new Swift-Hadoop component. We will need to create new public images for all plugins that use Hadoop and Swift. We will also need to update the publicly available versions of the Swift-Hadoop component to ensure that Sahara-image-elements continues to work.

We will also need to upgrade the gate testing machines to incorporate these changes and most likely I will need to be able to run these tests on a local cluster I can control before I push them for review. I am soliciting any advice about how I could run the gate tests from my local machine or cluster.

For the new Swift-Hadoop component I propose that we bump the version to 2.0 to indicate the incompatibility between it and the 1.0 version.

regards,
mike


[1]: https://blueprints.launchpad.net/sahara/+spec/edp-swift-trust-authentication