Open Stack

Stefano Marfuli wrote:

> John raises an important point below that I think deserves attention not only from SWIFT developers
>but also from the INFRA team. Testing patches for specific systems raises the need for testing environments,
> for Solaris and ZFS.

> INFRA-Team: what's your take on this?

An open source project needs to be extensible beyond universally available environments.

How else could device drivers ever be developed?

How could new processors be supported by any operating system? Do you limit an OS to using at most n cores because you cannot ask testers to acquire test machines with more than n cores?

People do test that environment specific enhancements have no impact on routine environments. They also *read* the code to protect against it become to riddled with #ifdefs (or even ifs) dealing with special environments.

But ultimately, testing of the patch in the environment that it was intended for is limited to those who work with that environment. Open source operating systems deal with that reality all the time. I don't see why OpenStack should not as well.

The LFS patches have from the beginning sought to allow variable performance from the local file systems in a way that allows enhanced results without penalizing the performance for the general case or the readability of the core code.

The first approach was to define a low level plugin. In response to reviewer feedback we shifted to a middleware approach instead. We remain open to suggestions on an improved interface.

But it is important to recognize that this is not just about Solaris/Illumos or even ZFS. The LFS patches enable deployment of enhanced storage servers that can offer customers better solutions to how data integrity is implemented. And rather than merely replacing Swift as a whole, we have tried to work within Swift to preserve as much of the basic Swift operation as possible.

The concept introduced into the ring builder is that a local file system can have a feature where it provides self-healing data protection that is equivalent of an extra network replica. This is as technology neutral as possible. Insisting that this be invisible to the ring builder amounts to refusing to give credit to a local file system that does provide any form of enhanced local data protection. It says that no matter what you do, Swift will deploy the same amount of network-based data protection. Which would really make providing extra data protection independently of Swift driven network replication a wasted effort. It would be like adding a second health insurance policy for your data, without even increasing the deductible on your existing health coverage.

The feature proposed is not specific to ZFS, it covers *any* forms of local file systems providing data protection.

So the patch should be evaluated on how well it accomplishes the goal of *enabling* third party extensions without compromising the core code in performance or readability. Which is the basis on which any extension enabling code should be evaluated.

There is a major procedural issue as to how all associated products will be kept as current as possible with the core, without making the core QA teams responsible for testing each extension.

I believe the following guidelines are applicable:

* Anyone offering an extension SHOULD enable others to test it. Even if that means making proprietary
   hardware available at cost. Nexenta does not have to provide hardware because our product is deployable
   as a Virtual Machine. We will provide a VM image suitable for testing that does not require registration.
* Those offering an extension should provide regular integration testing of new core software to guard
   against new code inadvertently breaking existing extensions. I think this is a major area for discussion.
   How often should an extension sponsor be expected to do this testing, and at what frequency with
   what expected deadlines. On one hand those making new patches reasonably expect those who are
   Negatively impacted by those patches to provide somewhat prompt feedback. But no third party can
   be expected to provide full time QA teams to run tests on demand for third parties. Getting QA testing
   scheduled for in-house development is enough of a challenge for most of us already.
   Ultimately there will have to be a window where we expect sponsors of associated projects to run QA
   on new core patches that is near the end of each cycle. How long does this window need to be? How
   many cycles of testing would be adequate for each general release?