<!DOCTYPE html>
<html>
<head>
<title></title>
</head>
<body><div> </div>
<div>On Wed, May 4, 2016, at 09:57 AM, Ethan Gafford wrote:<br></div>
<blockquote type="cite"><div dir="ltr"><div> </div>
<div><div defang_data-gmailquote="yes"><div>Sahara has support for several image generation-related cases:<br></div>
<div>1) Packing an image pre-cluster spawn in Nova.<br></div>
<div>2) Building clusters from a "clean" OS image post-Nova spawn, by downloading and installing Hadoop/Spark/Storm/WhatHaveYou.<br></div>
<div><div>3) Validating images after Nova spawn (to ensure that they're up to spec for the plugin in question.)<br></div>
</div>
<div><div>Because of this, we are pulling image generation logic into the plugins themselves, from a monolithic sahara-image-elements repository. This will aid us in our eventual hope to pull plugins out of the Sahara tree entirely; more immediately, it will allow us to define logic for all of these cases in one place, written eventually in Python. In our Sahara session at summit, which was also attended by several members of the Ironic team (thanks all), we discussed our current plan to use libguestfs rather than DIB as our toolchain; our intent to define a linear, idempotent set of steps to pack images for any plugin lends itself much more neatly to libguestfs' API than to DIB's.<br></div>
</div>
</div>
</div>
</div>
</blockquote><div> </div>
<div>Ok, there is clearly some kind of communication breakdown between DIB and some of its users that we *really* need to solve. AFAIK the primary set of DIB developers had no idea this was going on. I am going to push up a patch to our docs to make it a lot more clear where to find us (#tripleo or the ML). I'm not really sure what else we could do to make it easier for users to find us / raise issues with us so we can explore ways to solve them, but any suggestions would be a huge help.<br></div>
<div> </div>
<div>The idea that a linear set of idempotent steps is mutually exclusive with DIB's API is really interesting. IIUC this is something we grappled with in TripleO when creating the tool and simply never spent time looking in to solving for the existing distro elements, although that is an element issue not an API issue. DIB's API is just a linear set of commands you opt in to (by way of elements), if you make those commands idempotent then you have what you want, the problem is that the upstream element's themselves are not written in that way so you couldn't depend on them. I think there are some pretty obvious ways to solve this, potentially by making an element which provides the same API as libguestfs virt-customize (this would be very simple to do, I just haven't heard of anyone wanting it) or by hacking on some of the in-tree elements. Could you go in to some more detail on how you use this feature?<br></div>
<div> </div>
<blockquote type="cite"><div dir="ltr"><div><div defang_data-gmailquote="yes"><div><div>Beyond that, the Sahara team has certainly seen
profound difficulty in the field when customers attempt to generate
their own images, and even for Sahara cores, building images is
occasionally quite harrowing. These issues are seldom based on real
issues in the scripts themselves, but are frequently the result of bleed
between the host and guest; when these issues occur for a customer,
they become extremely difficult to diagnose remotely. Still, it's
entirely possible that DIB has answers to these problems, and it'd be a universal good to identify real flaws in DIB, or just to educate the uninitiated into how DIB can be made to work more cleanly if the features are already there (which they may well be; far be it from me to claim exhaustive mastery of DIB.) The technical reasons we like libguestfs over DIB are:<br></div>
</div>
</div>
</div>
</div>
</blockquote><div> </div>
<div>It would be *extremely* helpful to know about these issues when they come up. There's definitely some things we can do to prevent this but again, there hasn't been a lot of feedback that this is an issue people are running in to. There's a few different ways to prevent these types of problems ranging from doing something like virt-dib to having dib use another chroot during the image building process but I would really like to hear about some specific issues to get a better idea of what the root cause is. Hopefully we could even get these filed as bugs.<br></div>
<div> </div>
<blockquote type="cite"><div dir="ltr"><div><div defang_data-gmailquote="yes"><div>1) Libguestfs manipulates images in a clean helper VM created by libguestfs in a predictable way. As such, no mounts are made on the host, no scripts can affect the host system, and no variables on the host system are likely to affect the image packing process. See <a href="http://libguestfs.org/guestfs-security.1.html">http://libguestfs.org/guestfs-security.1.html</a> for information on libguestfs security.<br></div>
</div>
</div>
</div>
</blockquote><div> </div>
<div>Yep, isolation is definitely something DIB currently gives up in order to provide speed/lower resource usage. That being said, there are lots of things that could be done to either mitigate these issues or remove them altogether (by compromising on speed/resource requirements). My biggest concern is that we are creating two sets of code to perform the same task due to a false dichotomy - the obvious case being that at a minimum we could still be using something compatible with DIB while utilizing a vm/libguestfs. This would then have the benefits of having a common toolset among the community without the raised issues.</div>
<div> </div>
<blockquote type="cite"><div dir="ltr"><div><div defang_data-gmailquote="yes"><div>2) In-place image manipulation means that relatively minor changes to images (package installs, conf declaration, etc.) can occur swiftly, and subsequent changes can occur without uncompressing and recompressing an entire image.<br></div>
</div>
</div>
</div>
</blockquote><div> </div>
<div>This isn't an issue - its pretty trivial to make a dib element which modifies an existing image, and making it in-place versus outputting a new image is even more trivial. The ubuntu distro element effectively works this way, even - it is based off a cloud-image and then performs modifications to it.</div>
<div> </div>
<blockquote type="cite"><div dir="ltr"><div><div defang_data-gmailquote="yes"><div><div>3) DIB scripts' configuration settings, passed in freeform environment variables, can be difficult to understand and document for new users. Libguestfs' API demands more formal parameter passing, reducing the likelihood of accidental misconfiguration.<br></div>
</div>
</div>
</div>
</div>
</blockquote><div> </div>
<div>Back when there was a lot more development effort going in to DIB this was pretty high on the list of things to solve. There is even a spec somewhere to this effect. I simply haven't heard this issue raised much recently otherwise I would be pushing more for it to be solved since its really a matter of engineering effort at this point. Getting feedback like this is *extremely* helpful though, and again, any specific use cases / instances where this has caused issues would be even better.<br></div>
<div> </div>
<blockquote type="cite"><div dir="ltr"><div><div defang_data-gmailquote="yes"><div><div>I'm heartened that we're having this discussion: user-facing image packing tools are a critical part of any application-tier provisioning service, and cleaning our upstream image gen processes will be even more crucial as these projects mature and their support matrices grow.<br></div>
</div>
<div>Cheers,<br></div>
<div><div>-Egafford <br></div>
</div>
</div>
</div>
</div>
</blockquote><div> </div>
<div>Thanks,<br></div>
<div>Greg</div>
<div> </div>
</body>
</html>