Open Stack

Thanks for the feedback, really good stuff in here :)

On 01/15/2014 08:07 AM, James Slagle wrote:
> I'll start by laying out how I see editing or updating nodes working
> in TripleO without Tuskar:
>
> To do my initial deployment:
> 1.  I build a set of images for my deployment for different roles. The
> images are different based on their role, and only contain the needed
> software components to accomplish the role they intend to be deployed.
> 2.  I load the images into glance
> 3.  I create the Heat template for my deployment, likely from
> fragments that are already avaiable. Set quantities, indicate which
> images (via image uuid) are for which resources in heat.
> 4.  heat stack-create with my template to do the deployment
>
> To update my deployment:
> 1.  If I need to edit a role (or create a new one), I create a new image.
> 2.  I load the new image(s) into glance
> 3.  I edit my Heat template, update any quantities, update any image uuids, etc.
> 4.  heat stack-update my deployment
>
> In both cases above, I see the role of Tuskar being around steps 3 and 4.
>
> I may be misinterpreting, but let me say that I don't think Tuskar
> should be building images. There's been a fair amount of discussion
> around a Nova native image building service [1][2]. I'm actually not
> sure what the status/concensus on that is, but maybe longer term,
> Tuskar might call an API to kick off an image build.

I didn't mean to imply that Tuskar would be building images, just 
kicking them off.

As for whether or not it should, that's an interesting question. You and 
I are both on the same page on not having a generic image and having the 
services be configured outside of that, so I'll ignore that idea for now.

I've always thought of Tuskar as providing the user with everything 
they'd need. My gut reaction is that I don't like the idea of saying 
they have to go through a separate step of creating the image and then 
configuring the resource category in Tuskar and attaching the image to it.

That said, I suspect my gut is wrong, or at very least not in line with 
the OpenStack way of thinking.

> Ok, so given that frame of reference, I'll reply inline:
>
> On Mon, Jan 13, 2014 at 11:18 AM, Jay Dobies <jason.dobies at redhat.com> wrote:
>> I'm pulling this particular discussion point out of the Wireframes thread so
>> it doesn't get lost in the replies.
>>
>> = Background =
>>
>> It started with my first bulletpoint:
>>
>> - When a role is edited, if it has existing nodes deployed with the old
>> version, are the automatically/immediately updated? If not, how do we
>> reflect that there's a difference between how the role is currently
>> configured and the nodes that were previously created from it?
>
> I would think Roles need to be versioned, and the deployed version
> recorded as Heat metadata/attribute. When you make a change to a Role,
> it's a new version. That way you could easily see what's been
> deployed, and if there's a newer version of the Role to deploy.

+1, the more I've been thinking about this, the more I like it. We can't 
assume changes will be immediately applied to all provisioned instances, 
so we need some sort of record of what an instance was actually built 
against.

>> Replies:
>
> I know you quoted the below, but I'll reply here since we're in a new thread.
>
>> "I would expect any Role change to be applied immediately. If there is some
>> change where I want to keep older nodes how they are set up and apply new
>> settings only to new added nodes, I would create new Role then."
>
> -1 to applying immediately.

Agreed. At large scales, there are a number of problems with this.

> When you edit a Role, it gets a new version. But nodes that are
> deployed with the older version are not automatically updated.
>
>> "We will have to store image metadata in tuskar probably, that would map to
>> glance, once the image is generated. I would say we need to store the list
>> of the elements and probably the commit hashes (because elements can
>> change). Also it should be versioned, as the images in glance will be also
>> versioned.
>
> I'm not sure why this image metadata would be in Tuskar. I definitely
> like the idea of knowing the versions/commit hashes of the software
> components in your images, but that should probably be in Glance.

+1

>> We can't probably store it in the Glance, cause we will first store the
>> metadata, then generate image. Right?
>
> I'm not sure I follow this point. But, mainly, I don't think Tuskar
> should be automatically generating images.
>
>> Then we could see whether image was created from the metadata and whether
>> that image was used in the heat-template. With versions we could also see
>> what has changed.
>
> We'll be able to tell what image was used in the heat template, and
> thus the deployment,  based on it's UUID.
>
> I love the idea of seeing differences between images, especially
> installed software versions, but I'm not sure that belongs in Tuskar.
> That sort of utility functionality seems like it could apply to any
> image you might want to launch in OpenStack, not just to do a
> deployment.  So, I think it makes sense to have that as Glance
> metadata or in Glance somehow. For instance, if I wanted to launch an
> image that had a specific version of apache, it'd be nice to be able
> to see that when I'm choosing an image to launch.

Agreed that displaying this is probably not Tuskar's jurisdiction. If 
we're placing the image creation step on the user outside of Tuskar, 
it's on them to handle this.

>> But there was also idea that there will be some generic image, containing
>> all services, we would just configure which services to start. In that case
>> we would need to version also this.
>
> -1 to this.  I think we should stick with specialized images per role.
> I replied on the wireframes thread, but I don't see how
> enabling/disabling services in a prebuilt image should work. Plus, I
> don't really think it fits with the TripleO model of having an image
> created based on it's specific "role" (I hate to use that term and
> muddy the water....i mean in the generic sense here).
>
>
>> = New Comments =
>>
>> My comments on this train of thought:
>>
>> - I'm afraid of the idea of applying changes immediately for the same
>> reasons I'm worried about a few other things. Very little of what we do will
>> actually finish executing immediately and will instead be long running
>> operations. If I edit a few roles in a row, we're looking at a lot of
>> outstanding operations executing against other OpenStack pieces (namely
>> Heat).
>>
>> The idea of immediately also suffers from a sort of "Oh shit, that's not
>> what I meant" when hitting save. There's no way for the user to review what
>> the larger picture is before deciding to make it so.
>
> +1
>
>> - Also falling into this category is the image creation. This is not
>> something that finishes immediately, so there's a period between when the
>> resource category is saved and the new image exists.
>
> Since I don't think Tuskar should be an image building service, and no
> other one currently exists, I think we should require the
> administrator to build their images and load them into glance as a
> prerequisite before using them in a deployment.
>
>> If the image is immediately created, what happens if the user tries to
>> change the resource category counts while it's still being generated? That
>> question applies both if we automatically update existing nodes as well as
>> if we don't and the user is just quick moving around the UI.
>>
>> What do we do with old images from previous configurations of the resource
>> category? If we don't clean them up, they can grow out of hand. If we
>> automatically delete them when the new one is generated, what happens if
>> there is an existing deployment in process and the image is deleted while it
>> runs?
>
> Both these points are not as relevant given my earlier statement.
> But, if I turn out to be wrong about that :), then I'd say that we
> don't want to clean up old images automatically.  I don't like
> surprises, even if I can configure how many old images to keep.  I
> think that deleting should require manual intervention.

It should be easy enough for us to resolve which resource categories 
(and their versions) have instances deployed against them and providing 
a UI for the user to clean up.

>> We need some sort of task tracking that prevents overlapping operations from
>> executing at the same time. Tuskar needs to know what's happening instead of
>> simply having the UI fire off into other OpenStack components when the user
>> presses a button.
>>
>> To rehash an earlier argument, this is why I advocate for having the
>> business logic in the API itself instead of at the UI. Even if it's just a
>> queue to make sure they don't execute concurrently (that's not enough IMO,
>> but for example), the server is where that sort of orchestration should take
>> place and be able to understand the differences between the configured state
>> in Tuskar and the actual deployed state.
>>
>> I'm off topic a bit though. Rather than talk about how we pull it off, I'd
>> like to come to an agreement on what the actual policy should be. My
>> concerns focus around the time to create the image and get it into Glance
>> where it's available to actually be deployed. When do we bite that time off
>> and how do we let the user know it is or isn't ready yet?
>
> I think this becomes simpler if you're not worried about building
> images. Even so, some task tracking will likely be needed. TaskFlow[3]
> and Mistral[4] may be relevant.

There are a number of cases where this is important, even without image 
creation. I'll take a look at the two links you included; my initial 
thoughts were something like Celery but I'm glad to have examples that 
have been used by OpenStack elsewhere.

>
>> - Editing a node is going to run us into versioning complications. So far,
>> all we've entertained are ways to map a node back to the resource category
>> it was created under. If the configuration of that category changes, we have
>> no way of indicating that the node is out of sync.
>>
>> We could store versioned resource categories in the Tuskar DB and have the
>> version information also find its way to the nodes (note: the idea is to use
>> the metadata field on a Heat resource to store the res-cat information, so
>> including version is possible). I'm less concerned with eventual reaping of
>> old versions here since it's just DB data, though we still hit the question
>> of when to delete old images.
>
> Is resource category the same as role?  Sorry :), I probably need to
> go back and re-read the terminology thread. If so, I think versioning
> them in the Tuskar db makes sense. That way you know what's been
> deployed and what hasn't, as well as any differences.
>
>> - For the comment on a generic image with service configuration, the first
>> thing that came to mind was the thread on creating images from packages [1].
>> It's not the exact same problem, but see Clint Byrum's comments in there
>> about drift. My gut feeling is that having specific images for each res-cat
>> will be easier to manage than trying to edit what services are running on a
>> node.
>
> +1.
>
> [1] http://lists.openstack.org/pipermail/openstack-dev/2013-August/013122.html
> [2] https://wiki.openstack.org/wiki/NovaImageBuilding
> [3] https://wiki.openstack.org/wiki/TaskFlow
> [4] https://wiki.openstack.org/wiki/Mistral

Thanks again  :)