Open Stack

On 01/10/15 20:34, Dan Prince wrote:
> On Wed, 2015-09-30 at 21:05 +0100, Steven Hardy wrote:
>> Hi all,
>>
>> So I wanted to start some discussion on $subject, because atm we have
>> a
>> couple of patches adding support for new services (which is great!):
>>
>> Manila: https://review.openstack.org/#/c/188137/
>> Sahara: https://review.openstack.org/#/c/220863/
>>
>> So, firstly I am *not* aiming to be any impediment to those landing,
>> and I
>> know they have been in-progress for some time.  These look pretty
>> close to
>> being ready to land and overall I think new service integration is a
>> very
>> good thing for TripleO.
>>
>> However, given the recent evolution towards the "big tent" of
>> OpenStack, I
>> wanted to get some ideas on what an effective way to selectively
>> enable
>> services would look like, as I can imagine not all users of TripleO
>> want to
>> deploy all-the-services all of the time.
>>
>> I was initially thinking we simply have e.g "EnableSahara" as a
>> boolean in
>> overcloud-without-mergepy, and wire that in to the puppet manifests,
>> such
>> that the services are not configured/started.  However comments in
>> the
>> Sahara patch indicate it may be more complex than that, in particular
>> requiring changes to the loadbalancer puppet code and os-cloud
>> -config.
>>
>> This is all part of the more general "composable roles" problem, but
>> is
>> there an initial step we can take, which will make it easy to simply
>> disable services (and ideally not pay the cost of configuring them at
>> all)
>> on deployment?
>>
>> Interested in peoples thoughts on this - has anyone already looked
>> into it,
>> or is there any existing pattern we can reuse?
> 
> A couple of ideas I would throw out that might help us pair down the
> larger controller role into more composable roles.
> 
> On the Heat side we could individual role templates directly. So new
> nested stack templates for Sahara and Manilla (or any service really).
> These templates would still wire into the controller.yaml in much the
> same way... but we would compose the resulting configuration metadata
> based on what was configured in the resource registry. So if Sahara or
> Manilla is set to a noop resource we would essentially get a controller
> without those service. If it is enabled then we would pull in the extra
> configuration metadata (hiera) and wire it into the structured config
> as normal. Perhaps a new 'roles' directory would help organize these
> services... The mechanism for composability on the Heat side is really
> the resource registry so we need to be careful to name and document
> these things correctly.
> 
> ----
> 
> On the Puppet side the initial goal was to avoid creating what we
> called a composition layer, which is basically a new set of puppet
> modules that act as a front end for all of your roles etc. This isn't
> to say that we don't want composability with the Puppet (we do) but
> just that our preference was to write role manifests that used direct
> include statements to OpenStack Puppet modules. This proved to be a
> more hackable set of templates and forces us to use (as much as
> possible) OpenStack puppet or add functionality there rather than go
> and write our own puppet-tripleo to rule the world.
> 
> In practice that didn't fully work out and we do have a small bit of
> Puppet code in puppet-tripleo. I consider much of this to be technical
> debt and over the mid to long term would prefer to refactor and add the
> puppet-tripleo bits elsewhere.
> 
> One area we could improve our puppet architecture might be with regards
> to how we actually deploy the puppet manifests themselves. Currently we
> deploy the manifests with Heat directly by using get_file to deploy the
> manifest alongside of the SoftwareConfig (looks something like this htt
> p://git.openstack.org/cgit/openstack/tripleo-heat
> -templates/tree/puppet/compute-post.yaml#n23). This works really nicely
> in that our puppet manifests can live alongside of the (lightly)
> coupled Heat templates which pass in parameters via Hiera. However the
> approach is limited in that we can only deploy a single manifest (not a
> directory of roles that could be dynamically assembled or included on
> the fly). Perhaps we could expand this capability a bit so that we
> could structure the puppet a bit more freely into decomposed role
> manifests. We already have a similar thing going on w/ the Hiera
> element where we can deploy multiple Hiera files for each role. So
> perhaps a script step to deploy multiple puppet files (generic so it
> could be used in multiple templates), or perhaps we could use Swift to
> deploy these alongside of Heat (I'm not sure we have the required
> OS::Swift heat resources for this ATM though).
> 
> Again we've already got mechanisms to compose puppet Hiera, we just
> need a few more tricks to more easily compose the role manifests
> themselves...
> 
> Dan
> 
> 
> 
>>
>> As mentioned above, not aiming to block anything on this, I guess we
>> can
>> figure it out and retro-fit it to whatever services folks want to
>> selectively disable later if needed.
> 
> Agree with not blocking the Manilla and Sahara integration. I don't see
> anything with regards to those that would prevent future refactoring to
> make the roles more composable.

one question clarification here - I am reviewing the manila-generic
patches @ https://review.openstack.org/#/c/188137/30 - on my downstream
setup none of the manila-* dependencies are landed. Regardless of the
how, I think (and sounds like from the previous posts) in the very least
some mechanism for enable/disable these 'new' services is a desirable
thing to have.

However what about the images? Will manila always be installed on
deployed image (what if I build my own images, explicitly without
manila?). The expectation on the current review above is that even if I
don't deploy/configure anything manila, the packages are available,
otherwise the include ::manila* on the puppet manifest fails as on the
review comments (which imo is wrong, especially if there is an easy way
to make it so for now, any new heat/templates methods as proposed by
dprince above notwithstanding). Grateful for any other opinions on the
review,

thanks, marios



> 
>>
>> Thanks,
>>
>> Steve
>>
>> _____________________________________________________________________
>> _____
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubs
>> cribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>