[openstack-dev] [EXTERNAL] Re: [Tripleo] deploy software on Openstack controller on the Overcloud
Abhishek Kane
Abhishek.Kane at veritas.com
Thu Jun 8 12:35:38 UTC 2017
Updated https://github.com/abhishek-kane/puppet-veritas-hyperscale with the attempts to reuse puppet modules for keystone, rabbitmq and nova_flavor. Still doing some operation via scripts.
Regards.
On 6/7/17, 8:08 PM, "Abhishek Kane" <Abhishek.Kane at veritas.com> wrote:
Hi,
On cinder node- we need to modify the cinder.conf. We don’t change any config apart from this. We want to keep the config changes in heat template, package installation in puppet, and trigger rest of the operations via Horizon (as it’s done today). We are also trying to get rid of the nova.conf file changes. Once the approach for cinder is sorted, will get back on this.
If this is correct approach for cinder, I will raise review requests for the following projects:
puppet-tripleo: http://paste.openstack.org/show/611697/
puppet-cinder: http://paste.openstack.org/show/611698/
tripleo-heat-templates: http://paste.openstack.org/show/611700/
Also, I am not sure which TripleO repos need to be patched for the controller components.
We have decomposed the controller bin installer into idempotent modules/scripts. Now, the installer is not a black box operation:
https://github.com/abhishek-kane/puppet-veritas-hyperscale
The inline replies below are w.r.t. this project. The product installer bin currently works in atomic fashion. One issue which we see in puppet is the error handling and rollback operations.
Thanks,
Abhishek
On 6/1/17, 8:41 PM, "Emilien Macchi" <emilien at redhat.com> wrote:
On Thu, Jun 1, 2017 at 3:47 PM, Abhishek Kane <Abhishek.Kane at veritas.com> wrote:
> Hi Emilien,
>
> The bin does following things on controller:
> 1. Install core HyperScale packages.
Should be done by Puppet, with Package resource.
Ak> It’s done.
> 2. Start HyperScale API server
Should be done by Puppet, with Service resource.
AK> It’s done.
> 3. Install UI packages. This will add new files to and modify some existing files of Horison.
Should be done by Puppet, with Package resource and also some changes
in puppet-horizon maybe if you need to change Horizon config.
Ak> We have got rid of the horizon dependency right now. Our GUI components get installed via separate package.
> 4. Create HyperScale user in mysql db. Create database and dump config. Add permissions of nova and cinder DB to HyperScale user.
We have puppet-openstacklib which already manages DBs, you could
easily re-use it. Please look at puppet-nova for example to see how
things works in nova::db::mysql, etc.
AK> TBD
> 5. Add ceilometer pollsters for additional stats and modify ceilometer files.
puppet-ceilometer I guess. What do you mean by "files"? Config files?
Ak> We are trying to get rid of this dependency as well. TBD.
> 6. Change OpenStack configuration:
> a. Create rabbitmq exchanges
puppet-* modules already does it.
AK> It’s done via script. Do we need to patch any module?
> b. Create keystone user
puppet-keystone already does it.
AK> It’s done via script. Do we need to patch keystone module?
> c. Define new flavors
puppet-nova can manage flavors.
AK> It’s done via script. Do we need to patch nova module?
> d. Create HyperScale volume type and set default volume type to HyperScale in cinder.conf.
we already support multi backends in tripleo, HyperScale would just be
a new addition. Re-use the bits please: puppet-cinder and
puppet-tripleo will need to be patched.
AK> It’s done via script. Do we need to patch cinder module?
> e. Restart openstack’s services
Already done by openstack/puppet-* modules.
AK> We are trying to get rid of all OpenStack config file changes that we used to do. TBD.
> 7. Configure HyperScale services
Should be done by your module, (you can either write a _config
provider if it's ini standard otherwise just do a template that you
ship in the module, like puppet-horizon).
AK> It’s done.
> Once the controller is configured, we use HyperScale’s CLI to configure data and compute nodes-
>
> On data node (cinder):
> 1. Install HyperScale data node packages.
Should be done by Puppet, with Package resource.
> 2. Change cinder.conf to add backend and change rpc_backend.
puppet-cinder
> 3. Give the raw data disks and meta disks to HyperScale storage layer for usage.
what does it means? Do you run a CLI for that?
> 4. Configure HyperScale services.
Should be done by Puppet, with Service resource.
> 5. Dump config in the HyperScale database.
can be done by a script maybe?
>
> On compute (nova):
> 1. Install HyperScale compute packages.
Should be done by Puppet, with Package resource.
> 2. Configure cgroup.
we don't manage cgroups in TripleO AFIK yet but it's something we
could add, maybe with a puppet module.
> 3. Disable selinux.
Please don't do that. Disabling SElinux is a NOGO when adding new
features (sorry to care about Security).
> 4. Add ceilometer pollsters for additional stats and modify ceilometer files.
puppet-ceilometer
> 5. Modify qemu.conf to relax ACS checks.
puppet-nova maybe, but not sure we really want to do that:
https://vfio.blogspot.fr/2014/08/iommu-groups-inside-and-out.html
Any details on why you're doing it?
> 6. Modify libvirt.conf and libvirtd to allow live migration.
It's already supported by puppet-nova.
> 7. Change network settings.
Should be done by os-net-config in TripleO.
> 8. Configure HyperScale services.
Done by your module (again).
> 9. Dump config in the HyperScale database.
same as before.
>
> We assume that we will not require steps to install packages if we put packages in the overcloud image. We have started to convert the bin and the CLI into puppet modules.
>
>
> Regards,
> Abhishek
Hope it helped.
> On 6/1/17, 4:24 AM, "Emilien Macchi" <emilien at redhat.com> wrote:
>
> On Wed, May 31, 2017 at 6:29 PM, Dnyaneshwar Pawar
> <Dnyaneshwar.Pawar at veritas.com> wrote:
> > Hi Alex,
> >
> > Currently we have puppet modules[0] to configure our software which has
> > components on Openstack Controller, Cinder node and Nova node.
> > As per document[1] we successfully tried out role specific configuration[2].
> >
> > So, does it mean that if we have an overcloud image with our packages
> > inbuilt and we call our configuration scripts using role specific
> > configuration, we may not need puppet modules[0] ? Is it acceptable
> > deployment method?
>
> So running a binary from Puppet, to make configuration management is
> not something we recommend.
> Puppet has been good at managing configuration files and services for
> example. In your module, you just manage a file and execute it. The
> problem with that workflow is we have no idea what happens in backend.
> Also we have no way to make Puppet run idempotent, which is one
> important aspect in TripleO.
>
> Please tell us what does the binary, and maybe we can convert the
> tasks into Puppet resources that could be managed by your module. Also
> make the resources by class (service), so we can plug it into the
> composable services in TripleO.
>
> Thanks,
>
> > [0] https://github.com/abhishek-kane/puppet-veritas-hyperscale
> > [1]
> > https://docs.openstack.org/developer/tripleo-docs/advanced_deployment/node_config.html
> > [2] http://paste.openstack.org/show/611116/
> >
> > Thanks,
> > Dnyaneshwar
> >
> > On 5/30/17, 6:52 PM, "Alex Schultz" <aschultz at redhat.com> wrote:
> >
> > On Mon, May 29, 2017 at 5:05 AM, Dnyaneshwar Pawar
> > <Dnyaneshwar.Pawar at veritas.com> wrote:
> >
> > Hi,
> >
> > I am tying to deploy a software on openstack controller on the overcloud.
> > One way to do this is by modifying ‘overcloud image’ so that all packages of
> > our software are added to image and then run overcloud deploy.
> > Other way is to write heat template and puppet module which will deploy the
> > required packages.
> >
> > Question: Which of above two approaches is better?
> >
> > Note: Configuration part of the software will be done via separate heat
> > template and puppet module.
> >
> >
> > Usually you do both. Depending on how the end user is expected to
> > deploy, if they are using the TripleoPackages service[0] in their
> > role, the puppet installation of the package won't actually work (we
> > override the package provider to noop) so it needs to be in the
> > images. That being said, usually there is also a bit of puppet that
> > needs to be written to configure the end service and as a best
> > practice (and for development purposes), it's a good idea to also
> > capture the package in the manifest.
> >
> > Thanks,
> > -Alex
> >
> > [0]
> > https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/tripleo-packages.yaml
> >
> >
> > Thanks and Regards,
> > Dnyaneshwar
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
>
>
> --
> Emilien Macchi
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Emilien Macchi
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
More information about the OpenStack-dev
mailing list