[openstack-dev] [puppet] compatibility of puppet upstream modules
Ptacek, MichalX
michalx.ptacek at intel.com
Thu Feb 4 17:35:53 UTC 2016
Hi Emilien,
It seems that keystone database is not populated, because of something, which happened on previous runs (e.g. some packages installation),
Following rows are visible just in log from first attempt
Debug: Executing '/usr/bin/mysql -e CREATE USER 'keystone'@'127.0.0.1' IDENTIFIED BY PASSWORD '*936E8F7AB2E21B47F6C9A7E5D9FE14DBA2255E5A''
Debug: Executing '/usr/bin/mysql -e GRANT USAGE ON *.* TO 'keystone'@'127.0.0.1' WITH MAX_USER_CONNECTIONS 0 MAX_CONNECTIONS_PER_HOUR 0 MAX_QUERIES_PER_HOUR 0 MAX_UPDATES_PER_HOUR 0'
….
….
I tried to clean databases & uninstall packages installed during deployment, but maybe I miss something as it simply doesn’t work ☺
Is there any procedure, how can I restore system to “vanilla state” before puppet modules installation ?
It looks to me that when deployment failed, it’s very difficult to “unstack” it
Thanks in advance,
Michal
From: Ptacek, MichalX
Sent: Thursday, February 04, 2016 11:14 AM
To: OpenStack Development Mailing List (not for usage questions) <openstack-dev at lists.openstack.org>
Subject: RE: [openstack-dev] [puppet] compatibility of puppet upstream modules
-----Original Message-----
From: Emilien Macchi [mailto:emilien at redhat.com]
Sent: Thursday, February 04, 2016 10:06 AM
To: OpenStack Development Mailing List (not for usage questions) <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [puppet] compatibility of puppet upstream modules
On 02/03/2016 04:03 PM, Ptacek, MichalX wrote:
> Hi all,
>
>
>
> I have one general question,
>
> currently I am deploying liberty openstack as described in
> https://wiki.openstack.org/wiki/Puppet/Deploy
>
> Unfortunately puppet modules specified in
> puppet-openstack-integration/Puppetfile are not compatible
Did you take the file from stable/liberty branch?
https://github.com/openstack/puppet-openstack-integration/tree/stable/liberty
[Michal Ptacek] I am deploying scenario003 with stable/liberty
>
> and some are also missing as visible from following output of “puppet
> module list”
>
>
>
> Warning: Setting templatedir is deprecated. See
> http://links.puppetlabs.com/env-settings-deprecations
>
> (at /usr/lib/ruby/vendor_ruby/puppet/settings.rb:1139:in
> `issue_deprecation_warning')
>
> Warning: Module 'openstack-openstacklib' (v7.0.0) fails to meet some
> dependencies:
>
> 'openstack-barbican' (v0.0.1) requires 'openstack-openstacklib'
> (>=6.0.0 <7.0.0)
>
> 'openstack-zaqar' (v0.0.1) requires 'openstack-openstacklib'
> (>=6.0.0
> <7.0.0)
>
> Warning: Module 'puppetlabs-postgresql' (v4.4.2) fails to meet some
> dependencies:
>
> 'openstack-openstacklib' (v7.0.0) requires 'puppetlabs-postgresql'
> (>=3.3.0 <4.0.0)
>
> Warning: Missing dependency 'deric-storm':
>
> 'openstack-monasca' (v1.0.0) requires 'deric-storm' (>=0.0.1 <1.0.0)
>
> Warning: Missing dependency 'deric-zookeeper':
>
> 'openstack-monasca' (v1.0.0) requires 'deric-zookeeper' (>=0.0.1
> <1.0.0)
>
> Warning: Missing dependency 'dprince-qpid':
>
> 'openstack-cinder' (v7.0.0) requires 'dprince-qpid' (>=1.0.0 <2.0.0)
>
> 'openstack-manila' (v7.0.0) requires 'dprince-qpid' (>=1.0.0 <2.0.0)
>
> 'openstack-nova' (v7.0.0) requires 'dprince-qpid' (>=1.0.0 <2.0.0)
>
> Warning: Missing dependency 'jdowning-influxdb':
>
> 'openstack-monasca' (v1.0.0) requires 'jdowning-influxdb' (>=0.3.0
> <1.0.0)
>
> Warning: Missing dependency 'opentable-kafka':
>
> 'openstack-monasca' (v1.0.0) requires 'opentable-kafka' (>=1.0.0
> <2.0.0)
>
> Warning: Missing dependency 'puppetlabs-stdlib':
>
> 'antonlindstrom-powerdns' (v0.0.5) requires 'puppetlabs-stdlib' (>=
> 0.0.0)
>
> Warning: Missing dependency 'puppetlabs-corosync':
>
> 'openstack-openstack_extras' (v7.0.0) requires 'puppetlabs-corosync'
> (>=0.1.0 <1.0.0)
>
> /etc/puppet/modules
>
> ├──antonlindstrom-powerdns (v0.0.5)
>
> ├──duritong-sysctl (v0.0.11)
>
> ├──nanliu-staging (v1.0.4)
>
> ├──openstack-barbican (v0.0.1)
>
> ├──openstack-ceilometer (v7.0.0)
>
> ├──openstack-cinder (v7.0.0)
>
> ├──openstack-designate (v7.0.0)
>
> ├──openstack-glance (v7.0.0)
>
> ├──openstack-gnocchi (v7.0.0)
>
> ├──openstack-heat (v7.0.0)
>
> ├──openstack-horizon (v7.0.0)
>
> ├──openstack-ironic (v7.0.0)
>
> ├──openstack-keystone (v7.0.0)
>
> ├──openstack-manila (v7.0.0)
>
> ├──openstack-mistral (v0.0.1)
>
> ├──openstack-monasca (v1.0.0)
>
> ├──openstack-murano (v7.0.0)
>
> ├──openstack-neutron (v7.0.0)
>
> ├──openstack-nova (v7.0.0)
>
> ├──openstack-openstack_extras (v7.0.0)
>
> ├──openstack-openstacklib (v7.0.0) invalid
>
> ├──openstack-sahara (v7.0.0)
>
> ├──openstack-swift (v7.0.0)
>
> ├──openstack-tempest (v7.0.0)
>
> ├──openstack-trove (v7.0.0)
>
> ├──openstack-tuskar (v7.0.0)
>
> ├──openstack-vswitch (v3.0.0)
>
> ├──openstack-zaqar (v0.0.1)
>
> ├──openstack_integration (???)
>
> ├──puppet-aodh (v7.0.0)
>
> ├──puppet-corosync (v0.8.0)
>
> ├──puppetlabs-apache (v1.4.1)
>
> ├──puppetlabs-apt (v2.1.1)
>
> ├──puppetlabs-concat (v1.2.5)
>
> ├──puppetlabs-firewall (v1.6.0)
>
> ├──puppetlabs-inifile (v1.4.3)
>
> ├──puppetlabs-mongodb (v0.11.0)
>
> ├──puppetlabs-mysql (v3.6.2)
>
> ├──puppetlabs-postgresql (v4.4.2) invalid
>
> ├──puppetlabs-rabbitmq (v5.2.3)
>
> ├──puppetlabs-rsync (v0.4.0)
>
> ├──puppetlabs-stdlib (v4.6.0)
>
> ├──puppetlabs-vcsrepo (v1.3.2)
>
> ├──puppetlabs-xinetd (v1.5.0)
>
> ├──qpid (???)
>
> ├──saz-memcached (v2.8.1)
>
> ├──stankevich-python (v1.8.0)
>
> └── theforeman-dns (v3.0.0)
>
>
>
>
>
> Most of the warning can be probably ignored, e.g I assume that latest
> barbican & zaqar are compatible with liberty (7.0) version of
> openstack-openstacklib
>
> 'openstack-barbican' (v0.0.1) requires 'openstack-openstacklib'
> (>=6.0.0 <7.0.0)
>
> 'openstack-zaqar' (v0.0.1) requires 'openstack-openstacklib'
> (>=6.0.0
> <7.0.0)
>
>
>
> Am I right or I need to get rid of all of these compatibility warnings
> before proceeding further ?
>
If you look at our CI jobs, we also have some warnings:
http://logs.openstack.org/36/275836/1/gate/gate-puppet-openstack-integration-scenario001-tempest-dsvm-centos7/15a5ead/console.html#_2016-02-03_21_56_40_945
[Michal Ptacek] Great, it seems that it’s not that big problem. I can stick on versions from Puppetfile (if it’s working in CI)
>
> I tried both, but during subsequent deployments I reached some
> intermediate issue with number of parallel mysql connections
>
>
>
> 2016-02-03 00:01:03.326 90406 DEBUG oslo_db.api [-] Loading backend
> 'sqlalchemy' from 'nova.db.sqlalchemy.api' _load_backend
> /usr/lib/python2.7/dist-packages/oslo_db/api.py:238
>
> 2016-02-03 00:01:03.333 90406 WARNING oslo_db.sqlalchemy.engines [-]
> SQL connection failed. 10 attempts left.
>
> 2016-02-03 00:01:13.345 90406 WARNING oslo_db.sqlalchemy.engines [-]
> SQL connection failed. 9 attempts left.
>
> 2016-02-03 00:01:23.358 90406 WARNING oslo_db.sqlalchemy.engines [-]
> SQL connection failed. 8 attempts left.
>
> 2016-02-03 00:01:33.361 90406 WARNING oslo_db.sqlalchemy.engines [-]
> SQL connection failed. 7 attempts left.
>
> 2016-02-03 00:01:43.374 90406 WARNING oslo_db.sqlalchemy.engines [-]
> SQL connection failed. 6 attempts left.
>
> 2016-02-03 00:01:53.387 90406 WARNING oslo_db.sqlalchemy.engines [-]
> SQL connection failed. 5 attempts left.
>
> 2016-02-03 00:02:03.400 90406 WARNING oslo_db.sqlalchemy.engines [-]
> SQL connection failed. 4 attempts left.
>
> 2016-02-03 00:02:13.412 90406 WARNING oslo_db.sqlalchemy.engines [-]
> SQL connection failed. 3 attempts left.
>
> 2016-02-03 00:02:23.415 90406 WARNING oslo_db.sqlalchemy.engines [-]
> SQL connection failed. 2 attempts left.
>
> 2016-02-03 00:02:33.429 90406 WARNING oslo_db.sqlalchemy.engines [-]
> SQL connection failed. 1 attempts left.
>
> *2016-02-03 00:02:43.440 90406 CRITICAL nova [-] DBConnectionError:
> (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server
> on '127.0.0.1' ((1040, u'Too many connections'))")*
>
> 2016-02-03 00:02:43.440 90406 ERROR nova Traceback (most recent call last):
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File "/usr/bin/nova-manage",
> line 10, in <module>
>
> 2016-02-03 00:02:43.440 90406 ERROR nova sys.exit(main())
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File
> "/usr/lib/python2.7/dist-packages/nova/cmd/manage.py", line 1445, in
> main
>
> 2016-02-03 00:02:43.440 90406 ERROR nova ret = fn(*fn_args, **fn_kwargs)
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File
> "/usr/lib/python2.7/dist-packages/nova/cmd/manage.py", line 949, in
> archive_deleted_rows
>
> 2016-02-03 00:02:43.440 90406 ERROR nova table_to_rows_archived =
> db.archive_deleted_rows(max_rows)
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File
> "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 1922, in
> archive_deleted_rows
>
> 2016-02-03 00:02:43.440 90406 ERROR nova return
> IMPL.archive_deleted_rows(max_rows=max_rows)
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File
> "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line
> 6075, in archive_deleted_rows
>
> 2016-02-03 00:02:43.440 90406 ERROR nova meta =
> MetaData(get_engine(use_slave=True))
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File
> "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line
> 162, in get_engine
>
> 2016-02-03 00:02:43.440 90406 ERROR nova return
> main_context_manager._factory.get_legacy_facade().get_engine(
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py",
> line 256, in get_legacy_facade
>
> 2016-02-03 00:02:43.440 90406 ERROR nova self._start()
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py",
> line 338, in _start
>
> 2016-02-03 00:02:43.440 90406 ERROR nova engine_args, maker_args)
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py",
> line 362, in _setup_for_connection
>
> 2016-02-03 00:02:43.440 90406 ERROR nova
> sql_connection=sql_connection, **engine_kwargs)
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line
> 152, in create_engine
>
> 2016-02-03 00:02:43.440 90406 ERROR nova test_conn =
> _test_connection(engine, max_retries, retry_interval)
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File
> "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/engines.py", line
> 334, in _test_connection
>
> 2016-02-03 00:02:43.440 90406 ERROR nova six.reraise(type(de_ref),
> de_ref)
>
> 2016-02-03 00:02:43.440 90406 ERROR nova File "<string>", line 2, in
> reraise
>
> 2016-02-03 00:02:43.440 90406 ERROR nova DBConnectionError:
> (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server
> on '127.0.0.1' ((1040, u'Too many connections'))")
>
> 2016-02-03 00:02:43.440 90406 ERROR nova
>
> 2016-02-03 12:32:54.682 16003 CRITICAL nova [-] KeyError:
> <VerNum(312)>
>
>
Sounds like related to MySQL config, not really related to Puppet OpenStack modules.
[Michal Ptacek] Thanks, you’re right. I resolved that issue already. Now I am getting following error from keystone
/usr/bin/openstack endpoint list --quiet --format csv
Could not find project: openstack (Disable debug mode to suppress these details.) (HTTP 401) (Request-ID: req-981ac293-15e0-45c7-81da-63c44bdc28b8)
Keystone db looks quite empty …
mysql> use keystone;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from user;
Empty set (0.00 sec)
mysql> select * from project;
Empty set (0.00 sec)
From log it looks that relevant part passed,
Debug: Automatically imported keystone::roles::admin from keystone/roles/admin into production
but not sure from where exactly keystone db should be populated:
Thanks for any comment/suggestion,
Michal
>
>
> which didn’t disappear even after changing max_connection limit in
> /etc/mysql/my.cnf ….
>
>
>
> mysql> select @@global.max_connections;
>
> +--------------------------+
>
> | @@global.max_connections |
>
> +--------------------------+
>
> | 1024 |
>
> +--------------------------+
>
> 1 row in set (0.00 sec)
>
>
>
> mysql> use nova;
>
> Reading table information for completion of table and column names
>
> You can turn off this feature to get a quicker startup with -A
>
>
>
> Database changed
>
> mysql>
>
>
>
> Please help me to understand, how people should use this integration
> project properly and how to avoid avoidable issues J
>
Well, your error does not seem related to Puppet OpenStack modules, but we're willing to help if you provide your manifests so we can try to see what is wrong in your deployment.
FWIW, our current CI is running integration jobs on both liberty & mitaka, using https://github.com/openstack/puppet-openstack-integration
scripts.
HTH
>
> Thanks in advance,
>
> Michal
>
>
>
>
>
>
>
> --------------------------------------------------------------
> Intel Research and Development Ireland Limited Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County
> Kildare Registered Number: 308263
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
>
>
> ______________________________________________________________________
> ____ OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<mailto:OpenStack-dev-request at lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
--
Emilien Macchi
--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160204/a84a9290/attachment.html>
More information about the OpenStack-dev
mailing list