[openstack-dev] Barbican : Dependency of pyenv configuration in Barbican.sh script
Douglas Mendizabal
douglas.mendizabal at RACKSPACE.COM
Thu Apr 23 21:42:47 UTC 2015
Hi Asha,
I hope I can clear up some of your confusion about the Barbican server. Barbican is a standard WSGI application. [1] The WSGI application object is created by the create_main_app function in barbican.api.app [2]. WSGI should not be confused with uWSGI [3], which is a web server that can serve WSGI applications.
There are many ways to deploy a WSGI application. You could use apache with mod_wsgi [4], or you could use gunicorn [5], or you could use paste.httpserver [6] as the barbican-api script does, or you could use uWSGI as the barbican.sh script does. Whatever your choice of web server will not affect how Barbican works.
The barbican-api script that runs Barbican using paste.httpserver is a very lightweight script to get Barbican running quickly in development environments without any additional requirements. The barbican.sh script is a very opinionated script for setting up a development environment. Note that uwsgi and pyenv are not required by Barbican itself, only for the barbican.sh script. Neither script is intended to be used for production deployments.
The Barbican team currently defers all deployment decisions to the operator, so you will have to figure out which WSGI host is right for your deployment, and create your own deployment scripts.
The reason you’re seeing 406 errors with your curl commands is because you’re not specifying an “Accept” header with your requests. You should retry the curl commands with –H “Accept: application/json” and you should see the correct responses.
Thanks,
- Douglas Mendizabal
[1] https://www.python.org/dev/peps/pep-3333/
[2] http://git.openstack.org/cgit/openstack/barbican/tree/barbican/api/app.py#n74
[3] http://uwsgi-docs.readthedocs.org/en/latest/
[4] https://code.google.com/p/modwsgi/
[5] http://gunicorn.org/
[6] http://pythonpaste.org/modules/httpserver.html
From: Asha Seshagiri <asha.seshagiri at gmail.com<mailto:asha.seshagiri at gmail.com>>
Date: Thursday, April 23, 2015 at 4:17 PM
To: Adam Harwell <adam.harwell at RACKSPACE.COM<mailto:adam.harwell at RACKSPACE.COM>>
Cc: neetu jain <nutshi at gmail.com<mailto:nutshi at gmail.com>>, John Wood <john.wood at RACKSPACE.COM<mailto:john.wood at RACKSPACE.COM>>, openstack-dev <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>, "Reller, Nathan S." <Nathan.Reller at jhuapl.edu<mailto:Nathan.Reller at jhuapl.edu>>, Douglas Mendizábal <douglas.mendizabal at rackspace.com<mailto:douglas.mendizabal at rackspace.com>>, Paul Kehrer <paul.kehrer at RACKSPACE.COM<mailto:paul.kehrer at RACKSPACE.COM>>, Alexis Lee <alexisl at hp.com<mailto:alexisl at hp.com>>
Subject: Re: Barbican : Dependency of pyenv configuration in Barbican.sh script
Hi All ,
Would need help!
I tried executing the script present in the link https://github.com/openstack/barbican/blob/master/bin/barbican-api to start the barbican instance but the use cases of barbican are failing.
Please find the details of the investigations :
Usecase for posting and retrieving the secret.
[root at barbican-keystone2 ~]# curl -X POST -H 'content-type:application/json' -H 'X-Project-Id:12345' -d '{"payload": "my-secret-here", "payload_content_type": "text/plain"}' http://localhost:9311/v1/secrets
{"code": 406, "description": null, "title": "Not Acceptable"}[root at barbican-keystone2 ~]#
[root at barbican-keystone2 ~]# curl -H 'Accept: application/json' -H 'X-Project-Id:12345' http://localhost:9311/v1/secrets
{"code": 406, "description": null, "title": "Not Acceptable"}[root at barbican-keystone2 ~]#
[root at barbican-keystone2 ~]#
[root at barbican-keystone2 ~]# curl -X POST -H 'content-type:application/json' -H 'X-Project-Id:12345' -d '{"payload": "my-secret-here", "payload_content_type": "text/plain"}' http://127.0.0.1:9311/v1/secrets
{"code": 406, "description": null, "title": "Not Acceptable"}[root at barbican-keystone2 ~]#
The output of the ps command when the barbican instance is stood up using the python script as pointed in the above link
We do not see the instance of uwsgi in the response:
[root at barbican-keystone2 ~]# ps -ef | grep barbican
avahi 2920 1 0 Apr22 ? 00:00:00 avahi-daemon: running [barbican-keystone2.local]
root 14743 14554 2 14:54 pts/1 00:00:01 python bin/barbican-api
root 14781 13975 0 14:55 pts/0 00:00:00 grep --color=auto barbican
The output of the ps command when the barbican instance is stood up using the barbican.sh script
[root at barbican-keystone2 ~]# ps -ef | grep barbican
avahi 2920 1 0 Apr22 ? 00:00:00 avahi-daemon: running [barbican- keystone2.local]
root 14577 14554 0 14:50 pts/1 00:00:00 /bin/bash bin/barbican.sh start
root 14582 14577 0 14:50 pts/1 00:00:00 uwsgi --master --emperor /etc/barbican/vassals
root 14583 14582 0 14:50 pts/1 00:00:00 uwsgi --master --emperor /etc/barbican/vassals
root 14584 14583 0 14:50 pts/1 00:00:00 /usr/bin/uwsgi --ini barbican-admin.ini
root 14585 14583 0 14:50 pts/1 00:00:00 /usr/bin/uwsgi --ini barbican-api.ini
root 14586 14584 10 14:50 pts/1 00:00:01 /usr/bin/uwsgi --ini barbican-admin.ini
root 14587 14585 12 14:50 pts/1 00:00:01 /usr/bin/uwsgi --ini barbican-api.ini
root 14601 13975 0 14:50 pts/0 00:00:00 grep --color=auto barbican
The barbican instance needs to be started on top of uswgi server instance since uwsgi is the webserver which serves the request for barbican services. The script does not start the uwsgi server
Please correct me if I am wrong.
Any help would be highly appreciated.
Thanks and Regards,
Asha Seshagiri
On Thu, Apr 23, 2015 at 11:27 AM, Asha Seshagiri <asha.seshagiri at gmail.com<mailto:asha.seshagiri at gmail.com>> wrote:
Hi All,
Thanks Adam for your response.
I am able to run the barbican-api script without SQLLite installation .I guess SQLLite comes configured with barbican installation .Please correct me if I am wrong.
[root at barbican-keystone2 barbican]# bin/barbican-api
2015-04-23 11:12:31.571 8265 INFO barbican.model.repositories [-] Setting up database engine and session factory
2015-04-23 11:12:31.640 8265 INFO barbican.model.repositories [-] Updating schema to latest version
2015-04-23 11:12:31.640 8265 WARNING barbican.model.migration.commands [-] !!! Limited support for migration commands using sqlite databases; This operation may not succeed.
2015-04-23 11:12:31.643 8265 INFO alembic.migration [-] Context impl SQLiteImpl.
2015-04-23 11:12:31.644 8265 INFO alembic.migration [-] Will assume non-transactional DDL.
serving on http://127.0.0.1:9311
I would like to get confirmation from the team that barbican-api would be used only to standup the barbican instance , for installation of barbican ,debugging and stoping the barbican instance , we still need to use barbican.sh script .
Any help would highly be appreciated.
Thanks and Regards,
Asha Seshagiri
On Thu, Apr 23, 2015 at 10:28 AM, Adam Harwell <adam.harwell at rackspace.com<mailto:adam.harwell at rackspace.com>> wrote:
Do you have sqlite installed on your system, and do you have config.py in the root of your barbican directory? The database is configured there (assuming it hasn’t changed since I last ran Barbican locally), and mine looks like this:
config = {
'sqlalchemy': {
'url': 'sqlite:////tmp/barbican.db',
'echo': True,
'echo_pool': False,
'pool_recycle': 3600,
'encoding': 'utf-8'
}
}
--Adam
https://keybase.io/rm_you
From: neetu jain <nutshi at gmail.com<mailto:nutshi at gmail.com>>
Date: Thursday, April 23, 2015 at 10:07 AM
To: Asha Seshagiri <asha.seshagiri at gmail.com<mailto:asha.seshagiri at gmail.com>>
Cc: John Wood <john.wood at RACKSPACE.COM<mailto:john.wood at RACKSPACE.COM>>, openstack-dev <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>, "Reller, Nathan S." <Nathan.Reller at jhuapl.edu<mailto:Nathan.Reller at jhuapl.edu>>, Douglas Mendizabal <douglas.mendizabal at RACKSPACE.COM<mailto:douglas.mendizabal at RACKSPACE.COM>>, Paul Kehrer <paul.kehrer at RACKSPACE.COM<mailto:paul.kehrer at RACKSPACE.COM>>, Adam Harwell <adam.harwell at rackspace.com<mailto:adam.harwell at rackspace.com>>, Alexis Lee <alexisl at hp.com<mailto:alexisl at hp.com>>
Subject: Re: Barbican : Dependency of pyenv configuration in Barbican.sh script
Thanks John for you answer.
I tried running the script bin/barbican-api and ran into this issue (pasted at the end) . Seems like the script does not take care of the database side.
1) do we need to do something else to setup database? or its being worked on ?
2) Can we help in the process of removing dependencies in these scripts? Should that be through the launchpad ?
TASK: [barbican | install barbican] *******************************************
failed: [barbican-04] => {"changed": true, "cmd": "cd /root/barbican/; python bin/barbican-api", "delta": "0:00:00.553279", "end": "2015-04-23 14:56:45.773115", "rc": 1, "start": "2015-04-23 14:56:45.219836", "warnings": []}
stderr: 2015-04-23 14:56:45.736 6984 CRITICAL barbican [-] BarbicanException: No SQL connection configured
2015-04-23 14:56:45.736 6984 TRACE barbican Traceback (most recent call last):
2015-04-23 14:56:45.736 6984 TRACE barbican File "bin/barbican-api", line 17, in <module>
2015-04-23 14:56:45.736 6984 TRACE barbican run()
2015-04-23 14:56:45.736 6984 TRACE barbican File "bin/barbican-api", line 12, in run
2015-04-23 14:56:45.736 6984 TRACE barbican relative_to='.')
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
2015-04-23 14:56:45.736 6984 TRACE barbican return loadobj(APP, uri, name=name, **kw)
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 272, in loadobj
2015-04-23 14:56:45.736 6984 TRACE barbican return context.create()
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2015-04-23 14:56:45.736 6984 TRACE barbican return self.object_type.invoke(self)
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 144, in invoke
2015-04-23 14:56:45.736 6984 TRACE barbican **context.local_conf)
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/util.py", line 56, in fix_call
2015-04-23 14:56:45.736 6984 TRACE barbican val = callable(*args, **kw)
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib64/python2.7/site-packages/paste/urlmap.py", line 31, in urlmap_factory
2015-04-23 14:56:45.736 6984 TRACE barbican app = loader.get_app(app_name, global_conf=global_conf)
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 350, in get_app
2015-04-23 14:56:45.736 6984 TRACE barbican name=name, global_conf=global_conf).create()
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2015-04-23 14:56:45.736 6984 TRACE barbican return self.object_type.invoke(self)
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 203, in invoke
2015-04-23 14:56:45.736 6984 TRACE barbican app = context.app_context.create()
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 710, in create
2015-04-23 14:56:45.736 6984 TRACE barbican return self.object_type.invoke(self)
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 146, in invoke
2015-04-23 14:56:45.736 6984 TRACE barbican return fix_call(context.object, context.global_conf, **context.local_conf)
2015-04-23 14:56:45.736 6984 TRACE barbican File "/usr/lib/python2.7/site-packages/paste/deploy/util.py", line 56, in fix_call
2015-04-23 14:56:45.736 6984 TRACE barbican val = callable(*args, **kw)
2015-04-23 14:56:45.736 6984 TRACE barbican File "/root/barbican/barbican/api/app.py", line 89, in create_main_app
2015-04-23 14:56:45.736 6984 TRACE barbican repositories.setup_database_engine_and_factory()
2015-04-23 14:56:45.736 6984 TRACE barbican File "/root/barbican/barbican/model/repositories.py", line 109, in setup_database_engine_and_factory
2015-04-23 14:56:45.736 6984 TRACE barbican _ENGINE = _get_engine(_ENGINE)
2015-04-23 14:56:45.736 6984 TRACE barbican File "/root/barbican/barbican/model/repositories.py", line 170, in _get_engine
2015-04-23 14:56:45.736 6984 TRACE barbican u._('No SQL connection configured'))
2015-04-23 14:56:45.736 6984 TRACE barbican BarbicanException: No SQL connection configured
2015-04-23 14:56:45.736 6984 TRACE barbican
FATAL: all hosts have already failed -- aborting
On Wed, Apr 22, 2015 at 11:50 PM, Asha Seshagiri <asha.seshagiri at gmail.com<mailto:asha.seshagiri at gmail.com>> wrote:
Thanks a lot John for your response.
I appreciate for your time and effort in answering the queries and also pointing to the latest changes which you been always doing :)
Thanks and Regards,
Asha Seshagiri
On Wed, Apr 22, 2015 at 6:09 PM, John Wood <john.wood at rackspace.com<mailto:john.wood at rackspace.com>> wrote:
Hello Asha,
The barbican.sh script was originally intended to be a convenient way to boot up a Barbican instance locally to quickly start evaluating its API and functionality.
It was not intended to be used as a production script, deferring instead to deployments utilizing packages such as RDO RPMs and so forth for that purpose.
That said, changes to that script have been discussed, including removing pyenv and uWSGI as dependencies, hence such changes would be good to consider.
I’d also note that a solution based on this recently added script [1] might be in order.
Thanks,
John
[1] https://github.com/openstack/barbican/blob/master/bin/barbican-api
From: Asha Seshagiri <asha.seshagiri at gmail.com<mailto:asha.seshagiri at gmail.com>>
Date: Wednesday, April 22, 2015 at 4:57 PM
To: openstack-dev <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Cc: John Wood <john.wood at rackspace.com<mailto:john.wood at rackspace.com>>, "Reller, Nathan S." <Nathan.Reller at jhuapl.edu<mailto:Nathan.Reller at jhuapl.edu>>, Douglas Mendizabal <douglas.mendizabal at RACKSPACE.COM<mailto:douglas.mendizabal at RACKSPACE.COM>>, Paul Kehrer <paul.kehrer at RACKSPACE.COM<mailto:paul.kehrer at RACKSPACE.COM>>, Adam Harwell <adam.harwell at RACKSPACE.COM<mailto:adam.harwell at RACKSPACE.COM>>, Alexis Lee <alexisl at hp.com<mailto:alexisl at hp.com>>, "nutshi at gmail.com<mailto:nutshi at gmail.com>" <nutshi at gmail.com<mailto:nutshi at gmail.com>>
Subject: Barbican : Dependency of pyenv configuration in Barbican.sh script
Hi All,
I would like to know the reason behind the dependency of the pyenv virtual environment and pyenv in the barbican.sh script.
Ideally in the production environment , barbican would run on standalone virtual box with a particular python version .I feel that their dependecies needs to be removed from the script.
Was able to stand up the barbican instance without configuring pyenv and pyenv-virtualenv dependencies by modifying the barbican script , installing few additional packages and exporting the python path to PATH variable
Please find the change in barbican.sh script for installation and starting of the script below :
VENV_DIR=${VIRTUAL_ENV:-`pyenv prefix`} -> This line needs to be removed
uwsgi --master --emperor $CONFIG_DIR/vassals -H $VENV_DIR -> The $VENV_DIR variable need to be removed as an argument and -H as an option.
The barbican script has been tied to $VENV_DIR variable which is dependent on the pyenv for python configuration.Hence the barbican.sh script needs to be modified to remove $VENV_DIR variable by configuring python path in PATH variable.
On doing this , we can avoid the sourcing the pyenv and pyenv-virtualenv packages and its dependices on Barbican script.
Any help would be highly appreciated and also would like to know opinion from the openstack group on the changes indicated
Thanks in advance
Thanks and Regards,
Asha Seshagiri
--
Thanks and Regards,
Asha Seshagiri
--
Thanks and Regards,
Asha Seshagiri
--
Thanks and Regards,
Asha Seshagiri
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150423/8514d1cc/attachment.html>
More information about the OpenStack-dev
mailing list