Devstack Installation Fails with Internal Server Error (HTTP 500)
List, I am trying Devstack installation on a Virtual Machine to try TROVE DbaaS. All the time ./stack.sh command stops with the same error message at this particular point + functions:_upload_image:121 : openstack --os-cloud=devstack-admin --os-region-name=RegionOne image create cirros-0.5.2-x86_64-disk --public --container-format bare --disk-format qcow2 --property hw_rng_model=virtio Internal Server Error (HTTP 500) I have ./unstack.sh && ./clean.sh and re run ./stack.sh multiple time but no progress. Here always the script exit with error . Screenshot attached (using "stack" user created by the devstack/tool/ stack user script) my local.conf file as follows here 10.184.48.187 is my VMs IP Any hints to rectify this error most welcome!! ############################################################ [[local|localrc]] RECLONE=False HOST_IP=10.184.48.187 enable_plugin trove https://opendev.org/openstack/trove enable_plugin trove-dashboard https://opendev.org/openstack/trove-dashboard LIBS_FROM_GIT+=,python-troveclient DATABASE_PASSWORD=password ADMIN_PASSWORD=password SERVICE_PASSWORD=password SERVICE_TOKEN=password RABBIT_PASSWORD=password LOGFILE=$DEST/logs/stack.sh.log VERBOSE=True LOG_COLOR=False LOGDAYS=1 IPV4_ADDRS_SAFE_TO_USE=10.111.0.0/26 FIXED_RANGE=10.111.0.0/26 NETWORK_GATEWAY=10.111.0.1 FLOATING_RANGE=10.184.48.0/24 PUBLIC_NETWORK_GATEWAY=10.184.48.1 #pre requisites ENABLED_SERVICES=rabbit,mysql,key #nova enable_service horizon enable_service n-api enable_service n-cpu enable_service n-cond enable_service n-sch enable_service n-api-meta enable_service placement-api enable_service placement-client #Glance enable_service g-api enable_service g-reg #Cinder enable_service cinder enable_service c-api enable_service c-vol enable_service c-sch #Neutron enable_service q-svc enable_service q-agt enable_service q-dhcp enable_service q-l3 enable_service q-meta #enable DVR Q_AGENT=openvswitch Q_DVR_MODE=legacy Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch Q_ML2_TENANT_NETWORK_TYPE=vxlan Q_PLUGIN=ml2 #SWIFT ENABLED_SERVICES+=,swift SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5 SWIFT_REPLICAS=1 stack@XenaCtrl1:/home/cloud/trove_devstack/devstack$ #################################################################
On Fri, Jun 10, 2022, at 6:01 AM, KK CHN wrote:
List,
I am trying Devstack installation on a Virtual Machine to try TROVE DbaaS.
All the time ./stack.sh command stops with the same error message at this particular point
+ functions:_upload_image:121 : openstack --os-cloud=devstack-admin --os-region-name=RegionOne image create cirros-0.5.2-x86_64-disk --public --container-format bare --disk-format qcow2 --property hw_rng_model=virtio Internal Server Error (HTTP 500)
You should examine the glance api logs to see why the service had an error.
I have ./unstack.sh && ./clean.sh and re run ./stack.sh multiple time but no progress. Here always the script exit with error . Screenshot attached (using "stack" user created by the devstack/tool/ stack user script)
my local.conf file as follows here 10.184.48.187 is my VMs IP
Any hints to rectify this error most welcome!!
############################################################ [[local|localrc]] RECLONE=False HOST_IP=10.184.48.187
enable_plugin trove https://opendev.org/openstack/trove enable_plugin trove-dashboard https://opendev.org/openstack/trove-dashboard
LIBS_FROM_GIT+=,python-troveclient DATABASE_PASSWORD=password ADMIN_PASSWORD=password SERVICE_PASSWORD=password SERVICE_TOKEN=password RABBIT_PASSWORD=password LOGFILE=$DEST/logs/stack.sh.log VERBOSE=True LOG_COLOR=False LOGDAYS=1
IPV4_ADDRS_SAFE_TO_USE=10.111.0.0/26 FIXED_RANGE=10.111.0.0/26 NETWORK_GATEWAY=10.111.0.1 FLOATING_RANGE=10.184.48.0/24 PUBLIC_NETWORK_GATEWAY=10.184.48.1
#pre requisites ENABLED_SERVICES=rabbit,mysql,key
#nova enable_service horizon
enable_service n-api
enable_service n-cpu
enable_service n-cond
enable_service n-sch enable_service n-api-meta enable_service placement-api enable_service placement-client
#Glance
enable_service g-api enable_service g-reg
#Cinder enable_service cinder enable_service c-api enable_service c-vol
enable_service c-sch
#Neutron
enable_service q-svc enable_service q-agt
enable_service q-dhcp enable_service q-l3 enable_service q-meta
#enable DVR Q_AGENT=openvswitch Q_DVR_MODE=legacy Q_ML2_PLUGIN_MECHANISM_DRIVERS=openvswitch Q_ML2_TENANT_NETWORK_TYPE=vxlan Q_PLUGIN=ml2
#SWIFT ENABLED_SERVICES+=,swift SWIFT_HASH=66a3d6b56c1f479c8b4e70ab5c2000f5 SWIFT_REPLICAS=1 stack@XenaCtrl1:/home/cloud/trove_devstack/devstack$
#################################################################
Attachments: * DevstackError1.png
You should examine the glance api logs to see why the service had an error.
This is actually not a glance problem, but a case of missing the dbcounter plugin when glance connects to the DB -- uploading an image is just the first thing we try to do, other than keystone stuff. I've been working with them on IRC and their devstack log clearly shows it is installed successfully, so I suspect something else is going on. They dropped out before we finished, but I'll continue chasing it down. --Dan
On Mon, 2022-06-13 at 12:46 -0700, Dan Smith wrote:
You should examine the glance api logs to see why the service had an error.
This is actually not a glance problem, but a case of missing the dbcounter plugin when glance connects to the DB -- uploading an image is just the first thing we try to do, other than keystone stuff. I've been working with them on IRC and their devstack log clearly shows it is installed successfully, so I suspect something else is going on. They dropped out before we finished, but I'll continue chasing it down. i hit the same issue the other day if you recall me asking about it. didnt check my devstack logs to see if it installed or not but i can see if i can repoduce it tomorrow.
in theory my ansibel role should hit it if its repoduceable reliable. i havent commit the change to my repo to disabel the perfromace counters so i shoudl be able to repoduce it either on my home server or laptop which ever does not have the change locally. ill let you know if i can and i can proably give you access to the vm if you want to take a look.
--Dan
i hit the same issue the other day if you recall me asking about it. didnt check my devstack logs to see if it installed or not but i can see if i can repoduce it tomorrow.
I checked with the OP and they have it installed properly, system-wide. I was assuming your issue might have been venv (or ordering) related, but theirs was not. There are some other weird things going on with their setup, so I'm not so sure. --Dan
On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote:
i hit the same issue the other day if you recall me asking about it. didnt check my devstack logs to see if it installed or not but i can see if i can repoduce it tomorrow.
I checked with the OP and they have it installed properly, system-wide. I was assuming your issue might have been venv (or ordering) related, but theirs was not. There are some other weird things going on with their setup, so I'm not so sure. nope, was not useing a vnev or anything special, it just devstack installed normaly more or less but driven by the ansible roles. the run devstack rull litrally just opens a shell and invokes stack.sh so i dont think there was anything specific ot how i was invoking it.
--Dan
This is my latest devstack installation error.. No clues why it fails repeatedly; earlier it was during the uploading stage of the cirros image, internal server error ( HTTP 500 ) occurred. This was repeated multiple times. So I posted for the first time in this thread. In this iteration for starting afresh, I have removed the stack user and /opt/stack directory manually, and freshly created 'stack' user with sudo permissions and /opt/stack directory with stack:stack ownership, and fresh git cloned(git clone https://opendev.org/openstack/devstack ) to /opt/DEVSTACK_stack folder and created local.conf ( attached here) . Then executed ./stack.sh . Got the following error this time ... Any clues why it fails here ? Platform (on a VM with Debian uname output) : Linux XenaCtrl1 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux [image: Trove_installationError.png] On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney <smooney@redhat.com> wrote:
On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote:
i hit the same issue the other day if you recall me asking about it. didnt check my devstack logs to see if it installed or not but i can see if i can repoduce it tomorrow.
I checked with the OP and they have it installed properly, system-wide. I was assuming your issue might have been venv (or ordering) related, but theirs was not. There are some other weird things going on with their setup, so I'm not so sure. nope, was not useing a vnev or anything special, it just devstack installed normaly more or less but driven by the ansible roles. the run devstack rull litrally just opens a shell and invokes stack.sh so i dont think there was anything specific ot how i was invoking it.
--Dan
Update to my Devstack installation Error. I have crossed this Internal Error ( HTTP 500) and ended up with a new error for the Trove Devstack Installation attempts. ################################################## 2022-06-16 05:46:40.183 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:26 : total_kB=16393076 2022-06-16 05:46:40.188 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:29 : RAM_NEEDED=4 2022-06-16 05:46:40.192 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 : '[' 16393076 -lt 4194304 ']' 2022-06-16 05:46:40.197 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 : return 0 2022-06-16 05:46:40.202 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:236 : timeout 120 sh -c 'while ! sudo umount -f /tmp/dib_image.WMpKoawa; do sleep 1; done' 2022-06-16 05:46:40.225 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:241 : rm -rf --one-file-system /tmp/dib_image.WMpKoawa 2022-06-16 05:46:40.231 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 : exit 1 Error on exit # Warning: iptables-legacy tables present, use iptables-legacy to see them # Warning: iptables-legacy tables present, use iptables-legacy to see them # Warning: iptables-legacy tables present, use iptables-legacy to see them stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ ############################################################################# I am able to login to the Horizon dash board inspite the ./stack.sh script Exit with the above error status.. And in the image section, I am able to view cirros-0.5.2-x86_64-disk <http://10.184.48.187/dashboard/ngdetails/OS::Glance::Image/be0c8769-286c-4b61-b989-a037e0e9601c> image in the dashboard. But on the terminal when I issue the command DEVSTACK_TROVE/devstack$ openstack datastore version list mysql it says " Datastore 'mysql' cannot found. HTTP(404). #############Here the tty output stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ source ./openrc WARNING: setting legacy OS_TENANT_NAME to support cli tools. stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore version list mysql Datastore 'mysql' cannot be found. (HTTP 404) stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack server list stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack image list +--------------------------------------+--------------------------+--------+ | ID | Name | Status | +--------------------------------------+--------------------------+--------+ | be0c8769-286c-4b61-b989-a037e0e9601c | cirros-0.5.2-x86_64-disk | active | +--------------------------------------+--------------------------+--------+ stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore version list mysql Datastore 'mysql' cannot be found. (HTTP 404) ####################################################################################### So I ended up with the TROVE installation failing ? Or what does this error mean ? How can I fix this to get a devstack installed successfully with Trove-DbaaS feature up and running ? Any help / directions much appreciated .... [ How I rectify the earlier errors of HTTP 500 : I just created a VM again with same debian 11 image and 'stack' user created with /opt/stack dir and freshly git cloned devstack to DEVSTACK_TROVE/ Dir.. Then created the same local.conf file which I posted in my earlier mail. and reran ./stack.sh script . This time I was able to get the dashboard up and be able to login(earlier it's not possible to get the dashboard too) ] Krish On Wed, Jun 15, 2022 at 12:31 PM KK CHN <kkchn.in@gmail.com> wrote:
This is my latest devstack installation error.. No clues why it fails repeatedly; earlier it was during the uploading stage of the cirros image, internal server error ( HTTP 500 ) occurred. This was repeated multiple times. So I posted for the first time in this thread.
In this iteration for starting afresh, I have removed the stack user and /opt/stack directory manually, and freshly created 'stack' user with sudo permissions and /opt/stack directory with stack:stack ownership, and fresh git cloned(git clone https://opendev.org/openstack/devstack ) to /opt/DEVSTACK_stack folder and created local.conf ( attached here) . Then executed ./stack.sh .
Got the following error this time ... Any clues why it fails here ? Platform (on a VM with Debian uname output) : Linux XenaCtrl1 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux
[image: Trove_installationError.png]
On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney <smooney@redhat.com> wrote:
On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote:
i hit the same issue the other day if you recall me asking about it. didnt check my devstack logs to see if it installed or not but i can see if i can repoduce it tomorrow.
I checked with the OP and they have it installed properly, system-wide. I was assuming your issue might have been venv (or ordering) related, but theirs was not. There are some other weird things going on with their setup, so I'm not so sure. nope, was not useing a vnev or anything special, it just devstack installed normaly more or less but driven by the ansible roles. the run devstack rull litrally just opens a shell and invokes stack.sh so i dont think there was anything specific ot how i was invoking it.
--Dan
Please find https://pastebin.com/yHgFKcx4 Complete Error Log (Note the line : 464 also ) On Thu, Jun 16, 2022 at 11:51 AM KK CHN <kkchn.in@gmail.com> wrote:
Update to my Devstack installation Error.
I have crossed this Internal Error ( HTTP 500) and ended up with a new error for the Trove Devstack Installation attempts. ################################################## 2022-06-16 05:46:40.183 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:26 : total_kB=16393076 2022-06-16 05:46:40.188 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:29 : RAM_NEEDED=4 2022-06-16 05:46:40.192 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 : '[' 16393076 -lt 4194304 ']' 2022-06-16 05:46:40.197 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 : return 0 2022-06-16 05:46:40.202 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:236 : timeout 120 sh -c 'while ! sudo umount -f /tmp/dib_image.WMpKoawa; do sleep 1; done' 2022-06-16 05:46:40.225 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:241 : rm -rf --one-file-system /tmp/dib_image.WMpKoawa 2022-06-16 05:46:40.231 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 : exit 1 Error on exit # Warning: iptables-legacy tables present, use iptables-legacy to see them # Warning: iptables-legacy tables present, use iptables-legacy to see them # Warning: iptables-legacy tables present, use iptables-legacy to see them stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$
#############################################################################
I am able to login to the Horizon dash board inspite the ./stack.sh script Exit with the above error status.. And in the image section, I am able to view cirros-0.5.2-x86_64-disk <http://10.184.48.187/dashboard/ngdetails/OS::Glance::Image/be0c8769-286c-4b61-b989-a037e0e9601c> image in the dashboard.
But on the terminal when I issue the command DEVSTACK_TROVE/devstack$ openstack datastore version list mysql it says " Datastore 'mysql' cannot found. HTTP(404).
#############Here the tty output stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ source ./openrc WARNING: setting legacy OS_TENANT_NAME to support cli tools.
stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore version list mysql Datastore 'mysql' cannot be found. (HTTP 404)
stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack server list
stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack image list
+--------------------------------------+--------------------------+--------+ | ID | Name | Status |
+--------------------------------------+--------------------------+--------+ | be0c8769-286c-4b61-b989-a037e0e9601c | cirros-0.5.2-x86_64-disk | active |
+--------------------------------------+--------------------------+--------+ stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore version list mysql Datastore 'mysql' cannot be found. (HTTP 404)
#######################################################################################
So I ended up with the TROVE installation failing ? Or what does this error mean ? How can I fix this to get a devstack installed successfully with Trove-DbaaS feature up and running ?
Any help / directions much appreciated ....
[ How I rectify the earlier errors of HTTP 500 : I just created a VM again with same debian 11 image and 'stack' user created with /opt/stack dir and freshly git cloned devstack to DEVSTACK_TROVE/ Dir.. Then created the same local.conf file which I posted in my earlier mail. and reran ./stack.sh script . This time I was able to get the dashboard up and be able to login(earlier it's not possible to get the dashboard too) ]
Krish
On Wed, Jun 15, 2022 at 12:31 PM KK CHN <kkchn.in@gmail.com> wrote:
This is my latest devstack installation error.. No clues why it fails repeatedly; earlier it was during the uploading stage of the cirros image, internal server error ( HTTP 500 ) occurred. This was repeated multiple times. So I posted for the first time in this thread.
In this iteration for starting afresh, I have removed the stack user and /opt/stack directory manually, and freshly created 'stack' user with sudo permissions and /opt/stack directory with stack:stack ownership, and fresh git cloned(git clone https://opendev.org/openstack/devstack ) to /opt/DEVSTACK_stack folder and created local.conf ( attached here) . Then executed ./stack.sh .
Got the following error this time ... Any clues why it fails here ? Platform (on a VM with Debian uname output) : Linux XenaCtrl1 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux
[image: Trove_installationError.png]
On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney <smooney@redhat.com> wrote:
i hit the same issue the other day if you recall me asking about it. didnt check my devstack logs to see if it installed or not but i can see if i can repoduce it tomorrow.
I checked with the OP and they have it installed properly, system-wide. I was assuming your issue might have been venv (or ordering) related, but theirs was not. There are some other weird
On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote: things
going on with their setup, so I'm not so sure. nope, was not useing a vnev or anything special, it just devstack installed normaly more or less but driven by the ansible roles. the run devstack rull litrally just opens a shell and invokes stack.sh so i dont think there was anything specific ot how i was invoking it.
--Dan
I have tried the Trove Devstack installation on Ubuntu20.04 too.. But no success.. ( As I have made no progress in Debian 11 Trove-Devstak installation, I tried a number of times but all failed with the error ). To my experience.. The same error for Ubuntu 20.04 also. Followed the same stuff : https://docs.openstack.org/trove/latest/install/install-devstack.html The compete Error log here https://paste.openstack.org/show/bwSXZXHRtHWHnaOO2Fb4/ It reports an Error starting with 2022-06-17 09:58:04.720 | ERROR: Cannot install trove==17.1.0.dev31 because these package versions have conflicting dependencies. 2022-06-17 09:58:04.721 | 2022-06-17 09:58:04.721 | The conflict is caused by: 2022-06-17 09:58:04.721 | trove 17.1.0.dev31 depends on Jinja2>=2.10 2022-06-17 09:58:04.721 | The user requested (constraint) jinja2===3.1.2 How to overcome this Jinja dependency, where we need to edit this and what jinja suitable ? 2022-06-17 09:58:04.721 | And finally Exit with error 2022-06-17 09:58:07.560 | + /usr/local/lib/python3.8/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 : exit 1 Error on exit stack@kk:~/devstack$ uname -a Linux kk 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11 15:58:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux stack@kk:~/devstack$ On Thu, Jun 16, 2022 at 11:51 AM KK CHN <kkchn.in@gmail.com> wrote:
Update to my Devstack installation Error.
I have crossed this Internal Error ( HTTP 500) and ended up with a new error for the Trove Devstack Installation attempts. ################################################## 2022-06-16 05:46:40.183 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:26 : total_kB=16393076 2022-06-16 05:46:40.188 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:29 : RAM_NEEDED=4 2022-06-16 05:46:40.192 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 : '[' 16393076 -lt 4194304 ']' 2022-06-16 05:46:40.197 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:tmpfs_check:30 : return 0 2022-06-16 05:46:40.202 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:236 : timeout 120 sh -c 'while ! sudo umount -f /tmp/dib_image.WMpKoawa; do sleep 1; done' 2022-06-16 05:46:40.225 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/common-functions:cleanup_image_dir:241 : rm -rf --one-file-system /tmp/dib_image.WMpKoawa 2022-06-16 05:46:40.231 | + /usr/local/lib/python3.9/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 : exit 1 Error on exit # Warning: iptables-legacy tables present, use iptables-legacy to see them # Warning: iptables-legacy tables present, use iptables-legacy to see them # Warning: iptables-legacy tables present, use iptables-legacy to see them stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$
#############################################################################
I am able to login to the Horizon dash board inspite the ./stack.sh script Exit with the above error status.. And in the image section, I am able to view cirros-0.5.2-x86_64-disk <http://10.184.48.187/dashboard/ngdetails/OS::Glance::Image/be0c8769-286c-4b61-b989-a037e0e9601c> image in the dashboard.
But on the terminal when I issue the command DEVSTACK_TROVE/devstack$ openstack datastore version list mysql it says " Datastore 'mysql' cannot found. HTTP(404).
#############Here the tty output stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ source ./openrc WARNING: setting legacy OS_TENANT_NAME to support cli tools.
stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore version list mysql Datastore 'mysql' cannot be found. (HTTP 404)
stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack server list
stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack image list
+--------------------------------------+--------------------------+--------+ | ID | Name | Status |
+--------------------------------------+--------------------------+--------+ | be0c8769-286c-4b61-b989-a037e0e9601c | cirros-0.5.2-x86_64-disk | active |
+--------------------------------------+--------------------------+--------+ stack@XenaCtrl2:/home/cloud/DEVSTACK_TROVE/devstack$ openstack datastore version list mysql Datastore 'mysql' cannot be found. (HTTP 404)
#######################################################################################
So I ended up with the TROVE installation failing ? Or what does this error mean ? How can I fix this to get a devstack installed successfully with Trove-DbaaS feature up and running ?
Any help / directions much appreciated ....
[ How I rectify the earlier errors of HTTP 500 : I just created a VM again with same debian 11 image and 'stack' user created with /opt/stack dir and freshly git cloned devstack to DEVSTACK_TROVE/ Dir.. Then created the same local.conf file which I posted in my earlier mail. and reran ./stack.sh script . This time I was able to get the dashboard up and be able to login(earlier it's not possible to get the dashboard too) ]
Krish
On Wed, Jun 15, 2022 at 12:31 PM KK CHN <kkchn.in@gmail.com> wrote:
This is my latest devstack installation error.. No clues why it fails repeatedly; earlier it was during the uploading stage of the cirros image, internal server error ( HTTP 500 ) occurred. This was repeated multiple times. So I posted for the first time in this thread.
In this iteration for starting afresh, I have removed the stack user and /opt/stack directory manually, and freshly created 'stack' user with sudo permissions and /opt/stack directory with stack:stack ownership, and fresh git cloned(git clone https://opendev.org/openstack/devstack ) to /opt/DEVSTACK_stack folder and created local.conf ( attached here) . Then executed ./stack.sh .
Got the following error this time ... Any clues why it fails here ? Platform (on a VM with Debian uname output) : Linux XenaCtrl1 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux
[image: Trove_installationError.png]
On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney <smooney@redhat.com> wrote:
i hit the same issue the other day if you recall me asking about it. didnt check my devstack logs to see if it installed or not but i can see if i can repoduce it tomorrow.
I checked with the OP and they have it installed properly, system-wide. I was assuming your issue might have been venv (or ordering) related, but theirs was not. There are some other weird
On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote: things
going on with their setup, so I'm not so sure. nope, was not useing a vnev or anything special, it just devstack installed normaly more or less but driven by the ansible roles. the run devstack rull litrally just opens a shell and invokes stack.sh so i dont think there was anything specific ot how i was invoking it.
--Dan
On 2022-06-17 18:42:16 +0200 (+0200), tjoen wrote:
On 6/17/22 12:16, KK CHN wrote: ...
2022-06-17 09:58:04.720 | ERROR: Cannot install trove==17.1.0.dev31 ... 2022-06-17 09:58:04.721 | The user requested (constraint) jinja2===3.1.2
Has to be a packaging error: one "=" too many
The "===" is valid syntax for pip. It means "exactly this version" rather than merely "equivalent to this version." That particular line is currently coming from here: https://opendev.org/openstack/requirements/src/commit/46db00e/upper-constrai... -- Jeremy Stanley
On 6/17/22 18:48, Jeremy Stanley wrote:
On 2022-06-17 18:42:16 +0200 (+0200), tjoen wrote:
On 6/17/22 12:16, KK CHN wrote:
2022-06-17 09:58:04.721 | The user requested (constraint) jinja2===3.1.2
Has to be a packaging error: one "=" too many
The "===" is valid syntax for pip. It means "exactly this version"
OK, never too old to learn something new
On Fri, Jun 17, 2022, at 3:16 AM, KK CHN wrote:
I have tried the Trove Devstack installation on Ubuntu20.04 too.. But no success.. ( As I have made no progress in Debian 11 Trove-Devstak installation, I tried a number of times but all failed with the error ).
To my experience.. The same error for Ubuntu 20.04 also. Followed the same stuff : https://docs.openstack.org/trove/latest/install/install-devstack.html
The compete Error log here https://paste.openstack.org/show/bwSXZXHRtHWHnaOO2Fb4/
It reports an Error starting with 2022-06-17 09:58:04.720 | ERROR: Cannot install trove==17.1.0.dev31 because these package versions have conflicting dependencies. 2022-06-17 09:58:04.721 | 2022-06-17 09:58:04.721 | The conflict is caused by: 2022-06-17 09:58:04.721 | trove 17.1.0.dev31 depends on Jinja2>=2.10 2022-06-17 09:58:04.721 | The user requested (constraint) jinja2===3.1.2How to overcome this Jinja dependency, where we need to edit this and what jinja suitable ?2022-06-17 09:58:04.721 | And finally Exit with error
2022-06-17 09:58:07.560 | + /usr/local/lib/python3.8/dist-packages/diskimage_builder/lib/img-functions:trap_cleanup:38 : exit 1 Error on exit stack@kk:~/devstack$ uname -a Linux kk 5.11.0-27-generic #29~20.04.1-Ubuntu SMP Wed Aug 11 15:58:17 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux stack@kk:~/devstack$
You are running devstack on Ubuntu 22.04 which installs diskimage-builder under python3.8. Then the trove devstack process attempts to build a trove database image with its agent installed. The image build run by diskimage-builder appears to be running under Ubuntu 18.04 which has python3.6 and pip 9.0.1 (this pip version is what I'm using to make this inference). Unfortunately, the openstack master constraints no longer support python3.6. In particular jinja2 3.1.2 requires python3.7 or newer which is why you get the error above about there being a version conflict. Pip should report this more clearly (and I think very new pip may), but that is what it is trying to tell you. You have requested a specific version that cannot be installed due to a python interpreter error. Depending on what you are trying to achieve you may be better off installing older versions of trove and openstack (using one of the stable branches of devstack). Or as an alternative you can update the trove diskimage-builder base image from Ubuntu 18.04 to 20.04 and fix whatever new problems arise from that.
It can't reach the neutron service, that's what failing here. I'm not familiar with devstack, is there any way to verify the basic services before deploying trove or is it "all-in-one"? Anyway, in this case I'd recommend to check neutron logs and then fix it. Zitat von KK CHN <kkchn.in@gmail.com>:
This is my latest devstack installation error.. No clues why it fails repeatedly; earlier it was during the uploading stage of the cirros image, internal server error ( HTTP 500 ) occurred. This was repeated multiple times. So I posted for the first time in this thread.
In this iteration for starting afresh, I have removed the stack user and /opt/stack directory manually, and freshly created 'stack' user with sudo permissions and /opt/stack directory with stack:stack ownership, and fresh git cloned(git clone https://opendev.org/openstack/devstack ) to /opt/DEVSTACK_stack folder and created local.conf ( attached here) . Then executed ./stack.sh .
Got the following error this time ... Any clues why it fails here ? Platform (on a VM with Debian uname output) : Linux XenaCtrl1 5.10.0-9-amd64 #1 SMP Debian 5.10.70-1 (2021-09-30) x86_64 GNU/Linux
[image: Trove_installationError.png]
On Tue, Jun 14, 2022 at 8:46 PM Sean Mooney <smooney@redhat.com> wrote:
On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote:
i hit the same issue the other day if you recall me asking about it. didnt check my devstack logs to see if it installed or not but i can see if i can repoduce it tomorrow.
I checked with the OP and they have it installed properly, system-wide. I was assuming your issue might have been venv (or ordering) related, but theirs was not. There are some other weird things going on with their setup, so I'm not so sure. nope, was not useing a vnev or anything special, it just devstack installed normaly more or less but driven by the ansible roles. the run devstack rull litrally just opens a shell and invokes stack.sh so i dont think there was anything specific ot how i was invoking it.
--Dan
On Tue Jun 14 2022 08:15:57 GMT-0700 (Pacific Daylight Time), Sean Mooney <smooney@redhat.com> wrote:
On Tue, 2022-06-14 at 06:37 -0700, Dan Smith wrote:
i hit the same issue the other day if you recall me asking about it. didnt check my devstack logs to see if it installed or not but i can see if i can repoduce it tomorrow.
I checked with the OP and they have it installed properly, system-wide. I was assuming your issue might have been venv (or ordering) related, but theirs was not. There are some other weird things going on with their setup, so I'm not so sure. nope, was not useing a vnev or anything special, it just devstack installed normaly more or less but driven by the ansible roles. the run devstack rull litrally just opens a shell and invokes stack.sh so i dont think there was anything specific ot how i was invoking it.
I saw this today while trying to re-install (stack.sh) an existing devstack (unstack.sh and clean.sh). I did a fresh devstack after this and it worked without any problems. So maybe the behavior is intermittent? The error in the g-api log [1] during service startup was: Jun 16 19:25:26 node1 devstack@g-api.service[421305]: ERROR glance.common.wsgi sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.plugins:dbcounterplugin=dbcounterplugin=dbcounter The format of the above plugin message is <group>:<name> [2] which would mean the group is 'sqlalchemy.plugins' and the name is 'dbcounterplugin=dbcounterplugin=dbcounter'. I thought the latter looked strange. I thought the name would just be 'dbcounter'. The dbcounter package was installed: $ pip show dbcounter Name: dbcounter Version: 0.1 Summary: A teeny tiny dbcounter plugin for use with devstack Home-page: http://github.com/openstack/devstack Author: Dan Smith Author-email: dms@danplanet.com License: Apache Location: /usr/local/lib/python3.8/dist-packages Requires: Required-by: When I tried something like this, it imported fine: $ python3 Python 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] on linux Type "help", "copyright", "credits" or "license" for more information.
import dbcounter dbcounter <module 'dbcounter' from '/usr/local/lib/python3.8/dist-packages/dbcounter.py'>
Is it possible dbcounter was not installed at the time glance was starting up? I didn't think so given the timestamp on the package dir: $ ls -l /usr/local/lib/python3.8/dist-packages/|grep dbcounter drwxr-sr-x 2 root staff 4096 Jun 16 19:14 dbcounter-0.1.dist-info -rw-r--r-- 1 root staff 4580 Jun 16 19:14 dbcounter.py and the error above happened at Jun 16 19:25:26. I couldn't figure out anything so finally I tried restarting the g-api service and it started without any error. Subsequent restart also had no error. -melwitt [1] https://paste.openstack.org/show/bEgbMLyUDH7zBW2vj43F/ [2] https://github.com/sqlalchemy/sqlalchemy/blob/rel_1_4_37/lib/sqlalchemy/util...
Is it possible dbcounter was not installed at the time glance was starting up? I didn't think so given the timestamp on the package dir:
Certainly doesn't seem like it, based on when we do it in the devstack sequence either.
I couldn't figure out anything so finally I tried restarting the g-api service and it started without any error. Subsequent restart also had no error.
Huh, interesting. I haven't been able to repro this locally so I haven't been able to poke at it. Restarting it and having it work certainly supports your sequencing theory. I guess I'll try to chase that possibility tomorrow. If lots of people start hitting this, we might want to just disable by default and enable the performance gathering in our jobs. It's already been helpful a couple times for debugging resource usage in the gate. --Dan
On 2022-06-16 15:11:54 -0700 (-0700), melanie witt wrote: [...]
Jun 16 19:25:26 node1 devstack@g-api.service[421305]: ERROR glance.common.wsgi sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.plugins:dbcounterplugin=dbcounterplugin=dbcounter
The format of the above plugin message is <group>:<name> [2] which would mean the group is 'sqlalchemy.plugins' and the name is 'dbcounterplugin=dbcounterplugin=dbcounter'. I thought the latter looked strange. I thought the name would just be 'dbcounter'. [...]
It looks like copies of this may be getting repeatedly concatenated into the plugin variable: https://opendev.org/openstack/devstack/src/commit/44d07f300150f7297773a21503... -- Jeremy Stanley
On 2022-06-16 23:06:05 +0000 (+0000), Jeremy Stanley wrote:
On 2022-06-16 15:11:54 -0700 (-0700), melanie witt wrote: [...]
Jun 16 19:25:26 node1 devstack@g-api.service[421305]: ERROR glance.common.wsgi sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.plugins:dbcounterplugin=dbcounterplugin=dbcounter
The format of the above plugin message is <group>:<name> [2] which would mean the group is 'sqlalchemy.plugins' and the name is 'dbcounterplugin=dbcounterplugin=dbcounter'. I thought the latter looked strange. I thought the name would just be 'dbcounter'. [...]
It looks like copies of this may be getting repeatedly concatenated into the plugin variable:
https://opendev.org/openstack/devstack/src/commit/44d07f300150f7297773a21503...
Perhaps the shell is interpolating &plugin and filling it with the existing value of plugin each time that function is called, resulting in nested copies of the value? -- Jeremy Stanley
It looks like copies of this may be getting repeatedly concatenated into the plugin variable:
https://opendev.org/openstack/devstack/src/commit/44d07f300150f7297773a21503...
I don't think that's actually happening. If it was, restarting glance wouldn't help (as Melanie indicated) and it also should never really work. I was thinking that was more likely an artifact of the error message or something. Looking at the config files, they seem right. For example: https://zuul.opendev.org/t/openstack/build/ff5a7a6fc87840579b102b244f28200f/... --Dan
participants (8)
-
Clark Boylan
-
Dan Smith
-
Eugen Block
-
Jeremy Stanley
-
KK CHN
-
melanie witt
-
Sean Mooney
-
tjoen