Open Stack

Jonathan, James,

A quick thank you - I wanted to report that I finally got it working.  A
lot of fiddling around with UID/GID settings on the NFS share.  (Oddly,
Cinder didn't need this.)

One thing that had me going for a bit - I thought I'd test the NFS
permissions outside of the glance-api process by using 'su -c "touch
/var/lib/glance/images/testfile" glance'.  When it didn't work, I assumed I
didn't have NFS configured correctly.  It turns out that the above command
won't work if the shell for user glance is set to /bin/false.

So, in the case of problems I would propose this as a
debugging/verification technique - make sure that user glance can write to
the NFS share by 'chsh -s /bin/bash glance' and then the above 'su -c'
command.

-Dave

--
Dave Hall
Binghamton University
kdhall at binghamton.edu
607-760-2328 (Cell)
607-777-4641 (Office)


On Wed, Aug 3, 2022 at 11:56 AM Jonathan Rosser <
jonathan.rosser at rd.bbc.co.uk> wrote:

> You should set this up like the example given previously by James Denton,
> with the variables set in user-variables.yml, not in
> openstack-user-config.yml.
>
> For reference we run a CI job that tests glance+NFS and the configuration
> used in that is here
> https://github.com/openstack/openstack-ansible/blob/master/tests/roles/bootstrap-host/templates/user_variables_nfs.yml.j2.
> You would be able to reproduce that using an all-in-one setup by adding
> 'nfs' to the SCENARIO environment variable.
>
> If you enable debug on a service and want to watch the log, you can use
> journalctl -fu <unit-name>, it should not be necessary to specifically run
> it in the foreground.
>
> Hope this helps,
> Jonathan.
> On 03/08/2022 15:35, Dave Hall wrote:
>
> Hello,
>
> (Any help on this would be greatly appreciated.  I've been chasing this
> for 2 weeks now...)
>
> I posted the other day that I had solved this issue, but now it's back.
> The primary glance-api error message is:
>
> Error in store configuration. Adding images to store is disabled.
>
> So why is my store disabled?  Is there some command or config line that
> controls this?  Shouldn't the store be enabled by default?
>
> The relevant stanza in my openstack-user-config.yml is
>
> image_hosts:
> #  infra38:
> #    ip: 172.29.236.38
> #    container_vars:
> #      limit_container_types: glance
> #      glance_remote_client:
> #        - what: "172.29.244.27:/images"
> #          where: "/var/lib/glance/images"
> #          type: "nfs"
> #          options: "_netdev,vers=3,proto=tcp,sec=sys"
>
>   infra38:
>     ip: 172.29.236.38
>     container_vars:
>       glance_default_store: file
>       glance_nfs_local_directory: "images"
>       glance_nfs_client:
>         - server: "172.29.244.27"
>           remote_path: "/images"
>           local_path: "/var/lib/glance/images"
>           type: "nfs"
>           options: "_netdev,vers=3,proto=tcp,sec=sys,noauto,user"
>           config_overrides: "{}"
>
> The stanza that's commented out produced the same result.  My
> user_variables.yaml does not have any lines pertaining to glance.  Also,
> I've removed the stanzas for the other 2 infra hosts.
>
> The glance-api.conf file from the container is:
>
> [DEFAULT]
> # Disable stderr logging
> use_stderr = False
> debug = False
> use_journal = True
> fatal_deprecations = False
> bind_host = 172.29.238.205
> bind_port = 9292
> http_keepalive = True
> digest_algorithm = sha256
> backlog = 4096
> workers = 16
> cinder_catalog_info = volumev3:cinderv3:internalURL
> enable_v2_api = True
> transport_url = rabbit://
> glance:9e7f205ef620983a542b4f915420e50d33f774 at 172.29.238.84:5671,
> glance:9e7f205ef620983a542b4f915420e50d33f774 at 172.29.238.174:5671,
> glance:9e7f205ef620983a542b4f915420e50d33f774 at 172.29.239.33:5671//glance?ssl=1&ssl_version=TLSv1_2&ssl_ca_file=
> scrub_time = 43200
> image_cache_dir = /var/lib/glance/cache/
> image_cache_stall_time = 86400
> image_cache_max_size = 10737418240
> # defaults to true if RBD is used as default store
> show_image_direct_url = False
> show_multiple_locations = True
> enabled_backends = file:file,http:http,cinder:cinder
>
> [task]
> task_executor = taskflow
>
> [database]
> connection = mysql+pymysql://
> glance:e6dd6f2ca946c9e6f72bb864387a at 172.29.236.36/glance?charset=utf8&ssl_verify_cert=true
> max_overflow = 50
> max_pool_size = 5
> pool_timeout = 30
> connection_recycle_time = 600
>
> [keystone_authtoken]
> insecure = False
> auth_type = password
> auth_url = http://172.29.236.36:5000/v3
> www_authenticate_uri = http://172.29.236.36:5000
> project_domain_id = default
> user_domain_id = default
> project_name = service
> username = glance
> password = c96a36e76208ee26851c78670d34dcaff1c870
> region_name = RegionOne
> service_token_roles_required = False
> service_token_roles = service
> service_type = image
> memcached_servers = 172.29.239.168:11211,172.29.239.242:11211,
> 172.29.236.80:11211
> token_cache_time = 300
> # if your memcached server is shared, use these settings to avoid cache
> poisoning
> memcache_security_strategy = ENCRYPT
> memcache_secret_key = 0b65b4b99155a6430e923fc9c24d9674
>
> [oslo_policy]
> policy_file = policy.yaml
> policy_default_rule = default
> policy_dirs = policy.d
>
> [oslo_messaging_notifications]
> topics = notifications
> driver = messagingv2
> transport_url = rabbit://
> glance:9e7f205ef620983a542b4f915420e50d33f774 at 172.29.238.84:5671,
> glance:9e7f205ef620983a542b4f915420e50d33f774 at 172.29.238.174:5671,
> glance:9e7f205ef620983a542b4f915420e50d33f774 at 172.29.239.33:5671//glance?ssl=1&ssl_version=TLSv1_2&ssl_ca_file=
>
> [paste_deploy]
> flavor = keystone+cachemanagement
>
> [glance_store]
> default_backend = file
>
> [file]
> filesystem_store_datadir = /var/lib/glance/images/
>
> [profiler]
> enabled = False
>
> [oslo_middleware]
> enable_proxy_headers_parsing = True
>
> [cors]
> allow_headers =
> origin,content-md5,x-image-meta-checksum,x-storage-token,accept-encoding,x-auth-token,x-identity-status,x-roles,x-service-catalog,x-user-id,x-tenant-id,x-openstack-request-id
> allow_methods = GET,POST,PUT,PATCH,DELETE
> allowed_origin = https://osa-portal.cs.binghamton.edu
>
> I'd be glad to run glance-api in the foreground with debug (in the
> container, or course), but it's not obvious from the .service file how to
> do that.  I'd be glad to read source code, but a pointer or two would be
> handy.  If NFS (or Hao's GPFS) just aren't supported anymore please tell me.
>
> Thanks.
>
> -Dave
>
> --
> Dave Hall
> Binghamton University
> kdhall at binghamton.edu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220804/c290c389/attachment-0001.htm>