[openstack-dev] [nova][NFS] Inexplicable utime permission denied when launching instance
Neil Jerram
neil at tigera.io
Thu Oct 25 10:05:03 UTC 2018
I'm still seeing the same problem after disabling AppArmor, so I think
it must be some other root problem.
On Wed, Oct 24, 2018 at 2:41 PM Neil Jerram <neil at tigera.io> wrote:
>
> Thanks so much for these hints, Erlon. I will look closer at AppArmor.
>
> Neil
>
> On Wed, Oct 24, 2018 at 1:41 PM Erlon Cruz <sombrafam at gmail.com> wrote:
> >
> > PS. Don't forget that if you change or disable AppArmor you will have to reboot the host so the kernel gets reloaded.
> >
> > Em qua, 24 de out de 2018 às 09:40, Erlon Cruz <sombrafam at gmail.com> escreveu:
> >>
> >> I think that there's a change that AppArmor is blocking the access. Have you checked the dmesg messages related with apparmor?
> >>
> >> Em sex, 19 de out de 2018 às 09:38, Neil Jerram <neil at tigera.io> escreveu:
> >>>
> >>> Wracking my brains over this one, would appreciate any pointers...
> >>>
> >>> Setup: Small test deployment with just 3 compute nodes, Queens on Ubuntu Bionic. The first compute node is an NFS server for /var/lib/nova/instances, and the other compute nodes mount that as NFS clients.
> >>>
> >>> Problem: Sometimes, when launching an instance which is scheduled to one of the client nodes, nova-compute (in imagebackend.py) gets Permission Denied (errno 13) when calling utime to touch the timestamp on the instance file.
> >>>
> >>> Through various bits of debugging and hackery, I've established that:
> >>>
> >>> - it looks like the problem never occurs when this is the call that bootstraps the privsep setup; but it does occur quite frequently on later calls
> >>>
> >>> - when the problem occurs, retrying doesn't help (5 times, with 0.5s in between)
> >>>
> >>> - the instance file does exist, and is owned by root with read/write permission for root
> >>>
> >>> - the privsep helper is running as root
> >>>
> >>> - the privsep helper receives and executes the request - so it's not a problem with communication between nova-compute and the helper
> >>>
> >>> - root is uid 0 on both NFS server and client
> >>>
> >>> - NFS setup does not have the root_squash option
> >>>
> >>> - there is some AppArmor setup, on both client and server, and I haven't yet worked out whether that might be relevant.
> >>>
> >>> Any ideas?
> >>>
> >>> Many thanks,
> >>> Neil
> >>>
> >>> __________________________________________________________________________
> >>> OpenStack Development Mailing List (not for usage questions)
> >>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
More information about the OpenStack-dev
mailing list