Open Stack

I proposed some patches through heat templates and puppet-cinder to remove
lock files older than 1 week and avoid file system growing.

This is a solution based on a cron job, to fix that on stable branches, in
a second time I'll help the fasteners project to fix the root cause by
reviewing and testing the proposed patch (lock based on file offset). In
next versions I hope we will use a patched fasteners and so we could drop
the cron based solution.

Please can you give /reviews/feedbacks:
- https://review.opendev.org/688413
- https://review.opendev.org/688414
- https://review.opendev.org/688415

Thanks

Le lun. 30 sept. 2019 à 03:35, Rikimaru Honjo <honjo.rikimaru at ntt-tx.co.jp>
a écrit :

> On 2019/09/28 1:44, Ben Nemec wrote:
> >
> >
> > On 9/23/19 11:42 PM, Rikimaru Honjo wrote:
> >> Hi Eric,
> >>
> >> On 2019/09/20 23:10, Eric Harney wrote:
> >>> On 9/20/19 1:52 AM, Rikimaru Honjo wrote:
> >>>> Hi,
> >>>>
> >>>> I'm using Queens cinder with the following setting.
> >>>>
> >>>> ---------------------------------
> >>>> [coordination]
> >>>> backend_url = file://$state_path
> >>>> ---------------------------------
> >>>>
> >>>> As a result, the files like the following were remained under the
> state path after some operations.[1]
> >>>>
> >>>> cinder-63dacb3d-bd4d-42bb-88fe-6e4180164765-delete_volume
> >>>> cinder-32c426af-82b4-41de-b637-7d76fed69e83-delete_snapshot
> >>>>
> >>>> In my understanding, these are lock-files created for synchronization
> by tooz.
> >>>> But, these lock-files were not deleted after finishing operations.
> >>>> Is this behaviour correct?
> >>>>
> >>>> [1]
> >>>> e.g. Delete volume, Delete snapshot
> >>>
> >>> This is a known bug that's described here:
> >>>
> >>> https://github.com/harlowja/fasteners/issues/26
> >>>
> >>> (The fasteners library is used by tooz, which is used by Cinder for
> managing these lock files.)
> >>>
> >>> There's an old Cinder bug for it here:
> >>> https://bugs.launchpad.net/cinder/+bug/1432387
> >>>
> >>> but that's marked as "Won't Fix" because Cinder needs it to be fixed
> in the underlying libraries.
> >> Thank you for your explanation.
> >> I understood the state.
> >>
> >> But, I have one more question.
> >> Can I think this bug doesn't affect synchronization?
> >
> > It does not. In fact, it's important to not remove lock files while a
> service is running or you can end up with synchronization issues.
> >
> > To clean up the leftover lock files, we generally recommend clearing the
> lock_path for each service on reboot before the services have started.
>
> Thank you for your information.
> I think that I understood this issue completely.
>
> Best Regards,
>
>
> >>
> >> Best regards,
> >>
> >>> Thanks,
> >>> Eric
> >>>
> >>
> >
>
> --
> _/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
> Rikimaru Honjo
> E-mail:honjo.rikimaru at ntt-tx.co.jp
>
>
>

-- 
Hervé Beraud
Senior Software Engineer
Red Hat - Openstack Oslo
irc: hberaud
-----BEGIN PGP SIGNATURE-----

wsFcBAABCAAQBQJb4AwCCRAHwXRBNkGNegAALSkQAHrotwCiL3VMwDR0vcja10Q+
Kf31yCutl5bAlS7tOKpPQ9XN4oC0ZSThyNNFVrg8ail0SczHXsC4rOrsPblgGRN+
RQLoCm2eO1AkB0ubCYLaq0XqSaO+Uk81QxAPkyPCEGT6SRxXr2lhADK0T86kBnMP
F8RvGolu3EFjlqCVgeOZaR51PqwUlEhZXZuuNKrWZXg/oRiY4811GmnvzmUhgK5G
5+f8mUg74hfjDbR2VhjTeaLKp0PhskjOIKY3vqHXofLuaqFDD+WrAy/NgDGvN22g
glGfj472T3xyHnUzM8ILgAGSghfzZF5Skj2qEeci9cB6K3Hm3osj+PbvfsXE/7Kw
m/xtm+FjnaywZEv54uCmVIzQsRIm1qJscu20Qw6Q0UiPpDFqD7O6tWSRKdX11UTZ
hwVQTMh9AKQDBEh2W9nnFi9kzSSNu4OQ1dRMcYHWfd9BEkccezxHwUM4Xyov5Fe0
qnbfzTB1tYkjU78loMWFaLa00ftSxP/DtQ//iYVyfVNfcCwfDszXLOqlkvGmY1/Y
F1ON0ONekDZkGJsDoS6QdiUSn8RZ2mHArGEWMV00EV5DCIbCXRvywXV43ckx8Z+3
B8qUJhBqJ8RS2F+vTs3DTaXqcktgJ4UkhYC2c1gImcPRyGrK9VY0sCT+1iA+wp/O
v6rDpkeNksZ9fFSyoY2o
=ECSj
-----END PGP SIGNATURE-----
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20191015/92965e1b/attachment.html>