[OpenStack-Infra] afs02 r/o volume mirrors - ongoing incident

Ian Wienand iwienand at redhat.com
Fri May 25 10:00:27 UTC 2018


On 05/24/2018 11:36 PM, Ian Wienand wrote:
> Thanks to the help of Jeffrey Altman [1], we have managed to get
> mirror.pypi starting to resync again.

And thanks to user error on my behalf, and identified by jeblair, in
the rush of all this I ran this under k5start on mirror-update,
instead of on one of the afs hosts with -localauth, so the ticket
timed out and the release failed.

---
root at mirror-update01:~# k5start -t -f /etc/afsadmin.keytab 
service/afsadmin -- vos release mirror.pypi
Kerberos initialization for service/afsadmin at OPENSTACK.ORG

Release failed: rxk: authentication expired
Could not end transaction on a ro volume: rxk: authentication expired
  Could not update VLDB entry for volume 536870931
Failed to end transaction on the release clone 536870932
Could not release lock on the VLDB entry for volume 536870931
rxk: authentication expired
Error in vos release command.
rxk: authentication expired
---

If it is any consolation, it's the type of mistake you only make once :)

I am now re-running the sync in a root screen on afs02 with -localauth
so it won't timeout.  Expect it to finish about 20 hours from this
mail :/

Thanks,

-i



More information about the OpenStack-Infra mailing list