[OpenStack-Infra] Zuul get stuck when fetching patch from gerrit, how to deal with this?

liuxinguo liuxinguo at huawei.com
Wed Mar 4 13:29:13 UTC 2015


The network of our CI is poor so sometime the zuul will get stuck when fetching patch from gerrit. When running “ps -ef|grep zuul” I find it get stuck like this:

root at 3rd-ci-master:/var/log/zuul# ps -ef|grep zuul
root       698  9074  0 16:48 pts/0    00:00:00 grep --color=auto zuul
zuul     31730     1  2 03:01 ?        00:16:37 /usr/bin/python /usr/local/bin/zuul-server
zuul     31739     1  0 03:01 ?        00:00:00 /usr/bin/python /usr/local/bin/zuul-merger
zuul     31748 31730  0 03:01 ?        00:00:00 /usr/bin/python /usr/local/bin/zuul-server
zuul     32008 31739  0 05:08 ?        00:00:00 git remote update origin
zuul     32009 32008  0 05:08 ?        00:00:00 git fetch --multiple origin
zuul     32010 32009  0 05:08 ?        00:00:00 git fetch --append origin
zuul     32011 32010  0 05:08 ?        00:00:00 /bin/bash /var/lib/zuul/git/.ssh_wrapper -p 29418 huawei-volume-ci at review.openstack.org<mailto:huawei-volume-ci at review.openstack.org> git-upload-pack '/openstack/cinder'
zuul     32012 32011  0 05:08 ?        00:00:00 ssh -i /var/lib/zuul/ssh/id_rsa -p 29418 huawei-volume-ci at review.openstack.org<mailto:huawei-volume-ci at review.openstack.org> git-upload-pack '/openstack/cinder'‍

I have to add a Crontab to restart the zuul and zuul-merge service and kill all these stucked progesses or do this manually.

·         Is there any better method to avoid the “git remote update origin” and “ssh_wrapper” getting stuck when the network is poor.

Thanks for any input!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20150304/22c71666/attachment.html>


More information about the OpenStack-Infra mailing list