<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;">
<div>
<div>We included in all of our puppet manifest a tools directory that has a "restartworld.sh" [1]. We specify classes of servers [App, network, compute] which run classes of services and a specific service that we wish to restart or if you want to restart
everything you do "world". Then we have an ansible playbook [2] that will run restartworld.sh using the class of server automatically. So if you want to restart everything you just tell it to run against all hosts in an environment with a service of world
and it will do everything for you.</div>
<div>
<div><br>
</div>
<div>[1] - <a href="https://github.com/godaddy/openstack-puppet/blob/master/tools/restartworld.sh">https://github.com/godaddy/openstack-puppet/blob/master/tools/restartworld.sh</a></div>
<div>[2] - <a href="https://github.com/godaddy/openstack-ansible/blob/master/playbooks/restartworld.yaml">https://github.com/godaddy/openstack-ansible/blob/master/playbooks/restartworld.yaml</a></div>
<div><br>
</div>
<div>The following links are currently a bit old - we revamped them to work with systemd and our cells setup. I will work on getting those updated onto our public repo.</div>
<div>____________________________________________</div>
<div> </div>
<div>Kris Lindgren</div>
<div>Senior Linux Systems Engineer</div>
<div>GoDaddy, LLC.</div>
</div>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>David Medberry <<a href="mailto:openstack@medberry.net">openstack@medberry.net</a>><br>
<span style="font-weight:bold">Date: </span>Tuesday, March 17, 2015 at 9:39 AM<br>
<span style="font-weight:bold">To: </span>"<a href="mailto:openstack-operators@lists.openstack.org">openstack-operators@lists.openstack.org</a>" <<a href="mailto:openstack-operators@lists.openstack.org">openstack-operators@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>[Openstack-operators] Restart Services when Rabbit Croaks<br>
</div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">Hi all,
<div><br>
</div>
<div>In PHL Mid-Cycle Operators meetings, several folks in the rabbitmq session indicated they basically restart all openstack services when they have a rabbit issue. Can you please share your process/scripts with this list.</div>
<div><br>
</div>
<div>At the present time, with an Ubuntu Trusty base OS, I'm doing:</div>
<div><br>
</div>
<div>
<p style="margin:0px;padding:0px;color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;line-height:20px;background-color:rgb(245,245,245)">
sudo restart libvirt-bin ; sudo restart nova-compute ; sudo restart neutron-plugin-openvswitch-agent ; #ALL COMPUTE NODES<br>
sudo restart neutron-server ; sudo restart neutron-l3-agent; sudo restart neutron-dhcp-agent ; sudo restart neutron-metadata-agent ; sudo restart neutron-plugin-openvswitch-agent ;sudo restart nova-scheduler ; sudo restart nova-api ; sudo restart nova-cert;
sudo restart nova-conductor; sudo restart nova-consoleauth; sudo restart nova-novncproxy ; sudo restart nova-objectstore ; #ALL CONTROL NODES</p>
<p style="margin:10px 0px 0px;padding:0px;color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;line-height:20px;background-color:rgb(245,245,245)">
and sometimes</p>
<p style="margin:10px 0px 0px;padding:0px;color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;line-height:20px;background-color:rgb(245,245,245)">
sudo restart ceilometer-collect ; sudo restart ceilometer-agent-notification ; sudo restart ceilometer-agent-central ; sudo restart ceilometer-api #ALL CONTROL NODES</p>
<p style="margin:10px 0px 0px;padding:0px;color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;line-height:20px;background-color:rgb(245,245,245)">
for SVC in cinder-api cinder-backup cinder-scheduler cinder-volume; sudo restart $SVC; #All CONTROL NODES</p>
<p style="margin:10px 0px 0px;padding:0px;color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;line-height:20px;background-color:rgb(245,245,245)">
<br>
</p>
<p style="margin:10px 0px 0px;padding:0px;color:rgb(51,51,51);font-family:Arial,sans-serif;font-size:14px;line-height:20px;background-color:rgb(245,245,245)">
(or the equivalent via ansible)</p>
</div>
</div>
</div>
</div>
</span>
</body>
</html>