Project

General

Profile

Support #231

Managing Multiple Upgrades on Multiple Servers Using a Simple Shell Script

Added by Daniel Curtis almost 7 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Shell Scripts
Target version:
-
Start date:
11/18/2013
Due date:
% Done:

100%

Estimated time:
1.00 h
Spent time:

Description

Through the years I've had to manage a wide-ranging number of different servers. At one job, I started with only a few and expanded to around ten, while at another job, I've managed hundreds. In both cases, I've found that you just can't accomplish everything you need to do efficiently when you log in to machines one at a time. Over the years, I've discovered a couple tools and techniques that certainly make it easier. Now granted, even these techniques can scale only so far. If you have a very large environment, you probably will be best served with some sort of centralized management tool like Puppet, cfengine or other tools that you can buy from vendors. Even so, for those of you who have a small-to-medium environment at work (or at home), here are some tricks to help you manage those machines better.

Copy public SSH key

I have found that using SSH keys makes running scripts a lot easier. This can be done by copying the contents of /home/user/.ssh/id_rsa.pub from the primary server to the remote servers /etc/ssh/authorized_keys. Or a simpler method would be using ssh-copy-id:

ssh-copy-id user@machine1

SSH Loops

A common need you have when there are more than a few servers in your environment is to run the same command on more than one machine. When I first had this problem, I came up with a pretty simple shell script:

HOSTS="machine1 machine2 machine3 machine4"; 
for i in $HOSTS; do ssh $i uname -a; done;

This one-liner iterates through each machine I've listed in the HOSTS environment variable and runs uname -a. You can, of course, replace uname -a with any command-line command that you would want to run on the hosts. This is useful for getting initial connection problems out of the way. For instance, one need I had was to keep all of my Debian servers up to date.
Ultimately, I found I executed this one-liner so much, it warranted its own script, which I called update-all.sh:

#!/bin/sh
hosts="machine1 machine2 machine3 machine4" 

# Run the command on each remote host
for i in $hosts;
do
  echo $i;
  ssh $i sudo apt-get update && apt-get upgrade -y;
done;

# Also run the command on the local machine
sudo apt-get update && apt-get upgrade -y

Note: Be careful with the -y flag, I only use it to automate the process. Removing the -y flag will cause upgrades to be verified by the administrator.

Now, this system worked for me at the time, but it has plenty of room for improvement. For one, I potentially could set up a set of environment variables for different host groups. Then, instead of defining HOSTS each time I ran the one-liner, I could reference one of those groups.

Resources

http://www.linuxjournal.com/magazine/hack-and-manage-multiple-servers-efficiently

History

#1 Updated by Daniel Curtis almost 7 years ago

I needed to tweak the script in order to get the servers to upgrade properly. I also added separator lines that allows easier identification of which server is working at the moment.

#!/bin/bash
# Script to update and upgrade listed servers

hosts="machine1 machine2 machine3 machine4";

sep="==============================================================================";

# Run update and upgrade command on (1)localhost then (2)each remote host
apt-get update
apt-get -y upgrade

for i in $hosts;
do
  echo -e "\n"$sep
  echo -e "--> "$i"\n";
  ssh root@$i "apt-get update" 
  ssh root@$i "apt-get -y upgrade" 
done;

Note: I needed to use the -e flag with echo in order to allow the \n special character to work correctly.

#2 Updated by Daniel Curtis over 5 years ago

  • Project changed from 21 to GNU/Linux Administration
  • Category set to Shell Scripts

Also available in: Atom PDF