Managing Multiple Upgrades on Multiple Servers Using a Simple Shell Script
Through the years I've had to manage a wide-ranging number of different servers. At one job, I started with only a few and expanded to around ten, while at another job, I've managed hundreds. In both cases, I've found that you just can't accomplish everything you need to do efficiently when you log in to machines one at a time. Over the years, I've discovered a couple tools and techniques that certainly make it easier. Now granted, even these techniques can scale only so far. If you have a very large environment, you probably will be best served with some sort of centralized management tool like Puppet, cfengine or other tools that you can buy from vendors. Even so, for those of you who have a small-to-medium environment at work (or at home), here are some tricks to help you manage those machines better.
Copy public SSH key¶
I have found that using SSH keys makes running scripts a lot easier. This can be done by copying the contents of
/home/user/.ssh/id_rsa.pub from the primary server to the remote servers
/etc/ssh/authorized_keys. Or a simpler method would be using ssh-copy-id:
A common need you have when there are more than a few servers in your environment is to run the same command on more than one machine. When I first had this problem, I came up with a pretty simple shell script:
HOSTS="machine1 machine2 machine3 machine4"; for i in $HOSTS; do ssh $i uname -a; done;
This one-liner iterates through each machine I've listed in the HOSTS environment variable and runs
uname -a. You can, of course, replace
uname -a with any command-line command that you would want to run on the hosts. This is useful for getting initial connection problems out of the way. For instance, one need I had was to keep all of my Debian servers up to date.
Ultimately, I found I executed this one-liner so much, it warranted its own script, which I called update-all.sh:
#!/bin/sh hosts="machine1 machine2 machine3 machine4" # Run the command on each remote host for i in $hosts; do echo $i; ssh $i sudo apt-get update && apt-get upgrade -y; done; # Also run the command on the local machine sudo apt-get update && apt-get upgrade -y
Note: Be careful with the -y flag, I only use it to automate the process. Removing the -y flag will cause upgrades to be verified by the administrator.
Now, this system worked for me at the time, but it has plenty of room for improvement. For one, I potentially could set up a set of environment variables for different host groups. Then, instead of defining HOSTS each time I ran the one-liner, I could reference one of those groups.
Updated by Daniel Curtis almost 8 years ago
I needed to tweak the script in order to get the servers to upgrade properly. I also added separator lines that allows easier identification of which server is working at the moment.
#!/bin/bash # Script to update and upgrade listed servers hosts="machine1 machine2 machine3 machine4"; sep="=============================================================================="; # Run update and upgrade command on (1)localhost then (2)each remote host apt-get update apt-get -y upgrade for i in $hosts; do echo -e "\n"$sep echo -e "--> "$i"\n"; ssh root@$i "apt-get update" ssh root@$i "apt-get -y upgrade" done;
Note: I needed to use the
echoin order to allow the
\nspecial character to work correctly.