PSSH – Parallel SSH

Many times we use SSH to login to the remote machines, Copy files, perform administrative tasks. But in all the cases we do these operations machine by machine i.e. we log to one machine perform the operations and then log out to go to the next machine. If we want to increase the productivity with SSH, we can use Tools that allow commands to be executed on multiple remote machines simultaneously. Parallel SSH is one such tool which allows running commands on multiple servers at the same time.

This article will give you a brief introduction about the Parallel SSH tool.

1. Install Parallel SSH

Parallel SSH tool is based on Python. PSSH is supported on Python 2.4 or more.In order to install use the PIP tool provided by Python. Install the parallel tool using

Pip install pssh

[puppet@root$:~/.pip]$  pip install pssh

Collecting pssh

  Downloading pssh-2.3.1.tar.gz

Building wheels for collected packages: pssh

  Running setup.py bdist_wheel for pssh ... done

Stored in directory: /root/.cache/pip/wheels/b6/98/92/eab367fee5ded0129e0d91feb3207e76fcb282ccc87507a6d0

Successfully built pssh

Installing collected packages: pssh

Successfully installed pssh-2.3.1

A version of pssh with 2.3.1 version is installed.

Note – pip is a tool by Python to install python packages. In order to install pip use the command “yum install python-pip”. In most linux version python will be already installed and hence use the “pip install pssh” command to install the pssh tool.

2. PSSH tools

Pssh includes parallel versions of OpenSSH and there are many other related tools like

• pssh – is a program for running ssh in parallel on a multiple remote hosts.
• pscp – is a program for copying files in parallel to a number of hosts.
• prsync – is a program for efficiently copying files to multiple hosts in parallel.
• pnuke – kills processes on multiple remote hosts in parallel.
• pslurp – copies files from multiple remote hosts to a central host in parallel.

3. Starting PSSH

In order to see the help for the pssh, we can use

[puppet@root$:~/.pip]$  pssh --help

Usage: pssh [OPTIONS] command [...]

Options:

  --version             show program's version number and exit

  --help                show this help message and exit

  -h HOST_FILE, --hosts=HOST_FILE

                        hosts file (each line "[user@]host[:port]")

  -H HOST_STRING, --host=HOST_STRING

                        additional host entries ("[user@]host[:port]")

  -l USER, --user=USER  username (OPTIONAL)

  -p PAR, --par=PAR     max number of parallel threads (OPTIONAL)

  -o OUTDIR, --outdir=OUTDIR

                        output directory for stdout files (OPTIONAL)

  -e ERRDIR, --errdir=ERRDIR

                        output directory for stderr files (OPTIONAL)

  -t TIMEOUT, --timeout=TIMEOUT

                        timeout (secs) (0 = no timeout) per host (OPTIONAL)

  -O OPTION, --option=OPTION

                        SSH option (OPTIONAL)

  -v, --verbose         turn on warning and diagnostic messages (OPTIONAL)

  -A, --askpass         Ask for a password (OPTIONAL)

  -x ARGS, --extra-args=ARGS

                        Extra command-line arguments, with processing for

                        spaces, quotes, and backslashes

  -X ARG, --extra-arg=ARG

                        Extra command-line argument

  -i, --inline          inline aggregated output and error for each server

  --inline-stdout       inline standard output for each server

  -I, --send-input      read from standard input and send as input to ssh

  -P, --print           print output as we get it

In order to use pssh tool,we need to pass the host file as argument and this is done by using “-h <host File>” or “—hosts <host File>”

We can also pass the user name using “-l <user Name>” or “—user <user Name>”

Standard Error and Standard message on each host executions can be seen by passing the “-i or -–inlineoption” and we can use the “-o <path to Dir>” to save the standard Output.

If we want the pssh to ask for the password we can pass “-A” Option.

Since we will be doing a SSH to the remote machine we need to specify a Timeout to check how long a command takes. It defaults to 60 seconds. This means that if your command fails to complete within 60 seconds on a host, pssh will consider it an error and report it. By default, pssh uses at most 32 ssh processes in parallel to ssh to the various nodes. By default, it also uses a timeout of one minute to ssh to a node and obtain a result.

4. Using PSSH

Create a file containing the hosts that we need to ssh and run the commands. I created the file “pssh-hosts” and added the remote machines that I want to ssh

[puppet@root$:/work]$ cat  pssh-hosts 

vagrant@devm.foohost.vm

vagrant@devs.foohost.vm

During this point, we need to make sure both the above machines have the user vagrant available and both machines have the ssh keys configured for the root machine that we are running pssh commands. We can add the arguments which will ask for the password while executing the pssh command. Passing of the user vagrant is not mandatory as we can pass the user while running the command too.

Once the file is created lets run a sample command as,

[puppet@root$:/work]$  pssh -h pssh-hosts -l vagrant echo "hello World"

[1] 08:09:14 [SUCCESS] devs.foohost.vm

[2] 08:09:14 [SUCCESS] devm.foohost.vm

[puppet@root$:/work]$  pssh -h pssh-hosts -l vagrant "touch /tmp/hello"

[1] 08:10:47 [SUCCESS] devm.foohost.vm

[2] 08:10:47 [SUCCESS] devs.foohost.vm

If we want to see the output, we can use “-I” argument as,

[puppet@root$:/work]$  pssh -h pssh-hosts -l vagrant -i "df -hT"

[1] 08:11:54 [SUCCESS] devm.foohost.vm

Filesystem              Type      Size  Used Avail Use% Mounted on

/dev/mapper/centos-root xfs        50G  1.8G   49G   4% /

devtmpfs                devtmpfs  235M     0  235M   0% /dev

/dev/mapper/centos-home xfs        29G   49M   29G   1% /home

/dev/sda1               xfs       497M  148M  350M  30% /boot

vagrant                 vboxsf     49G   14G   36G  28% /vagrant

[2] 08:11:54 [SUCCESS] devs.foohost.vm

Filesystem              Type      Size  Used Avail Use% Mounted on

/dev/mapper/centos-root xfs        50G  1.1G   49G   3% /

devtmpfs                devtmpfs  235M     0  235M   0% /dev

/dev/mapper/centos-home xfs        29G   33M   29G   1% /home

vagrant                 vboxsf     49G   14G   36G  28% /vagrant

These are some of the basic usages of the Pssh command. As we already discussed we do have other tool available with the pssh command.

PSCP – This is much similar to the SCP command available in Linux but the only difference is that this will do the scp on multiple machines at same time. we can use the command as,

Basic Usage - pscp -h ip remote file on the local file directory

[puppet@root$:/work]$  pscp -h pssh-hosts -l root /work/roles.zip /tmp

[1] 01:13:22 [SUCCESS] devm.foohost.vm

[2] 01:13:23 [SUCCESS] devs.foohost.vm

Pnuke - The pnuke command is useful when you want to kill a bunch of processes on a set of machines. For example, suppose you've got a bunch of java processes running on three nodes that you'd like to nuke. Here you would do the following:

Basic Usage - plurp - h ip file - L local directory remote file local file name

[puppet@root$:/work]$ pnuke -h pssh-hosts -l vagrant java

Warning: do not enter your password if anyone else has superuser

privileges or access to your account.

Password: 

[1] 01:31:06 [SUCCESS] devm.foohost.vm

[2] 01:31:07 [SUCCESS] devm.foohost.vm

Pslurp – The Pslurp command is usefull when we want to copy files in parallel from multiple remote hosts to the central host

Basic Usage - plurp - h ip file - L local directory remote file local file name

- L specify the directory to store files locally

[puppet@root$:/work]$  pslurp -h pssh-hosts -L /work /tmp/master.txt /tmp

[1] 01:24:03 [SUCCESS] root@devm.foohost.vm

[2] 01:24:03 [SUCCESS] root@devs.foohost.vm

The above Command will grab a file or directory (/tmp/master.txt) from the remote machine defined in the hosts file to the local /tmp location. The –r option which is a recursive will make a new directory with the name of the remote host and inside the directory a copy of the files will be stored.

Prsync – This command is much similar to the rsync command in linux which will sync the local locations to the remote Location

Basic Use - pslurp -h hosts -L ~/tmp /tmp/example.txt example.txt

The above will download /tmp/example.txt as example.txt, and will store the file in ~/tmp/[host]/, where [host] is the corresponding hostname to that found in your hosts file. 

[puppet@root$:/work]$  prsync -r -h pssh-hosts -l vagrant ./roles /tmp

[1] 09:31:15 [SUCCESS] devs.foohost.vm

[2] 09:31:16 [SUCCESS] devm.foohost.vm

Note – Make Sure the Rysnc command is available on the remote machines if this needs to work.

More to Come. Happy learning J