We were running a bash script containing a while loop to process a large amount of data. The basics of the script looked like the following:
#!/bin/bash
#####################################################################
# Read input filename
#####################################################################
read filename
#####################################################################
# Understand input file and variable assignment
#####################################################################
{
while IFS=';' reat -r IP_ADDRESS VAR1 VAR2 VAR3 VAR4 rest; do
#####################################################################
# Do the magic
#####################################################################
ssh -q -o ConnectTimeout=30 $IP_ADDRESS exit 2>&1
if [ $? != "0" ]; then
echo "FAILED TO CONNECT TO IP ADDRESS VIA SSH"
echo "FAILED TO CONNECT TO IP ADDRESS VIA SSH" > output/HOSTNAME-"$IP_ADDRESS"-FAILURE
else
echo "SUCCESSFULLY CONNECTED TO IP ADDRESS VIA SSH"
ssh -q -o ConnectTimeout=30 $IP_ADDRESS hostname > output/HOSTNAME-"$IP_ADDRESS"-SUCCESS
fi
done < "$filename"
}
When running the script whenever a line was reached that successfully connected via SSH and the else part of the script was run, e.g:
ssh -q -o ConnectTimeout=30 $IP_ADDRESS hostname > output/HOSTNAME-"$IP_ADDRESS"-SUCCESS
The while loop would terminate and fail to process further entries inside of the input file. After further investigation runs ssh commands and by default ssh reads from stdin which is your input file. As a result, you only see the first line processed, because the command consumes the rest of the file and your while loop terminates.
This happens not just for ssh, but for any command that reads stdin, including mplayer, ffmpeg, HandBrakeCLI, httpie, brew install, and more. To prevent this, we had to pass the -n option to the ssh command to make it read from /dev/null instead of stdin. Other commands have similar flags, or you can universally use < /dev/null.
The end script looks as follows with the changes above:
#####################################################################
# Read input filename
#####################################################################
read filename
#####################################################################
# Understand input file and variable assignment
#####################################################################
{
while IFS=';' reat -r IP_ADDRESS VAR1 VAR2 VAR3 VAR4 rest; do
#####################################################################
# Do the magic
#####################################################################
ssh -q -n -o ConnectTimeout=30 $IP_ADDRESS exit 2>&1
if [ $? != "0" ]; then
echo "FAILED TO CONNECT TO IP ADDRESS VIA SSH"
echo "FAILED TO CONNECT TO IP ADDRESS VIA SSH" > output/HOSTNAME-"$IP_ADDRESS"-FAILURE
else
echo "SUCCESSFULLY CONNECTED TO IP ADDRESS VIA SSH"
ssh -q -n -o ConnectTimeout=30 $IP_ADDRESS hostname > output/HOSTNAME-"$IP_ADDRESS"-SUCCESS
fi
done < "$filename"