Line-based text processing in bash
Recently, I've been writing a script to parse the output of ping
, and set the round-trip time as the title of terminal emulator, so that I can easily know if my host is up, and how the connection latency is.
Here is my script ttping.sh
.
Basically, this script calls the original ping
, passes through all the arguments, parses the output of ping
with grep
, and sets the terminal title.
Everything worked fine before I piped the processed output of grep
.
With script like this:
ping localhost | grep -Po '(?<=time.).+ms'
I got the output I wanted:
0.028 ms
0.036 ms
0.035 ms
0.035 ms
But when I piped the output of grep
to echo
in order to set terminal title with the following code:
ping localhost | grep -Po '(?<=time.).+ms' \
| while read; do
echo -ne "\e]0;$REPLY\a"
done
It's just not working any more, the console seemed stuck.
So I added set -x
in front of my script to start debugging this. And got something like this:
+ read
+ grep -Po '(?<=time.).+ms'
+ ping localhost
I had known that ping
and grep
paired well, and the echo
command in while read
loop was not being executed, so it seemed that read
was not getting any input from grep
. There must be something wrong or unexpected to me with grep
!
Referring to man grep
, I finally found this:
--line-buffered
Use line buffering on output. This can cause a performance
penalty.
So my guess was that when grep
is outputting to console or stdout, it's line-buffered, and when it is outputting to a pipe or a file, it's buffereing the whole output until its buffer is full, unless specified with --line-buffered
.
Adding --line-buffered
as an argument to grep
, things worked again.
Every time a ping response is received, ping
outputs a line regarding this. This line is then sent to grep
command. grep
processes input line by line, and when specified --line-buffered
, it outputs line by line as well. Now the output of grep
goes to read
, it also processes on a line basis, and records input in $REPLY
variable if not explicitly specified. Finally, echo
uses this variable and shell escape sequences to set terminal title. Then a new loop begins…
I've then added the ability to keep output of ping
to console with tee
while setting the terminal title at the same time, and the ability to detect Host unreachable and Request timed out.
Here is a screenshot and the final version of ttping
, or view it on GitHub:
# ping and set round-trip time as terminal title
ttping () {
# Pass on arguments to ping
ping "$@" \
| tee >(
# Regex for round-trip time
# Linux style: 'time=42 ms', 'time=0 ms'
# Windows style: 'time=42ms', 'time<1ms'
local time='(?<=time.).+ms'
# Regex for unreachable
# Linux style: 'Dest Unreachable'
# Windows style: 'Destination host unreachable.'
local unreachable='nreachable'
# Regex for timed out
# Linux style: (No Message)
# Windows style: 'Request timed out.'
local timedout='timed out'
# Parse output of ping
grep --line-buffered -Po "$time|$unreachable|$timedout" \
| while read; do
# Set terminal title.
# Redirect escape sequences to stderr,
# so that terminal title is set in time,
# even if output of this function is piped.
case "$REPLY" in
*ms)
>&2 echo -ne "\e]0;$REPLY\a"
;;
$unreachable|$timedout)
>&2 echo -ne "\e]0;!! Host Down\a"
;;
esac
done
)
}
Note:
In addition to grep
, many other text processing commands also buffer output.
Here is a list of some options to commands which make them unbuffered or less buffered:
grep
:--line-buffered
sed
:-u, --unbuffered