notify-when-up

notify-when-up is now deprecated; please use notify-when-up2 instead unless you encounter a specific problem in version 2 and need something to fall back on.

notify-when-up is a simple script that periodically checks for an event, and then notifies you the event has occured. Originally for checking when a port becomes active on a machine, it's since been extended to do many different sorts of one-off event monitoring in a convenient to use way.

Notification methods so far include:

A log message in your home directory (if you have write access to your home directory)
A message on the current tty
An e-mail message (by default to username@current.machine.com, but this can be overridden with -e or setting $NWUEMAIL to an e-mail address)
A little pygtk window

Installation:

Copy notify-when-up to somewhere on your $PATH - often ~/bin/notify-when-up is good.
Make the notify-when-up script executable, EG:
Install pdialog.py somewhere on your python path, or in ~/lib/pdialog.py
If you don't already have python installed, go get it. It's well worth your while. Nearly all Linuxes come with python these days, and the developer install of Solaris 10 includes it as well (IMO it should come with all versions of Solaris 10).

This software is owned by The university of California, Irvine - not any version of the GPL. GPL is a fine series of licenses, but the owners of the software need it to be distributed under these terms.

Here are some examples of using notify-when-up:

Simple, directly supported examples:

Some compound commands using notify-when-up, AKA idiomatic usages:

Notify when dcs.nac's ssh port is listening	notify-when-up -n dcs.nac.uci.edu 22
Notify when /bin/true gives a true result	notify-when-up -t /bin/true
Notify when /bin/false gives a false result	notify-when-up -f /bin/false
Notify when pid 12345 is no longer running	notify-when-up -p 12345
Search "ps" output for "process_name_substring". If only one, monitor that process. If more than one, ask the user which process to monitor, and monitor the selected process	notify-when-up -s process_name_substring
Monitor pid 54321, but only for a maximum of 300 seconds (5 minutes, aka "$(expr 60 \* 5)")	notify-when-up -p 54321 -m 300
Spawn a sleep command to monitor. Notify when it is done	notify-when-up -c 'sleep 120'
A basic countdown timer, ticking off by the second	notify-when-up -c 'sleep 600 # check mailroom' -i 1

Future directions
1. It'd be really nice to have something that will restart your notify-when-up process if a machine is rebooted while monitoring, or at least something that will tell you your notify-when-up was interrupted. Doing this really well is probably going to require a central server of some sort though, but it wouldn't be difficult to add a series of at jobs or a cron job that checks on whether jobs were interrupted, assuming logging is working.
2. It'd be nice to have some form of fallback logging pathname, instead of using ~/notify-when-up.log or zilch.
3. It may someday become useful to have notify-when-up trim logs
Back to Dan's tech tidbits

Notify when Solaris Volume Manger is done initializing a RAID volume	notify-when-up -f 'metastat \| grep "[I]nitialization in progress"'
Notify when Solaris Volume manager is done replacing a bad disk	notify-when-up -f 'metastat \| grep "[R]esync in progress"'
Notify when network I/O to/from atlantic.ess has ceased for a while	notify-when-up -l 20000 'maxtime 10 tcpdump -i en1 host atlantic.ess.uci.edu 2> /dev/null \| wc -l'
An example on Linux of waiting for a CD to finishing being ripped, and then poweroff the machine doing the ripping. We're basically waiting until sound-juicer isn't very active. Note: I have a suspicion this may really confuse sound-juicer, but this still stands as an example of waiting for a process to stop consuming lots of CPU time according to top	notify-when-up -l 7 'top -n 10 \| grep -c sound-juicer' -w; poweroff
A less troublesome example of notifying when a process is no longer CPU hungry: notifying when evolution quiets down	notify-when-up -l 7 'top -n 10 \| grep -c evolution'
Another top-based example: notify when a large file transfer via ssh is complete. Note that this is apparently better than monitoring an sftp-server process.	notify-when-up -l 7 "top -n 10 \| grep -c '23715.*sshd'"
Another example of waiting for the traffic to a machine to slow down (I've used this to let me know when a machine is done autoinstalling (jumpstarting) from our autoinstall machine, this time using tethereal instead of tcpdump. This example will gripe if the network activity doesn't cease in less than 4 hours (AKA "$(expr 60 \* 60 \* 4)"), and will only check for network activity every 180 seconds (3 minutes, AKA "$(expr 3 \* 60)", and e-mail notification will be sent to a specific e-mail address, not the default address.	notify-when-up -l 75 \ 'maxtime 1200 \ /dcs/packages/ethereal/bin/tethereal -c 105 host bingy.nac \| \ wc -l' \ -m $(expr 60 \* 60 \* 4) \ -i 180 \ -e strombrg@dcs.nac.uci.edu
Notify when bingy.nac becomes unpingable - nice if you're autoinstalling a system, and you're not sure if it'll make it through the install without crashing, for example	notify-when-up -f 'maxtime 30 cping bingy.nac.uci.edu 10 10 \| grep !'
Show all activity on port 6969 with an absolute timestamp (date+time), keeping no more than 10000 lines, notifying you when (if) the job completes, and notifying you if the job runs for over 3 days without completing normally.	notify-when-up -c 'tethereal port 6969 -t ad \| \ head -10000 > /var/tmp/sniff.out' -m $(expr 60 \* 60 \* 24 \* 3)
Notify when an A record becomes visible in the DNS for host bingy.nac	notify-when-up -t "dig bingy.nac.uci.edu \| grep 'bingy.nac.A.128'"
Notify when a specific console on the DCS console server goes idle for over 3 minutes (180 = $(expr 3 \* 60)). Nice for boot/install processes that require lots of "make a choice, wait a while, repeat" iterations. Complain if the console doesn't go idle within two hours ($(expr 60 \* 60)). Note the RHEL "pidof" command has different semantics from my pidof program :(. Also notice that this is an example of seeing what commands notify-when-up is running via "set -x" (Bourne-shell and POSIX-shell specific, not likely to work in *csh).	notify-when-up -l 3 "set -x; maxtime 180 strace -f -p $(~/bin/pidof -s 'console esmft1') 2>&1 \| \ head -10 \| \ wc -l" -m $(expr 60 \* 60 \* 2)
Make sure a filesystem doesn't drop below a given number of free blocks (assumes a long format GNU df - IE a filesystem that takes up two lines, not the usual one). Very useful when doing a large filetransfer from a source of unknown size - or even when transferring a filesystem of known size, but you know the df on the source system won't correspond perfectly with the df increase on the target system. Note that the grep -vi is a little bit hokey - someday I'll do this in a way that won't mess things up for filesystems with "used" in their mountpoint.	notify-when-up -l 10345656 "set -x; df -k . \| grep -vi used \| awk ' { print \$4 }'" -i 20
Make sure a directory doesn't grow larger than what will fit on a smallish CD-R or CD-RW. Note that this is an example of doing arithmetic with a POSIX shell-specific syntax - most of the other examples on this page 'til now have used expr instead.	notify-when-up -g $[640*1024] 'du -sk . \| awk "{ print \$1 }" '
Notify when ssh starts giving a banner. We're connecting to the ssh port for up to 10 seconds, and finishing when any single character is found in the output. If you want to be more specific, replace the "grep ." with the pattern you want to match.	notify-when-up -t 'maxtime 10 ~oacstats/bin/telnet-stdout esmft1.ess.uci.edu 22 \| \ grep .' -m $[606010]
Send a page when a Sun StorEdge 3511 RAID controller is done initializing.	notify-when-up -f './sccli /dev/es/ses0 show logical-drives \| \ grep -i initing' -w && echo "ESMF 3511 init done `date`" \| \ mailx -s username pager@pager.host.uci.edu
Power down the machine when there are no tar or mt processes running for a while	notify-when-up -l 1 'for i in `seq 10`; \ do ps -ef \| \ egrep "[t]ar\|[m]t"; \ sleep 1; \ done \| \ egrep -c "[t]ar\|[m]t"' -w && \ poweroff