loop

Note: loop has been deprecated in favor of the nicer looper.

This software is owned by The university of California, Irvine - not any version of the GPL. GPL is a fine series of licenses, but the owners of the software need it to be distributed under these terms.

Loop is a simple program that allows you to control the degree of parallelism used in running a bunch of programs.

You can download it here.

We've been using it for years with good results.

Usage is like:

seki-strombrg> loop
usage:
/dcslib/allsys/etc/loop -d save directory
/dcslib/allsys/etc/loop -n max concurrent proc
/dcslib/allsys/etc/loop -c command to execute
/dcslib/allsys/etc/loop -v (verbose)
stdin should consist of filename arguments
for each command to execute
Sun Jan 01 11:06:37

Example usage (from memory) might look like:

mkdir -p /tmp/savedir
rm /tmp/savedir/*
cat /tmp/hostname-list-one-host-per-line | sed 's/^$.*$$/\1 \1/' | loop -d /tmp/savedir -n 20 -c 'uname -n' -v
/tmp/savedir should then have one file per host, each containing the results from the command on each host. Then review the results with something like:
- vi /tmp/savedir/* # look at all the files
- vi $(ls /tmp/savedir | equivs2 -s | awk ' { print $1 } ') # only look at a single copy of any files that have the same content

Back to Dan's tech tidbits