sgmon -- Simple stargrave's monitoring system
Intended for my own personal use, as a replacement for Monit.
All of that is just an ordinary plain POSIX shell scripts.

Create some directory hierarchy for convenience:

    $ mkdir -p probes/example.com/{ping4,ping6,http}
    $ mkdir -p probes/jails/foobar

Make probes from them:

    $ echo '"$SGMONDIR"/helper/http http://example.com/ Some.Expected.Title'
        > probes/example.com/ping4/run <<EOF
    # only 3 failures will generated notification email
    $ echo 3 > probes/example.com/http/max-attempts

    $ echo '"$SGMONDIR"/helper/ping 4 example.com' > probes/example.com/ping4/run

    $ echo '"$SGMONDIR"/helper/ping 6 example.com' > probes/example.com/ping6/run
    $ echo 5 > probes/example.com/ping4/period # every 5 seconds

    $ cat > probes/jails/foobar/run <<EOF
    #!/bin/sh -e
    jls -j foobar-jail-name
    EOF
    $ chmod +x probes/jails/foobar/run

Set default configuration values:

    $ cat > rc <<EOF
    NOTIFY_EMAIL=user+sgmon@example.com
    PERIOD_DEFAULT=60
    ATTEMPTS_DEFAULT=1
    EOF

You can run probe individually for testing:

    $ cd probes/example.com/ping6
    $ /path/to/sgmon/run.sh
    [...]

Or start all of them in background:

    $ cd probes
    $ /path/to/sgmon/start.sh

Send signal to that script and it will exit when all probes will die.
It is advisable to run it under daemontools'es supervise utility for
convenience.

run.sh creates temporary directory for each probe and links it to
$probe/state. run-script expects that it is run from the $probe
directory, there is state/, where it can store its temporary files.
$SGMONDIR and $SGMONDIR are also always set too.

You can remove all state directories with clear.sh.

Generate HTML page with information about state of all services and
their output:

    $ cd probes
    $ /path/to/sgmon/status.sh > /path/to/index.html

If some probe is failed, then email message with its stdout/stderr will
be sent. If it succeeds after the fail, then another message is sent.
Each startup and shutdown are also notified by email.