input.txt

##
#
# Comments are supported and are prefixed with a leading '#'.
#
# NOTE: If an input file is executable it will be executed
# and the output will be parsed, instead of the literal contents.
#
# Although we've not discussed how tests are written yet, consider
# this example a simple demonstration:
#
#  --
#  #!/usr/bin/m4
#  # Define a macro for a tinc-protocol test
#  define(`tinc', `$1 must run tcp with port 655 with banner "^0 \S+ 17$"')
#
#  tinc(`foo.example.com')
#  tinc(`bar.example.com')
#
#  # When processed the output will be:
#  foo.example.com must run tcp with port 655 with banner "^0 \S+ 17$"
#  bar.example.com must run tcp with port 655 with banner "^0 \S+ 17$"
#  --
#
#
##

####
#
#
# The general form of our test definitions is:
#
#      TARGET must run PROTOCOL [test-specific options]
#
# Where:
#
# `TARGET` is either the hostname or an URI of the target to be tested.
#
# `PROTOCOL` is one of the protocol-handlers implemented in the application.
#
# Test-specific options are always written like so:
#
#      with $OPTION_NAME $OPTION_VALUE
#
# `OPTION_VALUE` may optionally be quoted with single or double-quotes,
# this is necessary if the option-value contains whitespace.
#
####


#
# A simple example of a test would be to ensure that a host is running
# an FTP daemon.
#
# The basic way to write this would be:
#
#   ftp.example.com must run ftp
#
# This validates that the remote host is running an FTP-server, but it doesn't
# making a complete test.  To do that you'd want to actually retrieve a file
# via FTP.
#
# To specify the file to retrieve we replace the target with an URI pointing
# to the file we wish to retrieve:
#
#   ftp://ftp.cpan.org/pub/gnu/=README must run ftp
#
# Downloading a file requires a login, so by default we'll try an anonymous
# one.  If you need to specify real credentials you can do so by adding
# the appropriate username & password:
#
#   ftp://ftp.example.com/path/to/README must run ftp with username 'user@host.com' with password 'secret'
#
# Of course the URI could also be used to specify the login details:
#
#   ftp://user@example.com:secret@ftp.cpan.org/pub/gnu/=README must run ftp
#
# Finally you can ensure that you retrieved sane contents by testing the
# resulting-file contains a specific string:
#
#   ftp://ftp.cpan.org/pub/gnu/=README must run ftp with content 'GNU'
#


#
# More people probably run HTTP/HTTPS servers than FTP-servers, so
# the next test-type to document is that one.  Unlike the previous
# case the target of the test is an an URL, rather than a hostname.
#
# The most basic HTTP test would be:
#
#      http://example.com/ must run http
#
# A HTTP status-code of 200 is regarded as a pass, and anything else
# as a failure.  You can choose to regard any other return-code as a
# success by setting your preferred result:
#
#       with status 302
#
# Or you might regard any response as valid, which could be specified by
# writing:
#
#       with status any
#
# In short `with status any` tests:
#
# * You could resolve the target's hostname.
#   * Any returned IPv4 and IPv6 address will be tested in turn.
# * You could make a HTTP-request.
# * You received some kind of response.
#
# If you wanted a more thorough test you might wish to look for
# some specific text in body of the response, which is possible
# via the `content` argument.
#
# For example I might wish to ensure my website has my name in it:
#
#       https://steve.fi/ must run http with content 'Steve Kemp'
#
# You can of course combine the status & content options:
#
#       https://steve.fi/ must run http with status any with content 'Kemp'
#
# Looking for a literal text-match in the body is usually sufficient
# for ensuring that your site is available, however if it is not you
# can also test that a specific regular-expression matches the content
# of the response.
#
# Rather than using `content` we specify our pattern via `pattern`:
#
#       https://steve.fi/ must run http with pattern 'Steve\s+Kemp'
#
# If you need to make a HTTP POST request, rather than a GET, you can
# do that by specifying the data to POST like so:
#
#        https://steve.fi/Security/XSS/Tutorial/filter.cgi must run http with data "text=test%20me" with content "test me"
#
# The CGI script in that example just echos arguments back, so it is
# a simple tset that the POSTed data was received.
#
# The HTTP-protocol tester sets a custom user-agent to allow filtering
# on the server-side - which might be required to remove noise if tests
# are repeated often.
#
# You can see this in action via:
#
#          http://httpbin.org/user-agent must run http with content 'overseer/'
#

#
# My website is at https://steve.fi/, there are redirections
# in place for HTTP and for www-prefixed access.
#
# Test that the HTTP versions redirect to the secure version.
#
# We look for redirections via:
#
#   1.  The status-code.
#   2.  The URL of the target in the body (which Apache does automatically).
#
# This works because our HTTP-probe does NOT follow HTTP-Redirection
# requests, as doing so would limit the kind of tests we could write.
#
http://steve.fi/ must run http with status 301 with content 'https://steve.fi'
http://www.steve.fi/ must run http with status 302 with content 'https://steve.fi'

#
# I prefer to avoid www.-names:
#
https://www.steve.fi/ must run http with status 302 with content 'https://steve.fi'

#
# So the final test is that we have decent content on the single "real" site.
#
https://steve.fi/ must run http with status 200 with content 'Steve Kemp'



#
# If your webserver uses HTTP basic-authentication you can submit the
# appropriate username/password as you would expect:
#
#     https://jigsaw.w3.org/HTTP/Digest/ must run http with username 'guest' with password 'guest' with content "Your browser made it"
#

#
# Macros are shortcuts for repeating tests against multiple hosts.
#
# Here we define the macro "REDIS" to have two IPs:
#
REDIS are 127.0.0.1, ::1

#
# Macro-names are always written in upper-case, and it is a fatal error
# to set the value of an existing macro.  Which means this is invalid:
#
# HOSTS are 1.2.3.4, 1.2.3.5,...
# HOSTS are 10.0.0.1, 10.0.0.2
#

#
# Although the examples above used IP-addresses using hostnames is fine
# too:
#
# HOSTS are host1.example.com, host2.example.com
#
# We'll see that later on when we run a bunch of DNS-tests against a
# pair of nameservers.
#


#
# Now we use the macro we defined, meaning that this single test will
# be applied against both hosts used in the definition.
#
REDIS must run redis


#
# The redis probe, used above, tested that Redis responded on port 6379.
# Rather than using the redis-specific protocol-test you could have instead
# used the generic TCP-based test:
#
#   REDIS must run tcp on 6379
#
# Using the redis-test is better, because it lets you specify an optional
# password, and really connects.  But as an example using the TCP-connection
# test allows you to test protocols that don't have specific handlers defined
# for them in overseer - please report this as a bug!
#


#
# Of course nobody could reach my website if there were no DNS entries
# present for it.  So we should test they exist too!
#
# The DNS lookup test requires you to specify several things, beyond the
# DNS-server to query (which is the target of the test and thus implicit):
#
#  * The name to lookup.
#  * The type of record to lookup.
#  * The expected result
#
# For consistency the DNS test uses the general mechanism already
# demonstrated to allow you to set those via text like this:
#
#    with lookup "example.com"
#    with type "A"
#    with result "127.0.0.1"
#

#
# First of all we define a pair of nameservers, using our macro-facility:
#
NAMESERVERS are rachel.ns.cloudflare.com, clark.ns.cloudflare.com

#
# Now we run some basic tests
#
NAMESERVERS must run dns with lookup steve.fi with type A with result '176.9.183.100'
NAMESERVERS must run dns with lookup steve.fi with type AAAA with result '2a01:4f8:151:6083::100'
NAMESERVERS must run dns with lookup www.steve.fi with type A with result '176.9.183.100'
NAMESERVERS must run dns with lookup www.steve.fi with type AAAA with result '2a01:4f8:151:6083::100'


#
# You can confirm that a record shouldn't exist by looking for an empty
# result (i.e. "", or '').
#
# The following host, alert.steve.fi, is deliberately setup as IPv4 only,
# so finding an AAAA record in DNS would indicate a mistake:
#
NAMESERVERS must run dns with lookup alert.steve.fi with type A with result  '176.9.183.100'
NAMESERVERS must run dns with lookup alert.steve.fi with type AAAA with result ''


#
# Now we should do more testing!
#
# Run our OpenSSH probe against localhost, on the non-standard port 2222.
#
localhost must run ssh with port 2222

#
# If you didn't want to use a non-standard port you'd just write:
#
#    localhost must run ssh
#

#
# Redis should run on localhost.
#
localhost must run redis

#
# If a password is required to connect to redis then set it like so:
#
#    localhost must run redis with password 'secrit!'
#
# If a non-standard port is used:
#
#    localhost must run redis with port 1234
#
# Of course these can be combined:
#
#    localhost must run redis with port 1234 with password 'p4ssw0rd'
#

#
# Now we can test that we get a response from a remote SMTP server
#
mail.steve.org.uk must run smtp
mail.steve.org.uk must run smtp with port 587



#
# Similarly you might wish to test SSH against a whole bunch of related
# hosts, so you might try this:
#
#   SSH_HOSTS are host1.example.com, host2.example.com, host3.example.com
#   SSH_HOSTS must run ssh
#
# NOTE:
#
#    All of the protocol-tests allow this expansion __EXCEPT__ for
#    the http-test, because the target of a HTTP-test is an URL, not a host.
#



#
# IMAPS is a good thing.
#
# In this context "insecure" means "don't validate the SSL certificate",
# in my case the SSL certificate is for "mail.steve.org.uk", but here you'll
# notice I'm testing a different name (which points to the same host).
#
# So here I'm disabling the strict validation here:
#
ssh.steve.org.uk must run imaps with tls insecure

#
# Without the disabling we'd see:
#
#   Test failed: x509: certificate is valid for
#     mail.steve.org.uk, webmail.steve.org.uk, not ssh.steve.org.uk
#

#
# But if I connect to the correct hostname it is fine to leave TLS alone:
#
mail.steve.org.uk must run imaps


##
## Further Examples
##


#
# To see the complete list of available protocol-tests, along with sample
# usage and supported arguments please run:
#
#     ./overseer examples
#
# This will show you test-types we've not covered here, including finger,
# telnet, NTTP, posgres, and MySQL.
#