source: mod_gnutls/test/README @ 37beb92

Unit Tests for Apache's mod_gnutls
==================================

Authors: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
         Thomas Klute <thomas2.klute@uni-dortmund.de>

There are a lot of ways that a TLS-capable web server can go wrong.  I
want to at least test for some basic/common configurations.


Running the tests
=================

From the top level of the source, or from test/ (where this README is),
just run:

  make check

from test/. You can also run specific test cases by passing their
script names to make in the TESTS variable:

  TESTS="test-03_cachetimeout_in_vhost.bash" make -e check

This should be handy when you're just trying to experiment with a new
test and don't want to wait for the full test suite to run.

The default configuration assumes that a loopback device with IPv4 and
IPv6 support is available (TEST_IP="[::1] 127.0.0.1") and that
TEST_HOST="localhost" resolves to at least one of these addresses. If
this does not apply to your system, you can pass different values to
./configure, e.g. to use IPv4 only:

  TEST_HOST="localhost" TEST_IP="127.0.0.1" ./configure

If tests fail due to expired certificates or PGP signatures, run

  make mostlyclean

to delete them and create fresh ones on the next test run. You could
also use "make clean", but in that case the keys will be deleted as
well and have to be recreated, too, which takes more time.


Adding a Test
=============

Please add more tests!

The simplest way to add a test is (from test/):

  ./newtest

This will prompt you for a simple name for the test and then copy a
starting set of files from tests/00_basic, and create a script which
you can add to TESTS in Makefile.am when your test is ready for
inclusion in the test suite.


Implementation
==============

Each test consists of a script in test/ and a directory in
test/tests/, which the test suite uses to spin up an isolated Apache
instance or two (for proxy tests) and try to connect to it with
gnutls-cli and make a simple HTTP 1.1 or 1.0 request.

Test directories usually contain the following files:

 * apache.conf -- Apache configuration to be used

 * gnutls-cli.args -- the arguments to pass to gnutls-cli

 * input -- the full HTTP request (including the final blank line)

 * backend.conf [optional] -- Apache configuration for the proxy
   backend server, if any

 * output [optional] -- the lines of this file will be checked against
   the same number of lines at the end of the output produced by the
   gnutls-cli process. "Date" and "Server" headers are filtered from
   the response because they are expected to change between runs
   (date) or builds (server version).

 * fail.server [optional] -- if this file exists, it means we expect
   the web server to fail to even start due to some serious
   configuration problem.

 * fail.client [optional] -- if this file exists, it means we expect
   the client to fail to fetch its file.  If you already have
   fail.server, do not also specify this; we know that a failed server
   should result in a failed file retrieval.

The "runtests" script is used to start one Apache instance and send a
request based on the files described above. Note that some tests take
additional steps, e.g. starting another server to act as proxy
backend, and at least one does not use "runtests" at all.

By default (if "unshare" is available and has the permissions required
to create network and user namespaces), each test case is run inside
its own network namespace. This avoids address and port conflicts with
other tests as well has the host system.

When writing your own tests, make sure to call netns_reexec (defined
in common.bash) if you need to start any network services outside of
runtests (which will create the namespace if it doesn't exist
already). However, some architectures might not support namespaces, so
traditional locking (using flock) and serial execution are still
supported.


Robustness and Tuning
=====================

Here are some things that you might want to tune about the tests based
on your expected setup (along with the variables that can be passed to
"make check" to adjust them):

 * They need a functioning loopback device.

 * They expect (by default) to have port 9932 [TEST_PORT] available
   and open for connections on the addresses listed in TEST_IP.

 * Depending on the compile time configuration of the Apache binary
   installed on your system you may need to load additional Apache
   modules. The recommended way to do this is to drop a configuration
   file into the test/apache-conf/ directory. Patches to detect such
   situations and automatically configure the tests accordingly are
   welcome.

 * If a machine is particularly slow or under heavy load, it's
   possible that these tests will fail for timing
   reasons. [TEST_QUERY_TIMEOUT (timeout for the HTTPS request in
   seconds)]

The first two of these issues are avoided when the tests are isolated
using network namespaces, which is the default (see "Implementation"
above). The ./configure script tries to detect if namespaces can be
used (some Linux distributions disable them for unprivileged
users). If this detection returns a false positive or you do not want
to use namespace isolation for some other reason, you can run
configure with the --disable-test-namespaces option.

In some situations you may want to see the exact environment as
configured by make, e.g. if you want to manually run an Apache
instance with Valgrind using the same configuration as a test
case. Use "make show-test-env" to dump AM_TESTS_ENVIRONMENT to stdout.

If you are building on an exotic architecture which does not support
flock (or timeouts using flock -w), ./configure should detect that and
disable locking, or you can disable it manually by passing
"--disable-flock" to ./configure. This will force serial execution of
tests, including environment setup.