ProFTPD: Testsuite


The proftpd source code is accompanied by a testsuite, for testing various features and functionality. At present, the testsuite is divided into two main parts: the API-level tests, and the integration tests. The API tests are written in C, and exercise the various APIs that proftpd provides to module developers. The integration tests are currently written in Perl, and seek to check a compiled proftpd's run-time behavior.

In order to run the testsuite, you must configure your proftpd build so that the testsuite is prepared. You do this by using the --enable-tests configure option, along with your other build options, e.g.:

  $ ./configure --enable-tests ...

API Tests
The API tests use the libcheck library; see:

  http://check.sourceforge.net/
This library must be installed on your machine in order to run the API tests. When the --enable-tests configure option is used, proftpd's configure script will check for libcheck, and will fail unless the library is present. If no libcheck is found and you've used --enable-tests, you will see:
    checking for tcase_create in -lcheck... no
    configure: error: libcheck support, required for tests, not present -- aborting
Note that, if necessary, the --with-includes and --with-libraries configure options can be used to specify the locations of the libcheck headers and libraries.

To run the testsuite, use the make check target:

  $ ./configure --enable-tests ...
  $ make
    ...
  $ make check
    ...
  $ ./api-tests
  Running suite(s): pool
   array
   str
   sets
   timers
   table
   var
   event
   env
   version
   feat
   netaddr
   netacl
   class
   regexp
   expr
   scoreboard
   modules
  100%: Checks: 140, Failures: 0, Errors: 0
The make check will also go on to run the integration tests, if the API tests all pass.

If one of the API tests fails, you will see an error message like:

  99%: Checks: 140, Failures: 1, Errors: 0
  api/env.c:53:F:base:env_get_test:0: Failed to handle null arguments
  -------------------------------------------------
   FAILED 1 test

   Please send email to:

     proftp-devel@lists.sourceforge.net

   containing the `api-tests.log' file (in the tests/ directory)
   and the output from running `proftpd -V'
  -------------------------------------------------
Please do follow those instructions; you will be helping in proftpd development!

Expected Failures
Unfortunately, due to variations in system configurations, it's possible that some of the API tests will fail, without it necessarily being an indication of a regression or problem.

Most notably, some of the most brittle tests make assumptions about the host's /etc/hosts file, particularly the list of names associated with the address 127.0.0.1. If the /etc/hosts file has a name other than "localhost" or "localhost.localdomain" immediately after the 127.0.0.1 address, e.g.:

  127.0.0.1	some.other.name localhost
Then the API tests may fail with errors like this:
  98%: Checks: 152, Failures: 2, Errors: 0
  api/netaddr.c:344:F:base:netaddr_get_dnsstr_test:0: Expected 'localhost or localhost.localdomain', got 'some.other.name'
  api/netacl.c:535:F:base:netacl_match_test:0: Failed to positively match ACL to addr: No such file or directory
If this is the case, it is an expected failure; please do not report the issue as a bug. (Unfortunately, I have not been able to think of a good solution/workaround for these situations other than ignoring the test failure, but I would rather not do that if possible.)

Integration Tests
The current integration tests are written in Perl, and rely on the Test-Unit Perl package. Specifically, you must use Test-Unit-0.14, not Test-Unit-0.25. Not sure which version of Test-Unit you have, if at all? Run the following command:

  $ perl -MTest::Unit -e 'print $Test::Unit::VERSION, "\n";'
  0.14

To run the integration tests manually, use the tests.pl script:

  $ cd tests/
  $ perl tests.pl
  t/logins.....................ok                                              
  t/commands/user..............ok                                              
  t/commands/pass..............ok                                              
  t/commands/pwd...............ok                                              
  t/commands/cwd...............ok                                              
  t/commands/cdup..............ok                                              
  t/commands/syst..............ok                                              
  t/commands/type..............ok                                              
  t/commands/mkd...............ok                                              
  t/commands/rmd...............ok                                              
  t/commands/dele..............ok                                              
  t/commands/mdtm..............ok                                              
  t/commands/size..............ok                                              
  t/commands/mode..............ok                                              
  t/commands/stru..............ok                                              
  t/commands/allo..............ok                                              
  t/commands/noop..............ok                                              
  t/commands/feat..............ok                                              
  t/commands/help..............ok                                              
  t/commands/quit..............ok                                              
  t/commands/rnfr..............ok                                              
  t/commands/rnto..............ok                                              
  t/commands/rest..............ok                                              
  t/commands/pasv..............ok                                              
  t/commands/epsv..............ok                                              
  t/commands/port..............ok                                              
  t/commands/eprt..............ok                                              
  t/commands/nlst..............ok                                              
  t/commands/list..............ok                                              
  t/commands/retr..............ok                                              
  t/commands/stor..............ok                                              
  t/commands/appe..............ok                                              
  t/config/displayconnect......ok                                              
  t/config/displaylogin........ok                                              
  t/config/maxloginattempts....ok                                              
  t/config/serverident.........ok                                              
  All tests successful.
  Files=37, Tests=141, 160 wallclock secs ( 4.33 cusr +  4.74 csys =  9.07 CPU)

Some of the integration require root privileges in order to perform the test properly, e.g. for <Anonymous>/DefaultRoot logins, etc. If you do not run `make check' as root, those tests which require root privileges are automatically skipped. Thus for doing a full regression, run the integration tests as the root user.

Adding New Tests
The following information is for those who are interested in adding new tests, updating existing tests, or otherwise helping with the testsuite development.

The API tests driver, api-tests, honors the PR_TEST_SUITE environment variable. This is useful for running just the test cases associated with a particular suite, such as env or pool. Only one suite can be specified using the PR_TEST_SUITE environment variable:

  $ make check
  $ cd tests/
  $ PR_TEST_SUITE=pool ./api-tests
  Running suite(s): pool
  100%: Checks: 5, Failures: 0, Errors: 0
This way, when you are adding to an existing test case or adding a new test case to an existing suite, you need not run the entire API testsuite in order to run your changes.


© Copyright 2017 The ProFTPD Project
All Rights Reserved