1. Short Overview

Generally the SIP Forum Basic UA Test Suite is separated into two parts. The first part is the SIP Forum Testing Framework which contains all the logic which is required to run the test, parse the incoming messages and e.g. create replies. The second part is the a subdirectory which contains the Basic UA Tests itself.

After starting the SIP Forum Testing Framework it will first load the desired tests from the subdirectory. When the tests are successfully loaded the tests will be executed one after another, until all tests are finished. At the end when all tests are done a result overview over all tests will be printed out.

2. Requirements

As the testing framework and the test cases are written in Python (Python Homepage) a working Python installation on the system is required. The software is developed under Linux with Python 2.3. It is also know to work under Windows XP. Python below 2.3 was never tested but may work too. Please report any experiences with other operating systems or Python versions.

As the test cases 702 and 703 test for working DNS SRV support they require the installation of the dnspython library (DNS Python Homepage) to check the DNS SRV setup.

_sip._udp SRV 0 0 5060 testhost.example.com.

SRV 1 0 5061 testhost.example.com.

3. Testing Framework Invocation

Invoke the SIP Forum Testing Framework by executing the SFTF script directly (Note: this requires a python interpreter installed at /usr/bin/python) or by calling your python interpreter with the SFTF script as parameter. If you called the SFTF script without a parameter it will print out a help screen like this:

SIP Forum Test Framework v0.9.1 by Nils Ohlmeier

Copyright (C) 2004 Sip Forum

SFTF.py [-acdChiIrRsSV] [-D directory] [-t testcasename] [testcasename]

-a                run all tests

-c                run UAC tests only

-C                dont use colors on output

-d                dont exit on keyboard interrupt

-D                add the dir to file search path

-h                print this help screen

-i                run non-interactive tests only

-I                run interactive tests only

-r                run tests without REGISTER only

-R                run tests with REGISTER only

-s                run UAS tests only

-S                turn off verbose test summary

-t testcasename   load and run testcasename

(can be given multiple times)

-V                print the version information

You can run a single test by calling e.g.

SFTF.py -t case201

This will try to load the test “case201” from the test directory (see ) and run it if loading was successfull. You can give as many -t parameters as you want (limited by the command line length of the operating system). E.g.

SFTF.py -t case201 -t case202

will first load the test case 201 and 202 and then run both test cases.

The easiest way to prevent giving all test cases you want to run by hand is to use combinations of the parameters c, i, I, r, R and s. For example calling the following command:

SFTF.py -i -r

will load and run all test cases which do not require any interaction by a person at the user agent (Note: this will be a subset of the UAC tests, UAC tests generally require interaction by the user) and omit all test cases which require to send a REGISTER to the testing framework.

Generally it is not recommended to run all test (with the -a parameter), because in this case all test will be loaded without any order and you will have to start calls, send REGISTER's and accept calls without any order. So it is recommended to run e.g. first all test which require to send a REGISTER, then all non-interactive tests, and finally all interactive tests (so you can use the hopefully existant redial button at the UA more effectively ;-).

4. Configuration

The most IMPORTANT part is the configuration of the user agent which should be tested. Please configure the host and port where the testing framework runs as outgoing proxy and registrar. Do NOT put a proxy between the user agent and the testing framework. This is necessary because several tests of the Basic UA Test Suite require to fake a proxy in front of the user agent. And a proxy between the UA and the testing framework can change the result of the test cases very much.

The other important part to configure the SIP Forum testing framework is the file Config.py. Generally all parameters in the config file should be commented so we will give here only a brief overview about the most important variables.

5. Bug Reports

If the testing framework ever fails during execution of a test it will print out a traceback an a message to report this bug. In this case please send the debug log file (debug.log by default) to develop at ohlmeier.org. Preferably set the log level (LOG_LEVEL) in Config.py to 5 and especially please turn on network logging by setting LOG_NETWORKS_PACKETS to True (and set LOG_NETWORKS_PACKETS_LEVEL to the value of LOG_LEVEL or lower) and try to reproduce the failure. Then send the debug.log file to develop at ohlmeier.org. Tanks in advance for your support.

6. Downloading the SFTF

Download the SIP Forum Test Framework from the SVN repository here at the SIP Foundry. The Repository URL is

https://scm.sipfoundry.org/rep/sftf/