QTest Namespace

Macro Documentation

QBENCHMARK

This macro is used to measure the performance of code within a test. The code to be benchmarked is contained within a code block following this macro.

For example:

 void TestBenchmark::simple()
 {
     QString str1 = QLatin1String("This is a test string");
     QString str2 = QLatin1String("This is a test string");
     QCOMPARE(str1.localeAwareCompare(str2), 0);
     QBENCHMARK {
         str1.localeAwareCompare(str2);
     }
 }

See also Creating a Benchmark and Writing a Benchmark.

QBENCHMARK_ONCE

The QBENCHMARK_ONCE macro is for measuring performance of a code block by running it once.

This macro is used to measure the performance of code within a test. The code to be benchmarked is contained within a code block following this macro.

Unlike QBENCHMARK, the contents of the contained code block is only run once. The elapsed time will be reported as "0" if it's to short to be measured by the selected backend. (Use)

This function was introduced in Qt 4.6.

See also Creating a Benchmark and Writing a Benchmark.

QCOMPARE(actual, expected)

The QCOMPARE() macro compares an actual value to an expected value using the equality operator. If actual and expected match, execution continues. If not, a failure is recorded in the test log and the test function returns without attempting any later checks.

Always respect QCOMPARE() parameter semantics. The first parameter passed to it should always be the actual value produced by the code-under-test, while the second parameter should always be the expected value. When the values don't match, QCOMPARE() prints them with the labels Actual and Expected. If the parameter order is swapped, debugging a failing test can be confusing and tests expecting zero may fail due to rounding errors.

When comparing floating-point types (float, double, and qfloat16), qFuzzyCompare() is used for finite values. If qFuzzyIsNull() is true for both values, they are also considered equal. Infinities match if they have the same sign, and any NaN as actual value matches with any NaN as expected value (even though NaN != NaN, even when they're identical).

QCOMPARE() tries to output the contents of the values if the comparison fails, so it is visible from the test log why the comparison failed.

Example:

 QCOMPARE(QString("hello").toUpper(), QString("HELLO"));

For your own classes, you can use QTest::toString() to format values for outputting into the test log.

Example:

 char *toString(const MyType &t)
 {
     char *repr = new char[t.reprSize()];
     t.writeRepr(repr);
     return repr;
 }

The return from toString() must be a new char []. That is, it shall be released with delete[] (rather than free() or plain delete) once the calling code is done with it.

See also QVERIFY(), QTRY_COMPARE(), QTest::toString(), and QEXPECT_FAIL().

QEXPECT_FAIL(dataIndex, comment, mode)

The QEXPECT_FAIL() macro marks the next QCOMPARE() or QVERIFY() as an expected failure. Instead of adding a failure to the test log, an expected failure will be reported.

If a QVERIFY() or QCOMPARE() is marked as an expected failure, but passes instead, an unexpected pass (XPASS) is written to the test log.

The parameter dataIndex describes for which entry in the test data the failure is expected. Pass an empty string ("") if the failure is expected for all entries or if no test data exists.

comment will be appended to the test log for the expected failure.

mode is a QTest::TestFailMode and sets whether the test should continue to execute or not.

Example 1:

 QEXPECT_FAIL("", "Will fix in the next release", Continue);
 QCOMPARE(i, 42);
 QCOMPARE(j, 43);

In the example above, an expected fail will be written into the test output if the variable i is not 42. If the variable i is 42, an unexpected pass is written instead. The QEXPECT_FAIL() has no influence on the second QCOMPARE() statement in the example.

Example 2:

 QEXPECT_FAIL("data27", "Oh my, this is soooo broken", Abort);
 QCOMPARE(i, 42);

The above testfunction will not continue executing for the test data entry data27.

See also QTest::TestFailMode, QVERIFY(), and QCOMPARE().

QFAIL(message)

This macro can be used to force a test failure. The test stops executing and the failure message is appended to the test log.

Example:

 if (sizeof(int) != 4)
     QFAIL("This test has not been ported to this platform yet.");

QFETCH(type, name)

The fetch macro creates a local variable named name with the type type on the stack. The name and type must match a column from the test's data table. This is asserted and the test will abort if the assertion fails.

Assuming a test has the following data:

 void TestQString::toInt_data()
 {
     QTest::addColumn<QString>("aString");
     QTest::addColumn<int>("expected");

     QTest::newRow("positive value") << "42" << 42;
     QTest::newRow("negative value") << "-42" << -42;
     QTest::newRow("zero") << "0" << 0;
 }

The test data has two elements, a QString called aString and an integer called expected. To fetch these values in the actual test:

 void TestQString::toInt()
 {
      QFETCH(QString, aString);
      QFETCH(int, expected);

      QCOMPARE(aString.toInt(), expected);
 }

aString and expected are variables on the stack that are initialized with the current test data.

QFETCH_GLOBAL(type, name)

This macro fetches a variable named name with the type type from a row in the global data table. The name and type must match a column in the global data table. This is asserted and the test will abort if the assertion fails.

Assuming a test has the following data:

 void TestQLocale::initTestCase_data()
 {
     QTest::addColumn<QLocale>("locale");
     QTest::newRow("C") << QLocale::c();
     QTest::newRow("UKish") << QLocale("en_GB");
     QTest::newRow("USAish") << QLocale(QLocale::English, QLocale::UnitedStates);
 }

 void TestQLocale::roundTripInt_data()
 {
     QTest::addColumn<int>("number");
     QTest::newRow("zero") << 0;
     QTest::newRow("one") << 1;
     QTest::newRow("two") << 2;
     QTest::newRow("ten") << 10;
 }

The test's own data is a single number per row. In this case, initTestCase_data() also supplies a locale per row. Therefore, this test will be run with every combination of locale from the latter and number from the former. Thus, with four rows in the global table and three in the local, the test function is run for 12 distinct test-cases (4 * 3 = 12).

 void TestQLocale::roundTripInt()
 {
     QFETCH_GLOBAL(QLocale, locale);
     QFETCH(int, number);
     bool ok;
     QCOMPARE(locale.toInt(locale.toString(number), &ok), number);
     QVERIFY(ok);
 }

The locale is read from the global data table using QFETCH_GLOBAL(), and the number is read from the local data table using QFETCH().

QFINDTESTDATA(filename)

Returns a QString for the testdata file referred to by filename, or an empty QString if the testdata file could not be found.

This macro allows the test to load data from an external file without hardcoding an absolute filename into the test, or using relative paths which may be error prone.

The returned path will be the first path from the following list which resolves to an existing file or directory:

• filename relative to QCoreApplication::applicationDirPath() (only if a QCoreApplication or QApplication object has been created).
• filename relative to the test's standard install directory (QLibraryInfo::TestsPath with the lowercased testcase name appended).
• filename relative to the directory containing the source file from which QFINDTESTDATA is invoked.

If the named file/directory does not exist at any of these locations, a warning is printed to the test log.

For example, in this code:

 bool tst_MyXmlParser::parse()
 {
     MyXmlParser parser;
     QString input = QFINDTESTDATA("testxml/simple1.xml");
     QVERIFY(parser.parse(input));
 }

The testdata file will be resolved as the first existing file from:

• /home/user/build/myxmlparser/tests/tst_myxmlparser/testxml/simple1.xml
• /usr/local/Qt-5.0.0/tests/tst_myxmlparser/testxml/simple1.xml
• /home/user/sources/myxmlparser/tests/tst_myxmlparser/testxml/simple1.xml

This allows the test to find its testdata regardless of whether the test has been installed, and regardless of whether the test's build tree is equal to the test's source tree.

This function was introduced in Qt 5.0.

QSKIP(description)

If called from a test function, the QSKIP() macro stops execution of the test without adding a failure to the test log. You can use it to skip tests that wouldn't make sense in the current configuration. For example, a test of font rendering may call QSKIP() if the needed fonts are not installed on the test system.

The text description is appended to the test log and should contain an explanation of why the test couldn't be executed.

If the test is data-driven, each call to QSKIP() in the test function will skip only the current row of test data, so an unconditional call to QSKIP() will produce one skip message in the test log for each row of test data.

If called from an _data function, the QSKIP() macro will stop execution of the _data function and will prevent execution of the associated test function. This entirely omits a data-driven test. To omit individual rows, make them conditional by using a simple if (condition) newRow(...) << ... in the _data function, instead of using QSKIP() in the test function.

If called from initTestCase_data(), the QSKIP() macro will skip all test and _data functions. If called from initTestCase() when there is no initTestCase_data(), or when it only sets up one row, QSKIP() will likewise skip the whole test. However, if initTestCase_data() contains more than one row, then initTestCase() is called (followed by each test and finally the wrap-up) once per row of it. Therefore, a call to QSKIP() in initTestCase() will merely skip all test functions for the current row of global data, set up by initTestCase_data().

Example:

 if (!QSqlDatabase::drivers().contains("SQLITE"))
     QSKIP("This test requires the SQLITE database driver");

Skipping Known Bugs

If a test exposes a known bug that will not be fixed immediately, use the QEXPECT_FAIL() macro to document the failure and reference the bug tracking identifier for the known issue. When the test is run, expected failures will be marked as XFAIL in the test output and will not be counted as failures when setting the test program's return code. If an expected failure does not occur, the XPASS (unexpected pass) will be reported in the test output and will be counted as a test failure.

For known bugs, QEXPECT_FAIL() is better than QSKIP() because a developer cannot fix the bug without an XPASS result reminding them that the test needs to be updated too. If QSKIP() is used, there is no reminder to revise or re-enable the test, without which subsequent regressions will not be reported.

See also QEXPECT_FAIL() and Select Appropriate Mechanisms to Exclude Tests.

QTEST(actual, testElement)

QTEST() is a convenience macro for QCOMPARE() that compares the value actual with the element testElement from the test's data. If there is no such element, the test asserts.

Apart from that, QTEST() behaves exactly as QCOMPARE().

Instead of writing:

 QFETCH(QString, myString);
 QCOMPARE(QString("hello").toUpper(), myString);

you can write:

 QTEST(QString("hello").toUpper(), "myString");

See also QCOMPARE().

QTEST_APPLESS_MAIN(TestClass)

Implements a main() function that executes all tests in TestClass.

Behaves like QTEST_MAIN(), but doesn't instantiate a QApplication object. Use this macro for really simple stand-alone non-GUI tests.

See also QTEST_MAIN().

QTEST_GUILESS_MAIN(TestClass)

Implements a main() function that instantiates a QCoreApplication object and the TestClass, and executes all tests in the order they were defined. Use this macro to build stand-alone executables.

Behaves like QTEST_MAIN(), but instantiates a QCoreApplication instead of the QApplication object. Use this macro if your test case doesn't need functionality offered by QApplication, but the event loop is still necessary.

This function was introduced in Qt 5.0.

See also QTEST_MAIN().

QTEST_MAIN(TestClass)

Implements a main() function that instantiates an application object and the TestClass, and executes all tests in the order they were defined. Use this macro to build stand-alone executables.

If QT_WIDGETS_LIB is defined, the application object will be a QApplication, if QT_GUI_LIB is defined, the application object will be a QGuiApplication, otherwise it will be a QCoreApplication. If qmake is used and the configuration includes QT += widgets, then QT_WIDGETS_LIB will be defined automatically. Similarly, if qmake is used and the configuration includes QT += gui, then QT_GUI_LIB will be defined automatically.

Example:

 QTEST_MAIN(TestQString)

See also QTEST_APPLESS_MAIN(), QTEST_GUILESS_MAIN(), QTest::qExec(), and QApplication::setNavigationMode().

QTRY_COMPARE(actual, expected)

Performs a comparison of the actual and expected values by invoking QTRY_COMPARE_WITH_TIMEOUT() with a timeout of five seconds.

This function was introduced in Qt 5.0.

See also QTRY_COMPARE_WITH_TIMEOUT(), QCOMPARE(), QVERIFY(), QTRY_VERIFY(), and QEXPECT_FAIL().

QTRY_COMPARE_WITH_TIMEOUT(actual, expected, timeout)

The QTRY_COMPARE_WITH_TIMEOUT() macro is similar to QCOMPARE(), but performs the comparison of the actual and expected values repeatedly, until either the two values are equal or the timeout (in milliseconds) is reached. Between each comparison, events will be processed. If the timeout is reached, a failure is recorded in the test log and the test won't be executed further.

This function was introduced in Qt 5.0.

See also QTRY_COMPARE(), QCOMPARE(), QVERIFY(), QTRY_VERIFY(), and QEXPECT_FAIL().

QTRY_VERIFY2(condition, message)

Checks the condition by invoking QTRY_VERIFY2_WITH_TIMEOUT() with a timeout of five seconds. If condition is then still false, message is output. The message is a plain C string.

Example:

 QTRY_VERIFY2_WITH_TIMEOUT(list.size() > 2, QByteArray::number(list.size()).constData());

This function was introduced in Qt 5.6.

See also QTRY_VERIFY2_WITH_TIMEOUT(), QTRY_VERIFY2(), QVERIFY(), QCOMPARE(), QTRY_COMPARE(), and QEXPECT_FAIL().

QTRY_VERIFY(condition)

Checks the condition by invoking QTRY_VERIFY_WITH_TIMEOUT() with a timeout of five seconds.

This function was introduced in Qt 5.0.

See also QTRY_VERIFY_WITH_TIMEOUT(), QTRY_VERIFY2(), QVERIFY(), QCOMPARE(), QTRY_COMPARE(), and QEXPECT_FAIL().

QTRY_VERIFY2_WITH_TIMEOUT(condition, message, timeout)

The QTRY_VERIFY2_WITH_TIMEOUT macro is similar to QTRY_VERIFY_WITH_TIMEOUT() except that it outputs a verbose message when condition is still false after the specified timeout (in milliseconds). The message is a plain C string.

Example:

 QTRY_VERIFY2_WITH_TIMEOUT(list.size() > 2, QByteArray::number(list.size()).constData(), 10000);

This function was introduced in Qt 5.6.

See also QTRY_VERIFY(), QTRY_VERIFY_WITH_TIMEOUT(), QVERIFY(), QCOMPARE(), QTRY_COMPARE(), and QEXPECT_FAIL().

QTRY_VERIFY_WITH_TIMEOUT(condition, timeout)

The QTRY_VERIFY_WITH_TIMEOUT() macro is similar to QVERIFY(), but checks the condition repeatedly, until either the condition becomes true or the timeout (in milliseconds) is reached. Between each evaluation, events will be processed. If the timeout is reached, a failure is recorded in the test log and the test won't be executed further.

This function was introduced in Qt 5.0.

See also QTRY_VERIFY(), QTRY_VERIFY2_WITH_TIMEOUT(), QVERIFY(), QCOMPARE(), QTRY_COMPARE(), and QEXPECT_FAIL().

QVERIFY2(condition, message)

The QVERIFY2() macro behaves exactly like QVERIFY(), except that it reports a message when condition is false. The message is a plain C string.

The message can also be obtained from a function call that produces a plain C string, such as qPrintable() applied to a QString, which may be built in any of its usual ways, including applying .args() to format some data.

Example:

 QVERIFY2(QFileInfo("file.txt").exists(), "file.txt does not exist.");

For example, if you have a file object and you are testing its open() function, you might write a test with a statement like:

 bool opened = file.open(QIODevice::WriteOnly);
 QVERIFY(opened);

If this test fails, it will give no clue as to why the file failed to open:

FAIL! : tst_QFile::open_write() 'opened' returned FALSE. ()

If there is a more informative error message you could construct from the values being tested, you can use QVERIFY2() to pass that message along with your test condition, to provide a more informative message on failure:

 QVERIFY2(file.open(QIODevice::WriteOnly),
          qPrintable(QString("open %1: %2")
                     .arg(file.fileName()).arg(file.errorString())));

If this branch is being tested in the Qt CI system, the above detailed failure message will be inserted into the summary posted to the code-review system:

FAIL! : tst_QFile::open_write() 'opened' returned FALSE. (open /tmp/qt.a3B42Cd: No space left on device)

See also QVERIFY(), QCOMPARE(), and QEXPECT_FAIL().

QVERIFY(condition)

The QVERIFY() macro checks whether the condition is true or not. If it is true, execution continues. If not, a failure is recorded in the test log and the test won't be executed further.

You can use QVERIFY2() when it is practical and valuable to put additional information into the test failure report.

For example, the following code shows this macro being used to verify that a QSignalSpy object is valid:

 QVERIFY(spy.isValid());

For more information about the failure, use QCOMPARE(x, y) instead of QVERIFY(x == y), because it reports both the expected and actual value when the comparison fails.

See also QCOMPARE(), QTRY_VERIFY(), QSignalSpy, and QEXPECT_FAIL().

QVERIFY_EXCEPTION_THROWN(expression, exceptiontype)

The QVERIFY_EXCEPTION_THROWN macro executes expression and tries to catch an exception thrown from expression.

There are several possible outcomes:

• If expression throws an exception that is either the same as exceptiontype or derived from exceptiontype, then execution will continue.
• Otherwise, if expression throws no exception, or the exception thrown derives from std::exception, then a failure will be recorded in the test log and the macro returns early (from enclosing function).
• If the thrown exception derives neither from std::exception nor from exceptiontype, a failure will be recorded in the test log, and the exception is re-thrown. This avoids problems with e.g. pthread cancellation exceptions.

This function was introduced in Qt 5.3.

QWARN(message)

Appends message as a warning to the test log. This macro can be used anywhere in your tests.

Note: This function is thread-safe.