Revision 5221 of Running automated tests

  • Revision slug: Running_automated_tests
  • Revision title: Running automated tests
  • Revision id: 5221
  • Created:
  • Creator: Nickolay
  • Is current revision? No
  • Comment

Revision Content

Mozilla code is covered by a large number of automated unit- and regression tests. These tests are run on Tinderbox constantly and developers are expected to make sure their changes do not break the automated test-suites.

Mozilla uses several home-grown automated testing frameworks, an overview of each framework with links to detailed documentation is available at Mozilla automated testing.

This page lists the steps to run the automated tests.

Configuring the build

In order to run most tests, you must have a properly configured build. Platform (Gecko, Toolkit) tests, as well as Firefox-specific tests, are usually run on a Firefox build.

Build Documentation has the general instructions on building Firefox. The default build options are suitable for running the automated tests. The tests can be run on both debug and release builds, but due to bugs in some tests, they may die because of an assertion failure. (XXX find/file a bug that tracks such tests.)

make check

The following commands execute the standalone (aka "compiled-code") tests and the xpcshell-based tests.

$ cd <OBJ_DIR>
$ make check

If any of the tests fail, you get a message like:

make: *** [check] Error 2

If <tt>make</tt> exits without an error, all the tests passed.

MochiTests

If a _tests/ directory exists under the top-level source directory, then the MochiTests have been built. In order to run them, one can run:

% cd _tests/testing/mochitest
%
% sh
$ perl ./runtests.pl --autorun --close-when-done --appname=<executable> --console-level=INFO | grep '^REFTEST '
...
$ perl ./runtests.pl --autorun --close-when-done --appname=<executable> --console-level=INFO -chrome | grep '^REFTEST '
...
$ exit
%

Reftests

% sh
$
$ <executable> --reftest layout/reftests/reftest.list
...
$ exit
%

Other tests not covered in this document

Revision Source

<p>Mozilla code is covered by a large number of automated unit- and regression tests. These tests are run on <a href="en/Tinderbox">Tinderbox</a> constantly and developers are expected to make sure their changes do not break the automated test-suites.
</p><p>Mozilla uses several home-grown automated testing frameworks, an overview of each framework with links to detailed documentation is available at <a href="en/Mozilla_automated_testing">Mozilla automated testing</a>.
</p><p>This page lists the steps to run the automated tests.
</p>
<h3 name="Configuring_the_build"> Configuring the build </h3>
<p>In order to run most tests, you must have a properly configured build. Platform (Gecko, Toolkit) tests, as well as Firefox-specific tests, are usually run on a Firefox build.
</p><p><a href="en/Build_Documentation">Build Documentation</a> has the general instructions on building Firefox. The default build options are suitable for running the automated tests. The tests can be run on both debug and release builds, but due to bugs in some tests, they may die because of an assertion failure. (XXX find/file a bug that tracks such tests.)
</p>
<h3 name="make_check"> <code>make check</code> </h3>
<p>The following commands execute the <a href="en/Compiled-code_automated_tests">standalone (aka "compiled-code") tests</a> and the <a href="en/Writing_xpcshell-based_unit_tests">xpcshell-based tests</a>.
</p>
<pre class="eval">$ cd <b>&lt;OBJ_DIR&gt;</b>
$ make check
</pre>
<p>If any of the tests fail, you get a message like:
</p>
<pre class="eval">make: *** [check] Error 2
</pre>
<p>If <tt>make</tt> exits without an error, all the tests passed.
</p>
<h3 name="MochiTests"> MochiTests </h3>
<p>If a <code>_tests/</code> directory exists under the top-level source directory, then the MochiTests have been built. In order to run them, one can run:
</p>
<pre>% cd _tests/testing/mochitest
%
% sh
$ perl ./runtests.pl --autorun --close-when-done --appname=&lt;executable&gt; --console-level=INFO | grep '^REFTEST '
...
$ perl ./runtests.pl --autorun --close-when-done --appname=&lt;executable&gt; --console-level=INFO -chrome | grep '^REFTEST '
...
$ exit
%
</pre>
<h3 name="Reftests"> Reftests </h3>
<pre>% sh
$
$ &lt;executable&gt; --reftest layout/reftests/reftest.list
...
$ exit
%
</pre>
<h3 name="Other_tests_not_covered_in_this_document"> Other tests not covered in this document </h3>
<ul><li> Tinderbox performance tests - Tp, Txul, Ts, etc. <a class="external" href="http://wiki.mozilla.org/Performance:Tinderbox_Tests">
</a></li><li><a class="external" href="http://wiki.mozilla.org/Performance:Tinderbox_Tests"> Talos
</a></li><li><a class="external" href="http://wiki.mozilla.org/Performance:Tinderbox_Tests"> running the above tests with/without leak, assertion checks
</a></li></ul><a class="external" href="http://wiki.mozilla.org/Performance:Tinderbox_Tests">
</a>
Revert to this revision