Browser chrome tests

From MDC

[edit] Introduction

The browser chrome test suite is an automated testing framework designed to allow testing of application chrome windows using JavaScript. It currently allows you to run JavaScript code in the same scope as the main Firefox browser window and report results using the same functions as the Mochitest test framework. The browser chrome test suite depends on runtests.pl from the Mochitest framework, so it won't work in a build with Mochitest disabled (--disable-mochitest).

[edit] Running the browser chrome tests

To run Mochitest you need to build Mozilla with your changes, change directory to $(OBJDIR)/_tests/testing/mochitest, and run the command:

perl runtests.pl --browser-chrome

This will launch your build and open a "browser chrome tests" window. The "run all tests" button will start the test run, and report the results in the UI and to stdout. There's also an option to output the results to file, using the same command line parameter used by Mochitest (--log-file=/path/to/file).

You can tell the test harness to run the tests automatically at startup without user interaction by passing the --autorun parameter to runtests.pl. This can be used in combination with the --close-when-done parameter to fully automate the tests.

If you want to run a single set of browser chrome tests, keep reading.

[edit] Writing browser chrome tests

Browser chrome tests are snippets of JavaScript code that run in the browser window's global scope. A simple test would look like this:

 function test() {
   ok(getBrowser(), "getBrowser() exists");
   is(gBrowser, getBrowser(), "getBrowser() and gBrowser are the same");
 }

The "test" function is invoked by the test harness when the test is run. The test file can contain other functions, they will be ignored unless invoked by test().

The comparison functions are identical to those supported by Mochitests, see how the comparison functions work in the Mochitest documentation for more details. The EventUtils helper functions are available on the "EventUtils" object defined in the global scope.

The test suite also supports asynchronous tests, using the same function names as Mochitest. Call waitForExplicitFinish() from test() if you want to delay reporting a result until after test() has returned. Call finish() once the test is complete. Be aware that the test harness will mark tests that take too long to complete as FAILED (the current timeout is 15 seconds).

 function test() {
   waitForExplicitFinish();
   setTimeout(completeTest, 1000);
 }
 
 function completeTest() {
   ok(true, "Timeout ran");
   finish();
 }

Any exceptions thrown under test() will be caught and reported in the test output as a failure. Exceptions thrown outside of test() (e.g. in a timeout, event handler, etc) will not be caught, but will result in a timed out test if they prevent finish() from being called.

The test file name must be prefixed with "browser_", and must have a file extension of ".js". Files that don't match this pattern will be ignored by the test harness. Putting the bug number in the file name is recommended, e.g. "browser_bug123456.js".

[edit] Adding a new browser chrome test to the tree

To add a new browser chrome test to the tree, follow the Mochitest instructions, keeping in mind that the browser chrome tests must be copied into _tests/testing/mochitest/browser instead of _tests/testing/mochitest/mochitest. Using _BROWSER_TEST_FILES rather than _TEST_FILES as the variable name for the list of tests to install is also recommended, to better differentiate the two sets of tests.

[edit] Running individual tests

The browser chrome test suite does respect (though not require) the --test-path argument, in order to run individual groups of tests. This argument must be a directory, not an individual file. Because browser-chrome is a variation on the standard mochitest framework, the path is relative to $(OBJDIR)/_tests/testing/mochitest/tests directory, even though browser-chrome tests are in $(OBJDIR)/_tests/testing/mochitest/browser.

For example, to run the tests in browser/base/content/test the command would be:

python runtests.py --browser-chrome --test-path=../browser/browser/base/content/test/