Mochitest

  • Revision slug: Mochitest
  • Revision title: Mochitest
  • Revision id: 9891
  • Created:
  • Creator: jlebar
  • Is current revision? No
  • Comment 22 words added

Revision Content

Mochitest is an automated testing framework built on top of the MochiKit JavaScript libraries. It's just one of the automated regression testing frameworks used by Mozilla. Tests report success or failure to the test harness using JavaScript function calls.

Mochitest's unique strength is that it runs tests written as webpages in a full browser environment where the tests have chrome (elevated) privileges. This allows JavaScript in the tests to do much, much more than it would otherwise be able to do. In addition to the capabilities a script would normally have (e.g. DOM manipulation), scripts can access XPCOM components and services, and even access the browser itself. This allows a script to, say, simulate user input to the browser's user interface, before examining the browser to verify that the input had the intended results.

Mochitest's use of JavaScript function calls to communicate test success or failure can make it unsuitable for certain types of test. Only things that can in some way be tested using JavaScript (with chrome privileges!) can be tested with this framework. Given some creativity, that's actually much more than you might first think, but it's not possible to write Mochitest tests to directly test a non-scripted C++ component, for example.  (Use a compiled-code test to do that.)

Running tests

The Mozilla build machines run Mochitest as part of the build process, so we get to know pretty quickly if someone commits a change to the source code that breaks something. However, you should still run Mochitest yourself before you commit any risky new code. You don't want to be the one who wastes everyone's time by breaking the tree if you can help it. :-)

Running the whole test suite

To run Mochitest, first build Mozilla with your changes; then:

  • In 1.9.1 and later (since {{ Bug("417516") }} was fixed) run the following command from the top-level directory:
    • make -C $(OBJDIR) mochitest-plain
  • To test on older branches, run Mochitest's runtests.py script without passing it any command line arguments:
    • cd $(OBJDIR)/_tests/testing/mochitest
      python runtests.py

Image:Mochitest.png

Note: you should keep focus on the browser window while the test are being run, as some may fail otherwise (like the one for {{ Bug("330705") }} for example). Linux users can save themselves this inconvenience by using a dummy X server (see Diverting X output below).

Running select tests

To run a single test (perhaps a new test you just added) or a subset of the entire Mochitest suite, use the --test-path option of runtests.py to specify the test or the subdirectory of tests that you want to run. For example, to run only the test {{ Source("content/base/test/test_CrossSiteXHR.html", "test_CrossSiteXHR.html") }} in the Mozilla source tree, you would run this command:

TEST_PATH=content/base/test/test_CrossSiteXHR.html make -C $(OBJDIR) mochitest-plain

Or on branches where the mochitest-plain target is not supported:

python runtests.py --test-path=content/base/test/test_CrossSiteXHR.html

To run all the tests in {{ Source("content/svg/") }}, this command would work:

TEST_PATH=content/svg/ make -C $(OBJDIR) mochitest-plain

Note that the path specified by the --test-path is the path to the test or directory within the Mozilla source tree. If the path is a directory, then the tests in that directory and all of its subdirectories will be loaded.

Running individual tests in different Firefox sessions

Generally speaking, running a specific mochitest means having a Firefox "client context" load a test over HTTP from the mochitest server.  The mochitest client and server don't have to be running in the same Firefox "session".  Sometimes it is useful to have them running in different sessions; for example, when debugging a failing test. it can be useful to load the test in a distinct Firefox child session to get better debugger support (e.g. with emacs gud-gdb).

To do this, first start the mochitest server with

python $OBJDIR/_tests/testing/mochitest/runtests.py

then start the new, client session as you wish.  For example

cd $OBJDIR/dist/bin; ./firefox-bin -no-remote -P [profile]

Once the child session has loaded, you can run any individual mochitest by navigating to the URL "http://localhost:8888/tests/PATH/TO/MY/TEST".  For example: "http://localhost:8888/tests/modules/...ginstream.html".

Finding errors

Search for the string "TEST-UNEXPECTED-FAIL" to find unexpected failures. You can also search for "SimpleTest FINISHED" to see the final test summary. This is particularly useful when viewing full Tinderbox logs, since the Mochitest output isn't necessarily at the end of the combined log.

Logging results

The output from a test run can be sent to the console and/or a file (by default the results are only displayed in the browser). There are several levels of detail to choose from. The levels are DEBUG, INFO, WARNING, ERROR and FATAL, where DEBUG produces the highest detail (everything), and FATAL produces the least (a message will only be logged for an event that causes the test run to abort prematurely).

To log to a file use --log-file=FILE. By default the file logging level is INFO but you can change this using --file-level=LEVEL.

To turn on logging to the console use --console-level=LEVEL.

For example, to log test run output to the file ~/mochitest.log at DEBUG level detail you would use:

python runtests.py --log-file=~/mochitest.log --file-level=DEBUG

Diverting X output

The tests must run in a focused window, which effectively prevents any other user activity on the engaged computer. Linux users can reclaim their boxes by telling the suite to use a hidden virtual desktop. If Xvfb is or can be installed, the following command launches the tests without blocking the active session:

nice xvfb-run python _tests/testing/mochitest/runtests.py --log-file=./mochitest-plain.log --file-level=DEBUG --autorun --close-when-done --console-level=DEBUG

Other possible configurations have also been discussed in {{ Bug("434365") }}.

Other runtests.py options

The runtests.py script recognizes several other options - use the --help option to get a list. Note that there is separate documentation for the --chrome, --browser-chrome and --a11y options.

Writing tests

A Mochitest test is simply an HTML, XHTML or XUL file that contains some JavaScript to test for some condition(s).

You can use Mochitest maker to run most tests without having to build Mozilla.

Try to avoid Mochitest

Yes, really. For many things Mochitest is overkill. In general you should always try to use one of the lighter-weight testing frameworks. For example, if you only want to test a single XPCOM component then you should use xpcshell. On the other hand there are some things that Mochitest cannot do, or isn't designed to do. For example, for visual output tests you should try to use the reftest framework. For more information on the different types of automated testing frameworks see Mozilla automated testing.

Test templates

You can avoid typing out boilerplate by using the {{ Source("testing/mochitest/gen_template.pl", "gen_template.pl") }} perl script to generate a test template. This script takes two optional arguments:

  1. -b : a bug number
  2. -type : template type. {html|xhtml|xul}. defaults to html.

For example:

cd mozilla/testing/mochitest/
perl gen_template.pl -b=123456 > path/to/test_bug123456.html
perl gen_template.pl -b=123456 --type=xul > path/to/test_bug123456.xul

Note that Mochitest requires the file name of all tests to begin with the string "test_". See the section below for help on deciding where your tests should go in the tree.

In addition to Mochitest boilerplate code, the script will generate an element with the id 'content' and an element with the id 'display'. Your test may manipulate these elements as well as other elements you add to the page.

Test functions

Each test must contain some JavaScript that will run and tell Mochitest whether the test has passed or failed. {{ Source("testing/mochitest/tests/SimpleTest/SimpleTest.js", "SimpleTest.js") }} provides a number of functions for the test to use that communicate the pass/fail to Mochitest. These include:

  • ok(expressionThatShouldBeTrue, "Error message") -- tests a value for truthiness
  • is(actualValue, expectedValue, "Error message") -- compares two values (using ==, not ===)
  • isnot(actualValue, unexpectedValue, "Error message") -- opposite of is()

See the {{ Source("testing/mochitest/README.txt", "README") }} for an example of their use.

If you want to include a test for something that currently fails, don't just comment it out!  Instead, use one of the "todo" equivalents so Tinderbox can notice if it suddenly starts passing (at which point the test can be reenabled):

  • todo(falseButShouldBeTrue, "Error message")
  • todo_is(actualValue, expectedValue, "Error message")
  • todo_isnot(actualValue, unexpectedValue, "Error message")

Helper functions

Right now all of Mochikit is available (this will change in {{ Bug("367393") }}); {{ Bug("367569") }} added sendChar, sendKey, and sendString helpers. These are available in {{ Source("testing/mochitest/tests/SimpleTest/EventUtils.js") }}.

Adding tests to the tree

Once you've written a new test you need to add it to the Mozilla source tree and tell the build system about it so that the Mozilla tinderboxes will run it automatically.

Choosing a location

New Mochitest tests should go somewhere close to the code they are testing, hopefully in the same module, so that ownership of the test cases is clear. For example, if you create a new test for some HTML feature, you probably want to put the test in {{ Source("content/html/content/test") }} or {{ Source("content/html/document/test") }}. If a test directory does not exist near the code you are testing you can add a new test directory as the patch in {{ Bug("368531") }} demonstrates.

Makefile changes

To tell the build system about your new test you need to add the name of your test file to _TEST_FILES in the test directory's Makefile.in.

If your test spans multiple files, only name the main one "test_...". This is the one that will show up in the list of testcases to run. The other files should have some other name, but must still be added to the _TEST_FILES in Makefile.in.

Keep in mind that if you're adding chrome tests, you'll need to change the Makefile to install the tests in _tests/testing/mochitest/chrome rather than _tests/testing/mochitest/tests.

Building and running new tests

Before committing a new test you should check that the Makefile.in changes are correct and that your tests pass as you expect them to. To check your test, first export it to the Mochitest directory by running the command:

make

in the object directory corresponding to the test file's location in the source tree. Now open Mochitest as described above, but this time, instead of clicking on the "Run Tests" link, search for your test and click on it.

SSL and https-enabled tests

 Mochitests must be run from http://localhost:8888 to succeed; however, some tests may require use of additional protocols, hosts, or ports to test cross-origin functionality.  The Mochitest harness addresses this need by mirroring all content of the original server onto a variety of other servers through the magic of proxy autoconfig and SSL tunneling.  The full list of schemes, hosts, and ports on which tests are served, all of which serve exactly the same content as http://localhost:8888, is specified in {{ Source("build/pgo/server-locations.txt") }}.  Note, however, that not all origins described there are equivalent: some specify particular SSL certificates for testing purposes, while some allow pages on that server to request elevated privileges; read the file for full details.

How it works


The Mochitest harness includes preference values which cause the browser to use proxy autoconfig to match requested URLs with servers.  The network.proxy.autoconfig_url preference is set to a data: URL which encodes a JavaScript function, FindProxyForURL, which determines the host to which a given URL is mapped.  In the case of SSL sites to be mirrored, the function maps them to an SSL tunnel which transparently forwards the traffic to the actual server, as per the description of the CONNECT method given in RFC 2817.  In this manner a single HTTP server at http://127.0.0.1:8888 can successfully emulate dozens of servers at distinct locations.

For further details on Mochitest SSL functionality and how to modify it to change a certificate or add a new https server, see Modifying Mochitest SSL behavior.

Getting Stack Traces

To get stack when Mochitest crashes:

  1. Get a minidump_stackwalk binary for your platform from http://hg.mozilla.org/build/tools/fi...e49a/breakpad/
  2. Set the MINIDUMP_STACKWALK environment variable to point to the binary.

If the resulting stack trace doesn't have line numbers, run make buildsymbols to generate the requisite symbol files.

FAQ

What if my tests aren't done when onload fires?

Call SimpleTest.waitForExplicitFinish() before onload fires. Then, when you're done, call SimpleTest.finish().

What if I need to change a preference to run my test?

netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect");
var prefService = Components.classes["@mozilla.org/preferences-service;1"]
                            .getService(Components.interfaces.nsIPrefService);
var domBranch = prefService.getBranch("dom.");
var oldVal = domBranch.getIntPref("max_script_run_time");
domBranch.setIntPref("max_script_run_time", 0);

// do what you need

domBranch.setIntPref("max_script_run_time", oldVal);

If you need to change a pref when running a test locally, you can set the environment variable EXTRA_TEST_ARGS when running the mochitest-plain make target.

EXTRA_TEST_ARGS='--setpref=javascript.options.jit.chrome=false'

Can tests be run under a chrome URL?

Yes, use python runtests.py --chrome. Keep in mind that the xpcshell test harness should be your first choice for XPCOM testing. Only use mochitest if you need events, browser features, networking, etc.

How can I get around the error "Permission denied to get property XPCComponents.classes"?

Adding the following line to your test file (and each event handler) will allow full XPCOM usage.

netscape.security.PrivilegeManager.enablePrivilege('UniversalXPConnect');

This approach is obviously inconvenient. That's why we're working on the build hacking necessary to copy tests into a chrome directory for testing.

How do I change the HTTP headers or status sent with a file used in a Mochitest?

Create a text file next to the file whose headers you want to modify. The name of the text file should be the name of the file whose headers you're modifying followed by ^headers^. For example, if you have a file foo.jpg, the text file should be named foo.jpg^headers^. (Don't try to actually use the headers file in any other way in the test, because the HTTP server's hidden-file functionality prevents any file ending in exactly one ^ from being served.) Edit the file to contain the headers and/or status you want to set, like so:

HTTP 404 Not Found
Content-Type: text/html
Random-Header-of-Doom: 17

The first line sets the HTTP status and (optionally a) description associated with the file. This line is optional; you don't need it if you're fine with the normal response status and description. Any other lines in the file describe additional headers which you want to add or overwrite (most typically the Content-Type header, for the latter case) on the response. The format follows the conventions of HTTP, except that you don't need to have HTTP line endings and you can't use a header more than once (the last line for a particular header wins). The file may end with at most one blank line to match Unix text file conventions, but the trailing newline isn't strictly necessary.

How do I test issues which only show up when tests are run across domains?

The Mochitest harness runs one web server to serve tests, but through the magic of proxy autoconfig, all test files are available on a variety of different domains and ports. Tests running on any of these servers (with two exceptions for testing privilege escalation functionality) automatically have the ability to request elevated privileges such as UniversalXPConnect. The full list of domains and ports on which tests are served, all of which serve exactly the same content as http://localhost:8888, is specified in {{ Source("build/pgo/server-locations.txt") }}.

How do I write tests that check header values, method types, etc. of HTTP requests?

To write such a test, you simply need to write an SJS (server-side JavaScript) for it. An SJS is simply a JavaScript file with the extension sjs which is loaded in a sandbox; the global property handleRequest defined by the script is then executed with request and response objects, and the script populates the response based on the information in the request.

Here's an example of a simple SJS:

function handleRequest(request, response)
{
  // avoid confusing cache behaviors
  response.setHeader("Cache-Control", "no-cache", false);

  response.setHeader("Content-Type", "text/plain", false);
  response.write("Hello world!");
}

The exact properties of the request and response parameters are defined in the nsIHttpRequestMetadata and nsIHttpResponse interfaces in {{ Source("netwerk/test/httpserver/nsIHttpServer.idl", "nsIHttpServer.idl") }}. Note carefully: the browser is free to cache responses generated by your script, so if you ever want an SJS to return different data for multiple requests to the same URL, you should add a Cache-Control: no-cache header to the response to prevent the test from accidentally failing if it's manually run multiple times in the same Mochitest session.

A simple example of an SJS used in reftests is {{ Source("modules/libpr0n/test/reftest/generic/check-header.sjs", "check-header.sjs") }}.

How do I keep state across loads of different server-side scripts?

Server-side scripts in Mochitest are run inside sandboxes, with a new sandbox created for each new load.  Consequently, any variables set in a handler don't persist across loads.  To support state storage, use the getState(k) and setState(k, v) methods defined on the global object.  These methods expose a key-value storage mechanism for the server, with keys and values as strings.  (Use JSON to store objects and other structured data.)  Note that because the myriad servers in Mochitest are in reality a single server with some proxying and tunnelling magic, stored state is the same in all servers at all times.

The getState and setState methods are scoped to the path being loaded.  For example, the absolute URLs /foo/bar/baz/foo/bar/baz?quux, and /foo/bar/baz#fnord all share the same state; the state for /foo/bar is entirely separate.  You should use per-path state whenever possible to avoid inter-test dependencies and bugs.  However, in rare cases it may be necessary for two scripts to collaborate in some manner, and it may not be possible to use a custom query string to request divergent behaviors from the script.  For this use case only you should use the getSharedState(k, v) and setSharedState(k, v) methods defined on the global object.  No restrictions are placed on access to this whole-server shared state, and any script may add new state that any other script may delete.  To avoid conflicts, you should use a key within a faux namespace so as to avoid accidental conflicts as far as is possible; for example, if you needed shared state for an HTML5 video test, you might use a key like dom.media.video:sharedState.

A further form of state storage is provided by the getObjectState(k) and setObjectState(k, v) methods, which will store any nsISupports object.  These methods reside on the nsIHttpServer interface in this form, but a limitation of the sandbox object used by the server to process SJS responses means that the former is present in the SJS request handler's global environment with the signature getObjectState(k, callback), where callback is a function to be invoked by getObjectState with the object corresponding to the provided key as the sole argument.  Note that this value mapping requires the value to be an XPCOM object; an arbitrary JavaScript object with no QueryInterface method is insufficient.  If you wish to store a JavaScript object, you may find it useful to provide the object with a QueryInterface implementation and then make use of wrappedJSObject to reveal the actual JavaScript object through the wrapping performed by XPConnect.

For further details on state-saving mechanisms provided by httpd.js, see {{ Source("netwerk/test/httpserver/nsIHttpServer.idl") }} and the nsIHttpServer.get(Shared|Object)?State methods.

How do I get access to the files on the server as XPCOM objects from an SJS script? (1.9.3 or later)

If you need access to a file, say, because it's easier to store image data in a file than directly in an SJS script, use the presupplied SERVER_ROOT object state available to SJS scripts running in Mochitest:

function handleRequest(req, res)
{
  var file;
  getObjectState("SERVER_ROOT", function(serverRoot)
  {
    file = serverRoot.getFile("tests/content/media/test/320x240.ogv");
  });

  // file is now an XPCOM object referring to the given file
  res.write("file: " + file);
}

The path you specify is used as a path relative to the root directory served by httpd.js , and an nsIFile corresponding to the file at that location is returned.  Beware of typos: the file you specify doesn't actually have to exist because file objects are mere encapsulations of string paths.

{{ languages( { "ja": "ja/Mochitest" } ) }}

Revision Source

<p>Mochitest is an <a class="internal" href="/en/Mozilla_automated_testing" title="En/Mozilla automated testing">automated testing framework</a> built on top of the <a class="external" href="http://mochikit.com/">MochiKit</a> JavaScript libraries. It's just one of the automated regression testing frameworks used by Mozilla. Tests report success or failure to the test harness using JavaScript function calls.</p>
<p>Mochitest's unique strength is that it runs tests written as webpages in a full browser environment where the tests have chrome (elevated) privileges. This allows JavaScript in the tests to do much, much more than it would otherwise be able to do. In addition to the capabilities a script would normally have (e.g. DOM manipulation), scripts can access XPCOM components and services, and even access the browser itself. This allows a script to, say, simulate user input to the browser's user interface, before examining the browser to verify that the input had the intended results.</p>
<p>Mochitest's use of JavaScript function calls to communicate test success or failure can make it unsuitable for certain types of test. Only things that can in some way be tested using JavaScript (with chrome privileges!) can be tested with this framework. Given some creativity, that's actually much more than you might first think, but it's not possible to write Mochitest tests to directly test a non-scripted C++ component, for example.  (Use a <a class="internal" href="/en/Compiled-code_automated_tests" title="En/Compiled-code automated tests">compiled-code test</a> to do that.)</p>
<h3 name="Running_tests">Running tests</h3>
<p>The Mozilla build machines run Mochitest as part of the build process, so we get to know pretty quickly if someone commits a change to the source code that breaks something. However, you should still run Mochitest yourself before you commit any risky new code. You don't want to be the one who wastes everyone's time by breaking the tree if you can help it. :-)</p>
<h4 name="Running_the_whole_test_suite">Running the whole test suite</h4>
<p>To run Mochitest, first <a href="/En/Developer_Guide/Build_Instructions" title="en/Build_Documentation">build Mozilla</a> with your changes; then:</p>
<ul> <li>In 1.9.1 and later (since {{ Bug("417516") }} was fixed) run the following command from the top-level directory:<br> <ul> <li><code>make -C $(OBJDIR) mochitest-plain</code></li> </ul> </li> <li>To test on older branches, run Mochitest's <code>runtests.py</code> script without passing it any command line arguments:<br> <ul> <li><code>cd $(OBJDIR)/_tests/testing/mochitest<br> python runtests.py</code></li> </ul> </li>
</ul>
<p><img alt="Image:Mochitest.png" class="internal" src="/@api/deki/files/269/=Mochitest.png"></p>
<p><strong>Note:</strong> you should keep focus on the browser window while the test are being run, as some may fail otherwise (like the one for {{ Bug("330705") }} for example). Linux users can save themselves this inconvenience by using a dummy X server (see <a href="#Diverting_X_output">Diverting X output</a> below).</p>
<h4 name="Running_select_tests">Running select tests</h4>
<p>To run a single test (perhaps a new test you just added) or a subset of the entire Mochitest suite, use the <code>--test-path</code> option of runtests.py to specify the test or the subdirectory of tests that you want to run. For example, to run only the test {{ Source("content/base/test/test_CrossSiteXHR.html", "test_CrossSiteXHR.html") }} in the Mozilla source tree, you would run this command:</p>
<pre><code>TEST_PATH=content/base/test/test_CrossSiteXHR.html make -C $(OBJDIR) mochitest-plain</code>
</pre>
<p>Or on branches where the mochitest-plain target is not supported:</p>
<pre>python runtests.py --test-path=content/base/test/test_CrossSiteXHR.html</pre>
<p>To run all the tests in {{ Source("content/svg/") }}, this command would work:</p>
<pre class="eval">TEST_PATH=content/svg/ <code>make -C $(OBJDIR) mochitest-plain</code>
</pre>
<p>Note that the path specified by the <code>--test-path</code> is the path to the test or directory within the Mozilla source tree. If the path is a directory, then the tests in that directory and all of its subdirectories will be loaded.</p>
<h4>Running individual tests in different Firefox sessions</h4>
<p>Generally speaking, running a specific mochitest means having a Firefox "client context" load a test over HTTP from the mochitest server.  The mochitest client and server don't have to be running in the same Firefox "session".  Sometimes it is useful to have them running in different sessions; for example, when debugging a failing test. it can be useful to load the test in a distinct Firefox child session to get better debugger support (e.g. with emacs gud-gdb).</p>
<p>To do this, first start the mochitest server with</p>
<pre>python $OBJDIR/_tests/testing/mochitest/runtests.py
</pre>
<p>then start the new, client session as you wish.  For example</p>
<pre>cd $OBJDIR/dist/bin; ./firefox-bin -no-remote -P [profile]</pre>
<p>Once the child session has loaded, you can run any individual mochitest by navigating to the URL "<a class=" external" href="http://localhost:8888/tests/PATH/TO/MY/TEST" rel="freelink">http://localhost:8888/tests/PATH/TO/MY/TEST</a>".  For example: "<a class=" external" href="http://localhost:8888/tests/modules/plugin/test/test_pluginstream.html" rel="freelink">http://localhost:8888/tests/modules/...ginstream.html</a>".</p>
<h4 name="Finding_errors">Finding errors</h4>
<p>Search for the string "TEST-UNEXPECTED-FAIL" to find unexpected failures. You can also search for "SimpleTest FINISHED" to see the final test summary. This is particularly useful when viewing full Tinderbox logs, since the Mochitest output isn't necessarily at the end of the combined log.</p>
<h4 name="Logging_results">Logging results</h4>
<p>The output from a test run can be sent to the console and/or a file (by default the results are only displayed in the browser). There are several levels of detail to choose from. The levels are DEBUG, INFO, WARNING, ERROR and FATAL, where DEBUG produces the highest detail (everything), and FATAL produces the least (a message will only be logged for an event that causes the test run to abort prematurely).</p>
<p>To log to a file use --log-file=FILE. By default the file logging level is INFO but you can change this using --file-level=LEVEL.</p>
<p>To turn on logging to the console use --console-level=LEVEL.</p>
<p>For example, to log test run output to the file <code>~/mochitest.log</code> at DEBUG level detail you would use:</p>
<pre class="eval">python runtests.py --log-file=~/mochitest.log --file-level=DEBUG
</pre>
<h4 name="Diverting_X_output">Diverting X output</h4>
<p>The tests must run in a focused window, which effectively prevents any other user activity on the engaged computer. Linux users can reclaim their boxes by telling the suite to use a hidden virtual desktop. If <a class="external" href="http://en.wikipedia.org/wiki/Xvfb">Xvfb</a> is or can be installed, the following command launches the tests without blocking the active session:</p>
<pre class="eval">nice xvfb-run python _tests/testing/mochitest/runtests.py --log-file=./mochitest-plain.log --file-level=DEBUG --autorun --close-when-done --console-level=DEBUG
</pre>
<p>Other possible configurations have also been discussed in {{ Bug("434365") }}.</p>
<h4 name="Other_.27runtests.27_options">Other <code>runtests.py</code> options</h4>
<p>The <code>runtests.py</code> script recognizes several other options - use the --help option to get a list. Note that there is separate documentation for the <a href="/en/Chrome_tests" title="en/Chrome_tests">--chrome</a>, <a href="/en/Browser_chrome_tests" title="en/Browser_chrome_tests">--browser-chrome</a> and <a href="/en/Accessibility" title="en/Accessibility">--a11y</a> options.</p>
<h3 name="Writing_tests">Writing tests</h3>
<p>A Mochitest test is simply an HTML, XHTML or XUL file that contains some JavaScript to test for some condition(s).</p>
<p>You can use <a class="external" href="http://ted.mielczarek.org/code/mozilla/mochitest-maker/">Mochitest maker</a> to run most tests without having to build Mozilla.</p>
<h4 name="Try_to_avoid_Mochitest">Try to avoid Mochitest</h4>
<p>Yes, really. For many things Mochitest is overkill. In general you should always try to use one of the lighter-weight testing frameworks. For example, if you only want to test a single XPCOM component then you should use <a href="/en/Writing_xpcshell-based_unit_tests" title="en/Writing_xpcshell-based_unit_tests">xpcshell</a>. On the other hand there are some things that Mochitest cannot do, or isn't designed to do. For example, for visual output tests you should try to use the <a href="/en/Creating_reftest-based_unit_tests" title="en/Creating_reftest-based_unit_tests">reftest</a> framework. For more information on the different types of automated testing frameworks see <a href="/en/Mozilla_automated_testing" title="en/Mozilla_automated_testing">Mozilla automated testing</a>.</p>
<h4 name="Test_templates">Test templates</h4>
<p>You can avoid typing out boilerplate by using the {{ Source("testing/mochitest/gen_template.pl", "gen_template.pl") }} perl script to generate a test template. This script takes two optional arguments:</p>
<ol> <li>-b : a bug number</li> <li>-type : template type. {html|xhtml|xul}. defaults to html.</li>
</ol>
<p>For example:</p>
<pre class="eval">cd mozilla/testing/mochitest/
perl gen_template.pl -b=123456 &gt; path/to/test_bug123456.html
perl gen_template.pl -b=123456 --type=xul &gt; path/to/test_bug123456.xul
</pre>
<p>Note that Mochitest requires the file name of all tests to begin with the string "test_". See the section below for help on deciding where your tests should go in the tree.</p>
<p>In addition to Mochitest boilerplate code, the script will generate an element with the id 'content' and an element with the id 'display'. Your test may manipulate these elements as well as other elements you add to the page.</p>
<h4 name="Test_functions">Test functions</h4>
<p>Each test must contain some JavaScript that will run and tell Mochitest whether the test has passed or failed. {{ Source("testing/mochitest/tests/SimpleTest/SimpleTest.js", "SimpleTest.js") }} provides a number of functions for the test to use that communicate the pass/fail to Mochitest. These include:</p>
<ul> <li><code>ok(<var>expressionThatShouldBeTrue</var>, "<var>Error message</var>")</code> -- tests a value for truthiness</li> <li><code>is(<var>actualValue</var>, <var>expectedValue</var>, "<var>Error message</var>")</code> -- compares two values (using ==, not ===)</li> <li><code>isnot(<var>actualValue</var>, <var>unexpectedValue</var>, "<var>Error message</var>")</code> -- opposite of is()</li>
</ul>
<p>See the {{ Source("testing/mochitest/README.txt", "README") }} for an example of their use.</p>
<p>If you want to include a test for something that currently fails, don't just comment it out!  Instead, use one of the "todo" equivalents so Tinderbox can notice if it suddenly starts passing (at which point the test can be reenabled):</p>
<ul> <li><code>todo(<var>falseButShouldBeTrue</var>, "<var>Error message</var>")</code></li> <li><code>todo_is(<var>actualValue</var>, <var>expectedValue</var>, "<var>Error message</var>")</code></li> <li><code>todo_isnot(<var>actualValue</var>, <var>unexpectedValue</var>, "<var>Error message</var>")</code></li>
</ul>
<h4 name="Helper_functions">Helper functions</h4>
<p>Right now all of Mochikit is available (this will change in {{ Bug("367393") }}); {{ Bug("367569") }} added <code>sendChar</code>, <code>sendKey</code>, and <code>sendString</code> helpers. These are available in {{ Source("testing/mochitest/tests/SimpleTest/EventUtils.js") }}.</p>
<h3 name="Adding_tests_to_the_tree">Adding tests to the tree</h3>
<p>Once you've written a new test you need to add it to the Mozilla source tree and tell the build system about it so that the Mozilla tinderboxes will run it automatically.</p>
<h4 name="Choosing_a_location">Choosing a location</h4>
<p>New Mochitest tests should go somewhere close to the code they are testing, hopefully in the same module, so that ownership of the test cases is clear. For example, if you create a new test for some HTML feature, you probably want to put the test in {{ Source("content/html/content/test") }} or {{ Source("content/html/document/test") }}. If a test directory does not exist near the code you are testing you can add a new test directory as the patch in {{ Bug("368531") }} demonstrates.</p>
<h4 name="Makefile_changes">Makefile changes</h4>
<p>To tell the build system about your new test you need to add the name of your test file to <code>_TEST_FILES</code> in the test directory's <code>Makefile.in</code>.</p>
<p>If your test spans multiple files, only name the main one "test_...". This is the one that will show up in the list of testcases to run. The other files should have some other name, but must still be added to the <code>_TEST_FILES</code> in <code>Makefile.in</code>.</p>
<p>Keep in mind that if you're adding chrome tests, you'll need to change the Makefile to install the tests in <code>_tests/testing/mochitest/<strong>chrome</strong></code> rather than <code>_tests/testing/mochitest/<strong>tests</strong></code>.</p>
<h4 name="Building_and_running_new_tests">Building and running new tests</h4>
<p>Before committing a new test you should check that the Makefile.in changes are correct and that your tests pass as you expect them to. To check your test, first export it to the Mochitest directory by running the command:</p>
<pre class="eval">make
</pre>
<p>in the object directory corresponding to the test file's location in the source tree. Now open Mochitest as described above, but this time, instead of clicking on the "Run Tests" link, search for your test and click on it.</p>
<h3 name="SSL">SSL and <code>https</code>-enabled tests</h3>
<p> Mochitests must be run from <code><span class="nowiki">http://localhost:8888</span></code> to succeed; however, some tests may require use of additional protocols, hosts, or ports to test cross-origin functionality.  The Mochitest harness addresses this need by <em>mirroring</em> all content of the original server onto a variety of other servers through the magic of proxy autoconfig and SSL tunneling.  The full list of schemes, hosts, and ports on which tests are served, all of which serve exactly the same content as <code><span class="nowiki">http://localhost:8888</span></code>, is specified in {{ Source("build/pgo/server-locations.txt") }}.  Note, however, that not all origins described there are equivalent: some specify particular SSL certificates for testing purposes, while some allow pages on that server to request elevated privileges; read the file for full details.</p>
<h4>How it works</h4>
<p><br>
The Mochitest harness includes preference values which cause the browser to use <a class="external" href="http://en.wikipedia.org/wiki/Proxy_auto-config" title="http://en.wikipedia.org/wiki/Proxy_auto-config">proxy autoconfig</a> to match requested URLs with servers.  The <code>network.proxy.autoconfig_url</code> preference is set to a <code>data:</code> URL which encodes a JavaScript function, <code>FindProxyForURL</code>, which determines the host to which a given URL is mapped.  In the case of SSL sites to be mirrored, the function maps them to an SSL tunnel which transparently forwards the traffic to the actual server, as per the description of the <code>CONNECT</code> method given in <a class="external" href="http://www.ietf.org/rfc/rfc2817.txt" title="http://www.ietf.org/rfc/rfc2817.txt">RFC 2817</a>.  In this manner a single HTTP server at <code><a class=" external" href="http://127.0.0.1:8888" rel="freelink">http://127.0.0.1:8888</a></code> can successfully emulate dozens of servers at distinct locations.</p>
<p>For further details on Mochitest SSL functionality and how to modify it to change a certificate or add a new https server, see <a class="internal" href="/En/Modifying_Mochitest_SSL_behavior" title="En/Modifying Mochitest SSL behavior">Modifying Mochitest SSL behavior</a>.</p>
<h3 name="stacks">Getting Stack Traces</h3>
<p>To get stack when Mochitest crashes:</p>
<ol> <li>Get a <code>minidump_stackwalk</code> binary for your platform from <code><a class=" external" href="http://hg.mozilla.org/build/tools/file/b680aba8e49a/breakpad/" rel="freelink">http://hg.mozilla.org/build/tools/fi...e49a/breakpad/</a></code></li> <li>Set the <code>MINIDUMP_STACKWALK</code> environment variable to point to the binary.</li>
</ol>
<p>If the resulting stack trace doesn't have line numbers, run <code>make buildsymbols</code> to generate the requisite symbol files.</p><h3 name="FAQ">FAQ</h3>
<h4 name="What_if_my_tests_aren.27t_done_when_onload_fires.3F">What if my tests aren't done when onload fires?</h4>
<p>Call <code>SimpleTest.waitForExplicitFinish()</code> before onload fires. Then, when you're done, call <code>SimpleTest.finish()</code>.</p>
<h4 name="What_if_I_need_to_change_a_preference_to_run_my_test.3F">What if I need to change a preference to run my test?</h4>
<pre class="eval">netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect");
var prefService = Components.classes["@mozilla.org/preferences-service;1"]
                            .getService(Components.interfaces.nsIPrefService);
var domBranch = prefService.getBranch("dom.");
var oldVal = domBranch.getIntPref("max_script_run_time");
domBranch.setIntPref("max_script_run_time", 0);

// do what you need

domBranch.setIntPref("max_script_run_time", oldVal);
</pre>
<p>If you need to change a pref when running a test locally, you can set the environment variable <code>EXTRA_TEST_ARGS</code> when running the <code>mochitest-plain</code> make target.</p>
<pre>EXTRA_TEST_ARGS='--setpref=javascript.options.jit.chrome=false'
</pre>
<h4 name="Can_tests_be_run_under_a_chrome_URL.3F">Can tests be run under a chrome URL?</h4>
<p>Yes, use <code>python runtests.py --chrome</code>. Keep in mind that the <a href="/en/Writing_xpcshell-based_unit_tests" title="en/Writing_xpcshell-based_unit_tests">xpcshell test harness</a> should be your first choice for XPCOM testing. Only use mochitest if you need events, browser features, networking, etc.</p>
<h4 name="How_can_I_get_around_the_error_.22Permission_denied_to_get_property_XPCComponents.classes.22.3F">How can I get around the error "Permission denied to get property XPCComponents.classes"?</h4>
<p>Adding the following line to your test file (and each event handler) will allow full XPCOM usage.</p>
<pre>netscape.security.PrivilegeManager.enablePrivilege('UniversalXPConnect');</pre>
<p>This approach is obviously inconvenient. That's why we're working on the build hacking necessary to copy tests into a chrome directory for testing.</p>
<h4 name="How_do_I_change_the_HTTP_headers_or_status_sent_with_a_file_used_in_a_Mochitest.3F">How do I change the HTTP headers or status sent with a file used in a Mochitest?</h4>
<p>Create a text file next to the file whose headers you want to modify. The name of the text file should be the name of the file whose headers you're modifying followed by <code>^headers^</code>. For example, if you have a file <code>foo.jpg</code>, the text file should be named <code>foo.jpg^headers^</code>. (Don't try to actually use the headers file in any other way in the test, because the HTTP server's hidden-file functionality prevents any file ending in exactly one <code>^</code> from being served.) Edit the file to contain the headers and/or status you want to set, like so:</p>
<pre class="eval">HTTP 404 Not Found
Content-Type: text/html
Random-Header-of-Doom: 17
</pre>
<p>The first line sets the HTTP status and (optionally a) description associated with the file. This line is optional; you don't need it if you're fine with the normal response status and description. Any other lines in the file describe additional headers which you want to add or overwrite (most typically the Content-Type header, for the latter case) on the response. The format follows the conventions of HTTP, except that you don't need to have HTTP line endings and you can't use a header more than once (the last line for a particular header wins). The file may end with at most one blank line to match Unix text file conventions, but the trailing newline isn't strictly necessary.</p>
<h4 name="How_do_I_test_issues_which_only_show_up_when_tests_are_run_across_domains.3F">How do I test issues which only show up when tests are run across domains?</h4>
<p>The Mochitest harness runs one web server to serve tests, but through the magic of proxy autoconfig, all test files are available on a variety of different domains and ports. Tests running on any of these servers (with two exceptions for testing privilege escalation functionality) automatically have the ability to request elevated privileges such as UniversalXPConnect. The full list of domains and ports on which tests are served, all of which serve exactly the same content as <code><span class="nowiki">http://localhost:8888</span></code>, is specified in {{ Source("build/pgo/server-locations.txt") }}.</p>
<h4 name="How_do_I_write_tests_that_check_header_values.2C_method_types.2C_etc._of_HTTP_requests.3F">How do I write tests that check header values, method types, etc. of HTTP requests?</h4>
<p>To write such a test, you simply need to write an SJS (server-side JavaScript) for it. An SJS is simply a JavaScript file with the extension <code>sjs</code> which is loaded in a sandbox; the global property <code>handleRequest</code> defined by the script is then executed with request and response objects, and the script populates the response based on the information in the request.</p>
<p>Here's an example of a simple SJS:</p>
<pre class="eval">function handleRequest(request, response)
{
  // avoid confusing cache behaviors
  response.setHeader("Cache-Control", "no-cache", false);

  response.setHeader("Content-Type", "text/plain", false);
  response.write("Hello world!");
}
</pre>
<p>The exact properties of the request and response parameters are defined in the <code>nsIHttpRequestMetadata</code> and <code>nsIHttpResponse</code> interfaces in <code>{{ Source("netwerk/test/httpserver/nsIHttpServer.idl", "nsIHttpServer.idl") }}</code>. Note carefully: the browser is free to cache responses generated by your script, so if you ever want an SJS to return different data for multiple requests to the same URL, you should add a <code>Cache-Control: no-cache</code> header to the response to prevent the test from accidentally failing if it's manually run multiple times in the same Mochitest session.</p>
<p>A simple example of an SJS used in reftests is <code>{{ Source("modules/libpr0n/test/reftest/generic/check-header.sjs", "check-header.sjs") }}</code>.</p>
<h4 name="How_do_I_test_issues_which_only_show_up_when_tests_are_run_across_domains.3F">How do I keep state across loads of different server-side scripts?</h4>
<p>Server-side scripts in Mochitest are run inside sandboxes, with a new sandbox created for each new load.  Consequently, any variables set in a handler don't persist across loads.  To support state storage, use the <code>getState(k)</code> and <code>setState(k, v)</code> methods defined on the global object.  These methods expose a key-value storage mechanism for the server, with keys and values as strings.  (Use JSON to store objects and other structured data.)  Note that because the myriad servers in Mochitest are in reality a single server with some proxying and tunnelling magic, stored state is the same in all servers at all times.</p>
<p>The <code>getState</code> and <code>setState</code> methods are scoped to the path being loaded.  For example, the absolute URLs <code>/foo/bar/baz</code>,  <code>/foo/bar/baz?quux</code>, and <code>/foo/bar/baz#fnord</code> all share the same state; the state for <code>/foo/bar</code> is entirely separate.  You should use per-path state whenever possible to avoid inter-test dependencies and bugs.  However, in rare cases it may be necessary for two scripts to collaborate in some manner, and it may not be possible to use a custom query string to request divergent behaviors from the script.  <em>For this use case only</em> you should use the <code>getSharedState(k, v)</code> and <code>setSharedState(k, v)</code> methods defined on the global object.  No restrictions are placed on access to this whole-server shared state, and any script may add new state that any other script may delete.  To avoid conflicts, you should use a key within a faux namespace so as to avoid accidental conflicts as far as is possible; for example, if you needed shared state for an HTML5 video test, you might use a key like <span style="font-family: monospace;">dom.media.video</span><code>:sharedState</code>.</p>
<p>A further form of state storage is provided by the <code>getObjectState(k)</code> and <code>setObjectState(k, v)</code> methods, which will store any <code>nsISupports</code> object.  These methods reside on the <code>nsIHttpServer</code> interface in this form, but a limitation of the sandbox object used by the server to process SJS responses means that the former is present in the SJS request handler's global environment with the signature <code>getObjectState(k, callback)</code>, where <code>callback</code> is a function to be invoked by <code>getObjectState</code> with the object corresponding to the provided key as the sole argument.  Note that this value mapping requires the value to be an XPCOM object; an arbitrary JavaScript object with no <code>QueryInterface</code> method is insufficient.  If you wish to store a JavaScript object, you may find it useful to provide the object with a <code>QueryInterface</code> implementation and then make use of <a class="internal" href="/en/wrappedJSObject" title="en/wrappedJSObject">wrappedJSObject</a> to reveal the actual JavaScript object through the wrapping performed by XPConnect.</p>
<p>For further details on state-saving mechanisms provided by httpd.js, see {{ Source("netwerk/test/httpserver/nsIHttpServer.idl") }} and the <code>nsIHttpServer.get(Shared|Object)?State</code> methods.</p>
<h4 name="How_do_I_change_the_HTTP_headers_or_status_sent_with_a_file_used_in_a_Mochitest.3F">How do I get access to the files on the server as XPCOM objects from an SJS script? (1.9.3 or later)</h4>
<p>If you need access to a file, say, because it's easier to store image data in a file than directly in an SJS script, use the presupplied <code>SERVER_ROOT</code> object state available to SJS scripts running in Mochitest:</p>
<pre class="eval">function handleRequest(req, res)
{
  var file;
  getObjectState("SERVER_ROOT", function(serverRoot)
  {
    file = serverRoot.getFile("tests/content/media/test/320x240.ogv");
  });

  // file is now an XPCOM object referring to the given file
  res.write("file: " + file);
}
</pre>
<p>The path you specify is used as a path relative to the root directory served by httpd.js , and an <code>nsIFile</code> corresponding to the file at that location is returned.  Beware of typos: the file you specify doesn't actually have to exist because file objects are mere encapsulations of string paths.</p>
<p>{{ languages( { "ja": "ja/Mochitest" } ) }}</p>
Revert to this revision