This article is in need of a technical review.
As part of the Gaia/B2G source code, we have made various unit tests available to run, for testing different aspects of Gaia and B2G. This article explains how to access these.
Running unit tests
Note: If the following commands fail with cryptic errors while installing test-agent dependencies, it might be due to your Node.js/NPM versions being too old. Read Installing Node.js via package manager in order to install the very latest versions and delete the
In the Gaia repo comes the handy
bin/gaia-test script, that helps running the tests in an easy way.
Be careful: this script will generate a profile suited for unit tests in
profile-debug. If you already have another profile in this directory it will be overwritten. Bug 980496 aims to make this configurable.
Launching the test runner in Firefox
This will run the test server and launch your default Firefox as found in the path:
Choosing Firefox binary
You can export the FIREFOX environment variable to your firefox binary. For example, on MacOS X:
Alternatively, you can pass it as argument to bin/gaia-test:
bin/gaia-test <gaia directory> <firefox path>
Launching the test runner in B2G Desktop
This will download and launch B2G Desktop:
Run the tests from the Web Interface
You can simply click on specific tests and then the Execute button.
Run the tests from the command line
With the WebSocket server running, and the Test Agent app running in B2G Desktop/Firefox Nightly, run the following command:
If you only want to run one app's tests you can specify which via the
APP env variable:
make test-agent-test APP=calendar
You can also optionally provide a reporter to format the test output:
make REPORTER=List test-agent-test
Run the tests as you save
When the server is running, the tests for a file are run automatically when a file is saved or even just touched:
- When you save a test file, the test file itself is run
- When you save another file, it finds a matching file in the
test/unitdirectory, by suffixing the file name with
Note: It watches only existing files so if you create a new file, you have to restart the agent.
Running tests like TBPL does
Gaia unit tests in TBPL are run using a separate runner; this explains how to use it. Please consult the virtualenv docs if you're not familiar with using a Python virtualenv.
virtualenv venv source venv/bin/activate cd $GAIA/tests/python/gaia-unit-tests python setup.py develop cd gaia_unit_test python main.py --binary /path/to/b2g/desktop/build --profile /path/to/gaia/profile
Note: When specifying the path to the B2G desktop build, you should specify the path to
b2g-bin, if it exists, otherwise use
The Gaia profile must be made using the following:
NO_LOCK_SCREEN=1 DEBUG=1 DESKTOP=0 make
By default, this profile will be generated in
bin/gaia-test generates the same profile so you don't need to regenerate it if you already run
Disabling a gaia unit test in TBPL
TBPL uses a blacklist to exclude certain gaia unit tests from being run. To prevent a test from running in TBPL, add its path to https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-unit-tests/gaia_unit_test/disabled.json.
Setting up your Gaia app
Although this guide should help make things easier, the best way to learn how to write, set up, and run tests is currently still to look at the source code; in particular, take a look at the gallery tests.
Advanced: what does the script do?
Generating a profile
You need a profile that is generated by this command:
NO_LOCK_SCREEN=1 DEBUG=1 DESKTOP=0 make
This generates a debug profile in
gaia/profile-debug, overriding a previous profile if you already have one.
DEBUG=1enables the httpd.js extension that makes it possible to directly use the files from the
NO_LOCK_SCREEN=1disables the lock screen, which is necessary in B2G Desktop because it's not possible to unlock it using the mouse.
DESKTOP=0disables the other addons we normally use in DEBUG mode to run Gaia in Firefox.
Launch the WebSocket server
Test agent (the test runner) ships with a built in WebSocket server that lets you remotely message the browser or device to queue a test run. Often you will want to develop with time saving features like a file watcher that will run your tests when a test file or implementation changes. To take advantage of these features you need to start the server:
Using the WebSocket server provides other tools such as a command line reporter for test results (watch the terminal you ran the command from), a Growl reporter, syntax error notifications, and more.
The agent also watches for modifications in files, and automatically runs the associated tests. It runs when you save the test or if you save the tested file (we use the convention where the test filename is the tested filename with
Running unit tests with Firefox Nightly
You can launch Gaia in Firefox Nightly with the following commands:
cd <path to gaia> <path to nightly>/firefox --no-remote -profile <path to gaia>/profile-debug/ http://test-agent.gaiamobile.org:8080/
Note: In Mac OSX, The profile path should be absolute path
You can use Firebug or the integrated debugger to debug the tests; use the
debugger keyword to break in the debugger.
Running unit tests with B2G Desktop
Launch Gaia and start the "Test Agent" app. From the Test gent app you can select tests to run from the UI.
Advanced: how does the test-agent work?
The Test Agent lives in its own Github repository. You can have look there to understand how it works under the hood.