Tests can be run in-tree with a locally built Firefox through mach, as well as with a downloaded Firefox binary against in- and out-of-tree tests with runtests.py.
Running in-tree tests, mach will automatically manage the Python virtual environment in which your tests are run. The Marionette client that is picked up is the one that is in-tree at testing/marionette/client.
If you want to run tests from a downloaded tests.zip file, you will first need to set up the Python Marionette client. Second, you will need to download the tests from the tests.zip package found attached to Treeherder jobs for your system. If you are using a tests.zip package, extract it, and find the tests under marionette/tests/testing/marionette/client/marionette/tests.
You can launch the tests using one of the methods below. If desired, you can replace the path to unit-tests.ini with the path to a particular test. See the Marionette Python TestRunner for more options.
For Desktop Firefox
With a local build
You can use mach (German for “do”) to run all the tests:
% ./mach marionette-test
Individual tests can be run by specifying the file name:
% ./mach marionette-test testing/marionette/harness/marionette/tests/unit/test_findelement.py
You can see what extra options are provided using the -h flag:
If you are developing the Marionette server component, it is often useful to redirect Gecko’s standard output to your shell’s, as such:
% ./mach marionette-test TEST --gecko-log -
Similarily the --jsdebugger flag is occasionally useful in order to attach a Developer Tools Browser Debugger to the Marionette session.
With a downloaded desktop Firefox build and tests.zip
Activate the virtualenv that you installed Marionette in (if you used one), then:
cd /path/to/extracted/tests.zip/marionette/marionette python runtests.py --binary=/path/to/firefox-executable --type=browser ../tests/testing/marionette/client/marionette/tests/unit-tests.ini
For Firefox OS
With a device build
Activate the virtualenv that you installed Marionette in (if you used one), make sure your device is connected via adb, then:
adb forward tcp:2828 tcp:2828 cd testing/marionette/client/marionette python runtests.py --address localhost:2828 --type=b2g tests/unit/unit-tests.ini
With an emulator build
Activate the virtualenv that you installed Marionette it (if you used one), make sure your device is connected via adb, then:
cd testing/marionette/client/marionette python runtests.py --emulator arm --homedir $B2G_DIR --type=b2g tests/unit/unit-tests.ini
B2G_DIR is either the path to your B2G directory (if you've built your own emulator), or the path to a downloaded and extracted emulator package.
With a B2G desktop build
The only tests that are supported right now for B2G Desktop build are gaia-ui-tests. See that page for instructions. However, do note that these tests are being phased out in favour of Node.js-backed Gaia integration tests.