python runtests.py --help
See Running Tests for examples of running tests. The examples refer to mach, but you can use the same options with runtests.py (
mach marionette-test just calls runtests.py)
Running tests against an emulator or device
To have Marionette launch an emulator for you and run tests on that emulator, you specify the
--emulator option. The emulator will be closed when the tests are done.
To have Marionette connect to an actual device where a browser is running, you
ADDRESS is the host:port that Marionette can connect to. Marionette runs on port 2828 by default, but you'll have to setup port forwarding using adb so that the Marionette client can connect to port 2828 on the device.
adb forward tcp:2828 tcp:2828
To have Marionette connect to an emulator which you've already launched and then launch a browser instance for you, you specify both the
Note that, currently, Marionette is available on Fennec debug builds.
Running tests against desktop builds
To have Marionette launch a desktop application such as Firefox, you specify the
--binary option. The application will be closed when the tests are done. If you want to use a specific profile you can also specify the
If you are testing against Fennec you should also specify the
--app=fennec option to ensure the correct preferences are set.
To have Marionette connect to a desktop application which you've already launched, you must specify the
Specifying tests to run
To specify tests to run, you can specify one or more individual files, one or more directories, or a manifest file.
When specifying directories, Marionette will attempt to run all files contained therein beginning with
browser_ and ending with
When specifying a manifest file, you can also specify a
--type=TYPE option. This controls what type of tests from the manifest are run.
TYPE can be any combination of values listed in the manifest file, combined with + or - symbols. For instance, to run tests which are compatible with both the browser and b2g, you could specify
--type=browser+b2g. To specify tests which are compatible with b2g and do not require an emulator, you could specify
--testvars option allows you to specify a path to a JSON file with variables that can be used by your tests.
To run a specific test on a device which has local port 2828 forwarded to port 2828 on the device, use:
python runtests.py --address=localhost:2828 /path/to/test