When you have finished making a change to the Gaia codebase and it seems to work ok, the next step is to go through a testing procedure to make sure your changes really work — and are working ok with the rest of Gaia — before submitting a patch back to the project. This article explains how.
The testing procedure generally consists of:
- Standard debugging procedure
- Running automated tests
Let's examine both of these areas now.
If your are an experienced web developer then debugging Gaia code should be a familiar process. We already discussed how to run Gaia in your Desktop Firefox installation, and how to make a basic change. For more complex code base additions, you'll want to make a lot more use of the Firefox debugging tools accessible in Desktop Firefox view.
Note: More instructions for using all these tools can be found in our Tools zone.
You should also run the standard automated test suites that code with Gaia before submitting a patch, to make sure that your code changes do not adversely affect any of the essential existing phone functionality. The tests you can run are:
- unit tests
- integration tests
- performance tests
- UI tests
We generally ask that you run the tests before submitting a patch; if it is your first contribution then you can submit without tests, but you need to ask for help with running the tests in future. You should update the Gaia repo before running tests, to make sure you have the latest ones.
Shortcut: If you don't know how to setup the testing environment, try MozITP — this toolset automatically sets up an Ubuntu VM and testing environment allowing you to run Integation tests and UI tests with little more than a single click.
Note: You can find more information on how these tests work at the Firefox OS Automated testing page.
Note: You should consider running each set of tests on a real device if available (some features/hardware aren't supported on an emulator), or on the B2G Desktop emulator or Firefox Nightly if not.
Unit tests are tests on individual units of code in a larger application — in the case of Gaia, individual apps. Gaia uses:
- mocha for its test framework
- chai as an assert library
- sinon.js for a mock & stub library
- blanket.js as a test coverage tool
You can run the following command to download, install and host a unittest server (running it takes a good few minutes, so this might be a good time to make a cup of tea):
DEBUG=1 make export FIREFOX=/Applications/FirefoxNightly.app/Contents/MacOS/firefox bin/gaia-test
This will open a local web page with a list of unit tests. To run them:
- Select the tests you want to run from the list in the page (an asterisk appears beside them).
- Click the "Execute" button.
- Scroll to the bottom of the page to see the results.
With the window open, you can also run the full test suite from a new terminal window:
Note: This can take quite a long time, as there are a lot of tests to run (possibly an hour or more), therefore you’ll probably just want to run the tests for the app you’ve modified. You can do this by appending
APP=<app folder name> onto the command, for example
Note: You can also read Gaia Unit Tests for more unit test information.
You can run the following command to trigger the integration tests:
Note: As with the unit tests, running the whole integration test suite can be very time consuming, so you can append
APP=<app folder name> onto the above command to test a single app, for example
Note: If you want to run integration tests on different (non-phone) device types, you should append
GAIA_DEVICE_TYPE=<device type name> onto the above command to signify the device type, for example
GAIA_DEVICE_TYPE=tv. If you don't do this, the tests may fail.
Note: for more on integration tests, read Gaia integration tests.
Gaia performance tests are run with the external Raptor CLI tool. With Raptor you can automate launching an application several times, and it will output statistics about the app's launch time. After running a test, Raptor also displays memory information about the application.
To run Raptor's performance tests, please visit Raptor: Getting Started.