Talos is a performance testing framework written in Python. It is used to test Firefox performance in general, and has been adapted to test add-on startup performance as well. The Talos system is currently in use on AMO to calculate the startup performance of add-ons hosted there and highlight add-ons with high overhead numbers.
The StandaloneTalos page explains how to download and get Talos up and running. It is not necessary to download or install a nightly build of Firefox if you're planning on testing your add-on's startup performance only on stable Firefox builds.
- You'll need 2 config files: one for your baseline numbers with no add-ons installed, and one to run with your add-on. Create a copy of addon.config and name it baseline.config.
- Make sure both files point to the right path of the Firefox executable in the browser_path variable. On Mac OS you need to point to the internal binary (/Applications/Firefox.app/Contents/MacOS/firefox-bin).
- To avoid having to close your main Firefox instance while testing, you should add set the extra_args variable to '-no-remote' on both files.
- In addon.config, add any preferences and their corresponding values to the preferences variable if they are necessary to reliably test your add-on's startup performance.
- In addon.config, add a path to your XPI file to the extensions variable. If your add-on needs to be unzipped in the profile folder in order to work, then the path should point to the root directory of the unzipped package.
- For both files, you can remove or comment out all tests in the tests section, except for ts and ts_paint.