mozilla

Revision 85696 of Measuring Add-on Startup Performance

  • Revision slug: User:Jorge.villalobos/Testing_Add-on_Startup_Performance
  • Revision title: Measuring Add-on Startup Performance
  • Revision id: 85696
  • Created:
  • Creator: Jorge.villalobos
  • Is current revision? No
  • Comment no changes

Revision Content

Using Talos

Talos is a performance testing framework written in Python. It is used to test Firefox performance in general, and has been adapted to test add-on startup performance as well. The Talos system is currently in use on AMO to calculate the startup performance of add-ons hosted there and highlight add-ons with high overhead numbers.

Prerequisites

The StandaloneTalos page explains how to download and get Talos up and running. It is not necessary to download or install a nightly build of Firefox if you're planning on testing your add-on's startup performance only on stable Firefox builds.

Set Up

  1. You'll need 2 config files: one for your baseline numbers with no add-ons installed, and one to run with your add-on. Create a copy of addon.config and name it baseline.config.
  2. Make sure both files point to the right path of the Firefox executable in the browser_path variable. On Mac OS you need to point to the internal binary (/Applications/Firefox.app/Contents/MacOS/firefox-bin).
  3. To avoid having to close your main Firefox instance while testing, you should add set the extra_args variable to '-no-remote' on both files.
  4. In addon.config, add any preferences and their corresponding values to the preferences variable if they are necessary to reliably test your add-on's startup performance.
  5. In addon.config, add a path to your XPI file to the extensions variable. If your add-on needs to be unzipped in the profile folder in order to work, then the path should point to the root directory of the unzipped package.
  6. For both files, you can remove or comment out all tests in the tests section, except for ts and ts_paint.

Running the tests

Interpreting the results

Using about:startup

Revision Source

<h2>Using Talos</h2>
<p><a class=" link-https" href="https://wiki.mozilla.org/Buildbot/Talos" title="https://wiki.mozilla.org/Buildbot/Talos">Talos</a> is a performance testing framework written in Python. It is used to test Firefox performance in general, and has been adapted to test add-on startup performance as well. The Talos system is currently in use on AMO to calculate the startup performance of add-ons hosted there and highlight <a class=" link-https" href="https://addons.mozilla.org/en-US/firefox/performance/" title="https://addons.mozilla.org/en-US/firefox/performance/">add-ons with high overhead</a> numbers.</p>
<h3>Prerequisites</h3>
<p>The <a class=" link-https" href="https://wiki.mozilla.org/StandaloneTalos#How_to_set_up_Talos_for_testing_at_home" title="https://wiki.mozilla.org/StandaloneTalos#How_to_set_up_Talos_for_testing_at_home">StandaloneTalos page</a> explains how to download and get Talos up and running. It is not necessary to download or install a nightly build of Firefox if you're planning on testing your add-on's startup performance only on stable Firefox builds.</p>
<h3>Set Up</h3>
<ol> <li>You'll need 2 config files: one for your baseline numbers with no add-ons installed, and one to run with your add-on. Create a copy of addon.config and name it baseline.config.</li> <li>Make sure both files point to the right path of the Firefox executable in the <em>browser_path</em> variable. On Mac OS you need to point to the internal binary (/Applications/Firefox.app/Contents/MacOS/firefox-bin).</li> <li>To avoid having to close your main Firefox instance while testing, you should add set the <em>extra_args</em> variable to '-no-remote' on both files.</li> <li>In addon.config, add any preferences and their corresponding values to the <em>preferences</em> variable if they are necessary to reliably test your add-on's startup performance.</li> <li>In addon.config, add a path to your XPI file to the <em>extensions</em> variable. If your add-on needs to be unzipped in the profile folder in order to work, then the path should point to the root directory of the unzipped package.</li> <li>For both files, you can remove or comment out all tests in the <em>tests</em> section, except for <em>ts</em> and <em>ts_paint</em>.</li>
</ol>
<h3>Running the tests</h3>
<h3>Interpreting the results</h3><h2>Using about:startup</h2>
Revert to this revision