Thunderbird Automated Testing

This is an archived page. It's not actively maintained.

Automated testing is increasingly becoming a significant and important part of Thunderbird's development. It allows us to quickly pick up regressions in patches and helps the developer to think about different cases.

This page and its sub-pages are intended to describe and provide pointers to the available test mechanisms within Thunderbird and supporting information for developers and testers.

Core Tests expanded for Thunderbird

xpcshell-tests and make check

These tests are run against almost every changeset that gets committed to the Thunderbird code base.

  • xpcshell-tests run in javascript without any chrome present. This makes them especially useful for testing at the component/module level. xpcshell-tests are the quickest tests to run. MailNews provides additional facilities as detailed on the MailNews automated testing page.
  • "make check" is used to run compiled code tests, these are tests for when xpcshell-tests isn't quite sufficient (e.g. need access to c++ code), and other options don't test at the right level.

Thunderbird specific tests harnesses

Thunderbird is currently working on supporting MozMill as a test harness within its source base. Mozmill is designed as a harness that will drive an application at its UI level, putting in button and key presses just like a normal user would.

Other Core Tests

There are other core tests that are not automatically run on Thunderbird for the reasons detailed below:

  • Mochitest, Chrome and Browser Chrome tests
    • These require a browser window to be run, as Thunderbird doesn't have this, these tests cannot currently be run.
  • Reftest, crashtest
    • These tests can be run on Thunderbird. However, as they are specific to graphical layout they are just testing the core Gecko, and therefore the Thunderbird code won't have a direct effect on them. Therefore we leave running these tests to the Firefox tinderboxes.

Useful References