mozilla
Your Search Results

    Mozmill commit policy

    The following details our policy for committing patches to the mozmill-tests and litmus-data repositories. If you have any questions or need help email Anthony Hughes or ping ashughes on irc.mozilla.org/#automation.

    What is a committer?

    A committer is someone who physically checks code into the repository. This is a privilege which comes with a lot of responsibility and is typically only granted to those who have proven themselves over an extended period of contribution.

    How do I become a committer?

    When the team feels you are ready to become a committer, you will need to go through the application process. Mozilla has a standard Committer Agreement with everyone who wishes to check code into any repository. If you want to be able to land patches to the mozmill-tests and litmus-data repositories, get in touch with Anthony Hughes.

    Privileges

    Once granted commit access you will be granted the following privileges:

    • Landing patches to skip/disable a failing test (patch must be approved by one peer)
    • Landing patches to fix a failing test (patch must be approved by one peer and one super-reviewer)
    • Landing patches to add a new test (patch must be approved by one peer and one super-reviewer)
    • Landing patches to litmus-data (patch must be approved by one peer and one super-reviewer)
    • Landing patches to endurance tests (patch must be approved by one peer, one super-reviewer, and the module owner)
    • Landing patches to shared modules (patch must be approved by one peer, one super-reviewer, and the module owner)

    Responsibilities

    Once granted commit access you will be granted the following responsibilities:

    • Landing your own patches after they have passed review (ideally within 24 hours of r+)
    • Checking the live test results the following day to see if your patch introduced a regression
    • Promptly backing out your patch when a regression is introduced
    • Landing patches for those without commit access after they have passed review
    Note: If you land a bad patch, it is your responsibility to back it out. If you write a bad patch, it is your responsibility to fix it.

    Life-cycle of a patch

    Patches for Endurance Tests

    • Land the patch on the default branch
    • Update the bug with the following changes
      • [landed] added to the patch summary
      • comment with the URL to the changeset
      • status set to RESOLVED FIXED
    • The next day, verify the test has not caused regressions by checking the Endurance Dashboard for failures
      • if no failures are found, set the bug status to VERIFIED FIXED
      • if the test fails, the bug status is set to REOPENED and the patch is backed out

    Patches for Functional Tests

    • Land the patch on the default branch
    • Update the bug with the following changes
      • [landed:default] added to the patch summary
      • comment with the URL to the changeset
      • status set to RESOLVED FIXED
    • The next day, verify the test has not caused regressions by checking the Functional Dashboard for failures
    • If no failures are found, land the patch on the mozilla-aurora, mozilla-beta, and mozilla-release branches, and update the bug with the following changes:
      • change [landed:default] to [landed] in the patch summary
      • comment with the URLs to the changesets for each of the branches
      • change the status to VERIFIED FIXED
    • The next day, verify the test has not caused regressions by checking the Functional Dashboard for failures
      • if the test fails, the bug status is set to REOPENED and the patch is backed out
    NOTE: Only land your patch on branches where the feature exists. For example, if your test is for a feature which is only active on Nightly, land the patch on default; if your test is for a feature which is active on Beta, land your patch on default, mozilla-aurora, and mozilla-beta; and so on.
    NOTE: Be sure to update the corresponding Litmus test with a link the the Mozmill test. Also, clone the Litmus test to the Aurora Mozmill Tests subgroup.

    Patches for Test Failures

    • Land the patch on the affected branch
    • Update the bug with the following changes
      • [landed] added to the patch summary
      • comment with the URL to the changeset
      • status set to RESOLVED FIXED
    • The next day, verify the test has not caused regressions by checking the Functional Dashboard for failures
    • If the test is still failing, set the bug status to REOPENED and back out the patch
    • Otherwise, change the status of the bug to VERIFIED FIXED
    NOTE: Any failures which only affect the mozilla-1.9.2 branch will be disabled, the bug resolved WONTFIX, otherwise follow this process:
         1. Disable the test
         2. Investigate if the bug is a failure of the test or Firefox
         3. If it's a problem with the test, the test will remain disabled until it can be fixed
         4. If it's a problem with Firefox, file a Firefox bug, re-enable the test and close the test failure bug
    NOTE: Any disabled test should also be disabled, any fixed test re-enabled, in Mozmill Tests subgroup in Litmus

    Litmus

    After you have landed a new test, or disabled a failing one, you need to update Litmus.

    • If the test has been disabled
      • find the Mozmill Tests subgroup for each branch in Litmus
      • remove the test from the subgroup
      • amend the Mozmill Test block to include the word DISABLED and strike-through the link to the test file
      • Remember to revert this change when the test is being re-enabled
    • If the test is new
      • find the Litmus testcase corresponding to your Mozmill test
      • add a comment to the test stating it is covered by Mozmill and provide a link to the file
      • add the Litmus test to the Mozmill Tests subgroup for that branch
      • This needs to be done for each and every branch the test has landed

    Nomenclature for patch landing

    When you land a patch you should do the following in Bugzilla:

    • Edit Details for the attached patch
      • If the patch is landed on all branches, add [landed] to the attachment summary
      • If the patch is landed on a single branch, add [landed:branch_name] to the attachment summary
    • Add a comment to the bug indicating where the patch was landed and provide a link to the changeset

    Example:

    Comment on attachment 588432 [details] [diff] [review]
    test v7 [landed]
    
    Landed:
    http://hg.mozilla.org/qa/mozmill-tests/rev/829eb7f3fc87 (mozilla-aurora)
    http://hg.mozilla.org/qa/mozmill-tests/rev/49cea842da76 (mozilla-beta)
    http://hg.mozilla.org/qa/mozmill-tests/rev/819e3aa37289 (mozilla-release)
    

    Non-responsibilities

    The following are not your responsibility as a committer and should never be done unless given explicit permission. These will be take care of by the module owners.

    • Landing patches for new tests on the mozilla-1.9.2 branch
    • Landing patches on the mozilla-esrN branch
    • Merging branches

    Document Tags and Contributors

    Contributors to this page: Sheppy, Ashughes
    Last updated by: Sheppy,