mozilla

Creating and applying Firefox OS update packages

If you want to allow Firefox OS users to easily update the version of the system code on their devices, you need to create an update package for them to use. This article goes through the different types of update package available and covers building the package, hosting the updates (and how the system polls for available updates), and applying and verifying those updates.

Creating and applying an update is split into four steps:

  1. Building incremental update packages from old version(s) to a new version on a build host.
  2. Finding the right update package to download on the client.
  3. Downloading the update.
  4. Applying the update to existing files on the device.

Each of these steps are covered below.

Note: There are a number of useful tools for building and testing Firefox OS system updates, available at b2g/tools/update-tools.

Prerequisites

To build and apply updates you must ensure that your build has the updater and associated update tools enabled. By default those are enabled only in userdebug and user variants. You can however force building them by adding the following line to your .userconfig file:

export B2G_UPDATER=1

To sign update packages you will need a Java runtime environment (JRE) or Java software development kit (JDK) installed and the java command available in the default path.

Types of update

There are two types of updates to know about: FOTA (Firmware Over-The-Air) and Gecko/Gaia OTA (Over-The-Air). Let's look at the differences between them.

FOTA updates

We can update the entire Firefox OS system through FOTA updates, the technology behind which is shared with the Android project. The locations on the phone's hard drive that can be changed using FOTA updates include the system partition, kernel, modem baseband, recovery image used for updating, or any other file on the device.

Firefox OS does not depend on a particular FOTA client; the interface is abstracted through an API we call librecovery. However, we recommend using the GOTA recovery client (see below for more details), and the discussion here assumes GOTA is being used.

FOTA update packages mainly consist of a file called update.zip. This package consists of

  • A set of binary diffs and new files required to update the client to the newer software version
  • An "update script" that controls how the diffs and new files are loaded onto the client
  • An internal signature used to verify the update package

This format and set of files are the same as those used in normal Android updates, except that Firefox OS additionally wraps the update.zip package in a mar wrapper (MAR stands for Mozilla ARchive). This mar wrapper allows an additional level of verification, which is explained below.

Gecko/Gaia OTA updates

Alteratively we can update just the Gecko and Gaia files on a Firefox OS device, through a mechanism we call Gecko/Gaia OTA updates. All of the Gecko and Gaia files — comprising the core Gecko runtime and the device's user interface — are in the /system/b2g directory on the device. This is the only directory that OTA updates can make changes to.

Gecko/Gaia OTA updates use the same technology that's used to update the Firefox desktop web browser. Much like the FOTA update.zip packages discussed above, OTA updates consist of a MAR file containing a set of binary diffs and new files needed to update the client to a newer software version.

The Gecko client verifies the integrity of MARs that it downloads, and MARs can be signed by multiple parties.

Why have two update technologies?

OTA updates are not as comprehensive as FOTA updates, but they are a lot more user friendly and easy to apply, and will often be fine for what you need to update:

  • Gecko/Gaia OTA updates can be applied "in the background", while Firefox OS continues to run normally. This provides a much better user experience because users don't need to reboot their phone and wait while an update is applied. Instead, the update is applied while the user continues to use the phone, and when the update is finished the user only needs to agree to restart the main b2g process. This takes a matter of seconds, instead of the minutes that are usually required to apply FOTA updates.
  • Gecko/Gaia OTA update packages can sometimes be smaller than FOTA update packages, though not always; they should never be larger. This means that users can sometimes have less data to download.

Of course, if you need to update files outside Gecko/Gaia, you will have to go for the full FOTA package route.

Let's move on and examine the package building process.

Building update packages

Building updates is the process of generating the files needed to update Firefox OS clients from version X of the software to a newer version Y. The update package that's needed to update the client depends on what files have changed between version X and version Y.

  • If only files in /system/b2g have changed, we will generate a Gecko/Gaia OTA update
  • If any files in a location outside of /system/b2g changed, we will generate a FOTA update

To generate an incremental update package (for both FOTA and Gecko/Gaia OTA updates), our tools require full builds of both version X and version Y. Full build means that the package includes all the files that are needed to Flash a client. When we produce a full build for version X, we don't know which future versions we will be updating to from version X . Because of that, we build both full FOTA packages and Gecko/Gaia packages for each version. This allows us to generate either a Gecko/Gaia OTA incremental update, or a FOTA incremental update if needed, between version X and all future versions.

At a high level, the process of building an update looks like this:

  1. With software version X
    • Generate a complete Gecko/Gaia OTA MAR for the contents of /system/b2g.
    • Generate a complete FOTA target files zip, optionally signing it, for the device partitions. The target files zip is referenced below as DEVICE-target_files-$VARIANT.$USER.zip, and is a zip containing the files to update the phone directories, including SYSTEM/, BOOT/, etc. A full FOTA update.zip can be generated from the target files zip.
  2. With software version Y
    • Generate a complete Gecko/Gaia OTA MAR for the contents of /system/b2g.
    • Generate a complete FOTA target files zip, optionally signing it, for the device partitions. The target files zip is referenced below as DEVICE-target_files-$VARIANT.$USER.zip, and is a zip containing the files to update the phone directories, including SYSTEM/, BOOT/, etc. A full FOTA update.zip can be generated from the target files zip.
  3. If only files in /system/b2g have changed, generate an incremental Gecko/Gaia OTA update MAR from version X to version Y.
  4. Otherwise, generate an incremental FOTA update.zip from version X to version Y. Wrap the incremental FOTA update.zip in a MAR for delivery to the B2G client.
  5. Sign the packages as required by delivery agreements.

The subsections below describe how to use B2G's tools to implement each of these steps.

Note: the steps below assume that you have already set up a b2g build environment at the location $b2g. The commands below reference the $b2g/build.sh helper script, but make can also be used.

Generating a complete Gecko/Gaia OTA update MAR

To generate a complete OTA update MAR from the last successful b2g build (e.g. that you built yourself), you need to invoke the gecko-update-full target. To place the MAR at $b2g/objdir-gecko/dist/b2g-update/b2g-gecko-update.mar, use the following commands:

$ cd $b2g
$ ./build.sh gecko-update-full
$ cp objdir-gecko/dist/b2g-update/b2g-gecko-update.mar <destination>

Generating a full FOTA update MAR

To generate a full FOTA update MAR from the last successful b2g build (e.g. that you built yourself), you need to invoke the gecko-update-fota-full target. This includes the contents of the entire /system partition. Here are the commands you need:

$ cd $b2g
$ ./build.sh gecko-update-fota-full

This will generate a ZIP file ($PRODUCT_OUT/fota/full/update.zip) and a MAR file ($PRODUCT_OUT/fota-$TARGET_DEVICE-update-full.mar). The ZIP file can be directly used with adb sideload, while the MAR is intended for distribution in the same manner as any other update package.

Generating a FOTA update MAR plus recovery package

As of Firefox OS 2.2 (mid April and beyond) we added a new make target, which can be invoked as follows:

$ cd $b2g
$ ./build.sh gecko-update-fota-fullimg

This is used to produce a recovery package that will dump a set of partitions images. The default set is controlled by the variable B2G_FOTA_FULLIMG_PARTS, defined in gonk-misc/Android.mk (along with most of the other new features seen below.) It's a space-separated string of mountpoint:image instances to include. The default value is "/boot:boot.img /system:system.img /recovery:recovery.img /cache:cache.img".

Along with this we have also introduced some new environment variables to control the production of the two other make targets — gecko-update-fota and gecko-update-fota-full:

  • The first is B2G_FOTA_PARTS, which follows the same syntax pattern as B2G_FOTA_FULLIMG_PARTS. This allows us to produce these update packages but arbitrarily dump partition images along with those, e.g. boot partition, modem firmware, etc.
  • B2G_FOTA_PARTS_FORMAT provides a way to describe a set of partitions that we want formatted during the installation of the recovery package. It's a space-separated list of mount points to make use of during the formatting.
  • We also have two new variables that allow us to wipe caches and/or data during the build procedure:
    • B2G_FOTA_WIPE_DATA
    • B2G_FOTA_WIPE_CACHE

Note: All of these new features heavily rely on having a proper recovery.fstab file provided for the device in question.

Generating a partial Gecko/Gaia FOTA update MAR

A partial FOTA update uses the same mechanism as a full FOTA update, but by default only includes Gecko/Gaia updates like a regular OTA update. Additional files outside of Gecko/Gaia (such as fonts) can also be included.

The rationale for generating a partial FOTA update package is mainly related to licensing issues: when building a complete FOTA update package, the whole system partition (at least) will be included. This may include blobs that you don't have the authorization to redistribute. However, since MAR distribution is useful and Gecko/Gaia themselves are free software, there is no reason we should not be able to distribute them in this manner. A partial FOTA allows you to only update a subset of the system. An OTA update could be used instead in this scenario but it does come at a cost: OTA updates require enough space on the system partition to hold both the existing Gecko/Gaia files as well as the unpacked update files. A partial FOTA update does not suffer from this limitation as it can overwrite the existing files with the updated ones.

To create a partial FOTA update from the last successful b2g build (e.g. that you built yourself), Invoke the gecko-update-fota target with the following commands:

$ cd $b2g
$ ./build.sh gecko-update-fota

This will generate a ZIP file ($PRODUCT_OUT/fota/partial/update.zip) and a MAR file ($PRODUCT_OUT/fota-$TARGET_DEVICE-update.mar). The ZIP file can be directly used with adb sideload, while the MAR is intended for distribution in the same manner as any other update package.

The construction can be controlled with a couple of environment variables, the most useful of which are documented below:

Variable Meaning
$B2G_FOTA_DIRS Space-separated list of directories to include in the update. Defaults to system/b2g.
$TARGET_UPDATE_BINARY Binary used to execute the Edify script inside the package. When none is provided, a pre-built updater binary from ICS is used.
$FOTA_FINGERPRINTS Comma-separated list of Android fingerprints to check against. The use case is to be able to distribute Gecko/Gaia update packages on top of a controlled Gonk base system that we cannot legally distribute. For example, Open C community builds are using this.

Note: A complete set of these variables is defined in the Android.mk file of the gonk-misc repository; note that $FOTA_FINGERPRINTS is used in our update_tools.py tool.

Generating a complete FOTA target files zip

Invoke the target-files-package target to build a target files zip that can be used to generate both incremental and full FOTA update packages. The target files zip can also be signed by custom keys to ensure that only FOTA updates from known sources can be installed. After signing target files, all images and updates (also OTA) need to be generated again to catch the inserted keys.

Note: The target files zip is generated in the location out/target/product/$DEVICE/obj/PACKAGING/target_files_intermediates/$DEVICE-target_files-$VARIANT.$USER.zip

The following commands will carry out this step:

$ cd $b2g
$ ./build.sh target-files-package
$ cp out/target/product/$DEVICE/obj/PACKAGING/target_files_intermediates/$DEVICE-target_files-$VARIANT.$USER.zip <destination>

The variable values in the commands listed above should be filled in as follows:

Variable Meaning
$DEVICE Device name for the AOSP product
$VARIANT eng, user, or userdebug
$USER The build username

Signing a complete FOTA target files zip

Proper releases should typically be signed by custom release keys only known to the vendor. Having such keys will prevent FOTA updates where the source is unknown from being installed, hence introducing an extra security layer. For this to work, the images flashed to a device need to include public keys while the updates need to be signed by the corresponding private key. 

The first step is to generate custom keys and store them in a safe place. The Android Open Source Project has a script for generating these keys. For full compatibility, get this script from the branch corresponding to the Gonk version of the device in question. Here is the master branch version.

A couple of keys are needed — create them with the following commands. releasekey is the key to use for signing FOTA update packages.

$ development/tools/make_key releasekey '/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com'
$ development/tools/make_key platform '/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com'
$ development/tools/make_key shared '/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com'
$ development/tools/make_key media '/C=US/ST=California/L=Mountain View/O=Android/OU=Android/CN=Android/emailAddress=android@android.com'

With keys present, the target files zip can be signed using the following commands. This will insert the public keys and modify build properties to reflect the fact that it has been signed.

$ cd $b2g
$ ./build/tools/releasetools/sign_target_files_apks \
  --default_key_mappings $RELEASEKEY_FOLDER \
  --replace_ota_keys \
  --signapk_path prebuilts/sdk/tools/lib/signapk.jar \
  $UNSIGNED_TARGET_FILES_ZIP \
  $SIGNED_TARGET_FILES_ZIP

The variable values in the commands listed above should be filled in as follows:

Variable Meaning
$RELEASEKEY_FOLDER The path to the folder containing the custom keys
$UNSIGNED_TARGET_FILES_ZIP The FOTA target files zip to sign.
$SIGNED_TARGET_FILES_ZIP The signed FOTA target files zip to be generated

Generating an incremental OTA update MAR

In this example, we're assuming that we're generating an update from software version X to version Y. The location of the full Gecko/Gaia OTA MAR built from software version X using the instructions above will be called $MAR_X below. This might be a path on a build server like /home/build/b2g/versions/X/update.mar. Similarly, the location of the full MAR built from version Y will be called $MAR_Y.

The tool build-gecko-mar.py will generate an incremental Gecko/Gaia OTA update MAR using $MAR_X and $MAR_Y. We'll call the destination of the generated file $GENERATED_INCREMENTAL_MAR_X_Y. Use the following commands for this step:

$ cd $b2g
$ ./tools/update-tools/build-gecko-mar.py --from $MAR_X --to $MAR_Y $GENERATED_INCREMENTAL_MAR_X_Y

Generating an incremental FOTA update zip

In this example, we're assuming that we're generating an update from software version X to version Y. The location of the full FOTA target zip built from software version X using the instructions above will be called $TARGET_FILES_X below. This might be a path on a build server like /home/build/b2g/versions/X/target_files.zip. Similarly, the location of the full FOTA target zip built from version Y will be called $TARGET_FILES_Y.

The tool build/tools/releasetools/ota_from_target_files will generate an incremental FOTA update.zip using $TARGET_FILES_X and $TARGET_FILES_Y. We'll call the destination of this intermediate file $INTERMEDIATE_FOTA_UPDATE_FOTA_X_Y.

After this update.zip is generated, the last step is to wrap it in a MAR for delivery to the B2G client. The tool tools/update-tools/build-fota-mar.p does this step. We'll call the destination of this generated file $GENERATED_INCREMENTAL_FOTA_X_Y.

Use the following commands to complete this step:

$ cd $b2g
$ ./build/tools/releasetools/ota_from_target_files -v \
    --incremental_from $TARGET_FILES_X \
    --signapk_path prebuilts/sdk/tools/lib/signapk.jar \
    --package_key $FOTA_SIGNING_KEY \
    $TARGET_FILES_Y \
    $INTERMEDIATE_FOTA_UPDATE_FOTA_X_Y
$ ./tools/update-tools/build-fota-mar.py $INTERMEDIATE_FOTA_UPDATE_FOTA_X_Y --output=$GENERATED_INCREMENTAL_FOTA_X_Y

The variable values in the commands listed above should be filled in as follows:

Variable Meaning
$TARGET_FILES_X The FOTA target files zip for version X
$TARGET_FILES_Y The FOTA target files zip for version Y
$INTERMEDIATE_FOTA_UPDATE_FOTA_X_Y A temporary update.zip to generate a MAR from
$GENERATED_INCREMENTAL_FOTA_X_Y The destination incremental update zip wrapped in a MAR for delivery to clients
$FOTA_SIGNING_KEY Path to the prefix for a private key and public cert for signing the update zip. $FOTA_SIGNING_ZIP.pk8 and $FOTA_SIGNING_ZIP.x509.pem should both exist on the file system. If $TARGET_FILES_X is not signed this option can be omitted; the default testkey will still be picked up. In case $TARGET_FILES_X is a custom release key, refer to the target files zip signing section on how to create it, and don't forget to sign $TARGET_FILES_Y.

Hosting updates and polling for updates on the client side

Firefox OS clients poll for updates by fetching and parsing an update manifest: update.xml. Firefox OS clients are configured to poll for updates on specific servers — they query a specially-constructed path on the server. HTTPS is the recommended protocol that the client uses to query the server, however HTTP is also supported. The server and path polled by clients can be changed by shipping an update to existing clients that changes the polling code.

In the examples below, we'll assume that updates are hosted on the server updates.b2g.com.

The URL polled by the client commonly contains the following parameters:

Parameter Explanation
PRODUCT_MODEL The name of the device model. This is the ro.product.model value in the B2G property database.
CHANNEL The update "channel". This is useful for testing: servers can be configured to host, for example, "nightly", "beta", and "release" channels.
VERSION The client's software version. For example, "18.0.2".
BUILD_ID A unique ID such as a timestamp, configured for a particular build.

The Firefox client uses the value of its configured update host and these values to construct a URL to poll at runtime. The structure is as follows:

https://aus4.mozilla.org/update/3/%PRODUCT%/%VERSION%/%BUILD_ID%/%PRODUCT_DEVICE%/%LOCALE%/%CHANNEL%/%OS_VERSION%/%DISTRIBUTION%/%DISTRIBUTION_VERSION%/update.xml

A real example of such a URL is as follows:

https://aus4.mozilla.org/update/3/B2G/37.0a1/20141214040212/flame/en-US/nightly-b2g37/Boot2Gecko%202.2.0.0-prerelease%20%28SDK%2019%29/default/default/update.xml?force=1

If the server returns a "404 Not Found" in response to the client's request, then there is no update available. If the server returns a "200" and a manifest file, then there may be an update available. The manifest describes the newly available build; that is, the build the client would update to. An example manifest is:

<?xml version="1.0"?>
<updates>
  <update type="major" appVersion="19.0" version="19.0" extensionVersion="19.0" buildID="20121210123456"
          licenseURL="http://www.mozilla.com/test/sample-eula.html"
          detailsURL="http://www.mozilla.com/test/sample-details.html">
    <patch type="partial" URL="https://updates.b2g.com/release/unagi1/18.0/20121203123456/update.mar"
           hashFunction="SHA512" hashValue="5111e033875752b7d9b32b4795152dea5ef954cb8a9d4a602dd19a923b464c43521287dcb5781faf3af76e6dc5e8a3dd9c13edea18c1f2c8f3bd89e17d103d6f"
           size="41901319"/>
  </update>
</updates>

This follows the same schema as the Firefox build manifest (see updates.xml Format for more details). The fields in the manifest describe:

  • Metadata used to show a user interface on the client.
  • Metadata about the newly-available version.
  • The location of the update package.
  • Metadata used to verify the download of the update package.

Note: There is a useful update script available at build-update-xml.py, which given a MAR file, builds a Firefox OS update.xml for testing.

Note: The client device or the user may wish to decline an update.

Using the mechanisms described above, servers can host update packages to update any old client version to the newest version. Or they may host only a "linear update history" in which clients must upgrade through a single path.

The details of the interaction between build servers and the update host is currently beyond the scope of this document. It is highly dependent on the production environment. You can find some more details on our Software Update wiki page.

Verifying and applying updates

After a Firefox OS client has successfully polled for an update (handled from within the system), downloaded it, and verified the integrity of the downloaded update package, the final step is to apply the update.

The first step in applying an update is to verify the signatures embedded in the MAR packages (see Generating an incremental FOTA update zip for how these are created). This is done by the Firefox OS client itself after checking the integrity of the downloaded package. The code used for this is the same for both FOTA and Gecko/Gaia OTA updates.

Note: It is not the MAR file that gets signed: it's the FOTA zip file that gets bundled into the MAR that's signed by build/tools/releasetools/ota_from_target_file. The signing of the FOTA update works the same as it does on Android; if you just run the script without specifying the key, it will use the developer key at build/target/product/security/testkeys.*. This is ok for testing but when you create a real update you need a secure key — i.e. one that no-one else knows about. The device will also verify that signature before applying the patch, so a device's initial images will need to contain the key as well.

Note: The keys referred to above are found in the Android build systems; we've forked it in our platform_build repo.

After signatures are verified, the process of applying an update diverges between Gecko/Gaia OTA updates and FOTA updates. Let's look at the differences between the two at this point.

Applying Gecko/Gaia OTA updates

The Firefox OS client applies these using the updater binary. This is part of the Gecko distribution and is the same code used to apply updates to desktop Firefox. As described above, the update is applied while the Firefox OS client continues to run normally. Users are able to make and receive calls, run apps, browse the web, etc. while updates are being applied.

The specific details of the updater binary are beyond the scope of this document, but it works approximately like so:

  • It makes a copy of the /system/b2g files.
  • It applies binary patches, removes old files, and adds new ones as specified by the MAR file.
  • It restarts the main b2g process so that it uses all the new files.

After the b2g process finishes restarting, the user will be running the new version of the B2G client software.

Applying FOTA updates

The FOTA client applies these. The Gecko client "hands off" the update to be applied by calling into the librecovery API. What happens after this step is specific to each FOTA client.

In the implementation of librecovery used for the GOTA client, the downloaded update package is staged to be applied and special commands are enqueued for the recovery client. librecovery then reboots the device into recovery mode. The recovery client then runs the update script in the update.zip to update files and partitions as needed. The recovery client may need to reboot multiple times in order to update all files.

After the final reboot, the device will be running the new version of the Firefox OS client software.

Document Tags and Contributors

Last updated by: chrisdavidmills,
Hide Sidebar