Out of memory management on Firefox OS

Firefox OS runs on some severely memory-constrained devices, and it's easy for apps to exhaust the memory available on such systems.  When a process exhausts the memory available on the system, the kernel must kill some other processes in order to free up memory. This article describes how the low memory killer and low memory notifications work — the two systems on the device that control what processes are killed to keep the main system running when it has run out of memory.

Firefox OS operation involves multiple processes — one "main process" running basic system services, and potentially many "child processes". In general every app runs in its own child process. Since in the Firefox OS environment applications are rarely closed by the user the system automatically manages their lifetime to make room for new apps or existing apps requiring additional memory.

Two subsystems are used to manage this: the Low memory killer (LMK) and Low memory notifications.

Low memory killer

The LMK is a subsystem of the Android kernel that automatically kills processes to make room for memory requests. In order to choose which process is killed first for freeing up memory, each process is assigned a priority via the /proc/<pid>/oom_adj or /proc/<pid>/oom_score_adj files. A process's priority is known as its adjustment score, or oom_adj.  Smaller oom_adj values correspond to higher priority processes.

In general, the larger the adjustment score is, the more likely a process is to be killed. The LMK provides multiple levels, each corresponding to a certain amount of free memory and a minimum adjustment score. Whenever the amount of free memory in the system drops below a certain threshold level, all processes with an adjustment score higher than the minimum specified for that level are eligible to be killed. The LMK will start killing these processes, larger ones first, and keep going until it has freed enough memory to go above the threshold again.

Note: When a background app is killed by the LMK, it is made available in the task manager/through edge swipes as a "zombie app": next time you browse to that app, it will be revived. The maximum number of apps that can be kept in this state is currently 10.

Note: The process killed when the device runs out of memory isn't necessarily the one that "caused" the out-of-memory condition.

Process priorities

In Firefox OS apps are killed in the following priority order policy, which is enforced by giving each application a priority level and associating an OOM adjustment score to these levels (the current values are set in prefs):

  1. The first apps to be killed will be the background apps, starting with the least recently used.
  2. Background applications that are perceivable by the user are killed next (for example, a music player playing audio in the background or an app holding a high-priority or cpu wakelock and having a registered handler for system messages.)
  3. If the keyboard application is running, it will be killed next.
  4. Foreground applications will be killed next.
  5. Finally, foreground applications that have requested high-priority or cpu wakelocks are the last to get killed.

Note: Most child processes run with oom_adj 2 while they're in the foreground.  Child processes in the background run with oom_adj between 3 and 6 (inclusive).  Exactly what oom_adj a child process has while in the background depends on a number of factors, such as whether it's playing sound, whether it's the keyboard app, etc.

There's a couple of exceptions to these rules:

  • The main process is never killed by the LMK as doing so would kill all child processes and restart the operating system. The main process runs with oom_adj 0.
  • We keep a process around to speed up the startup of new applications called the preallocated process. This process is usually always kept alive because it consumes very little memory and significantly speeds up application startup. The only case under which it can be killed is if there's not enough memory available for the main process to keep running after having killed all other processes.

Low memory notifications

The second mechanism we use to free memory is low memory notifications. The LMK provides a special threshold that, when crossed, can send notifications to the userspace that is running low on memory. Both the system application and regular user apps continuously wait for this condition and will react upon it by sending a memory-pressure event via the observer service. This event is visible only to C++ and chrome JS code and not directly by an application. Through the Gecko codebase we use this event to free as much memory as we can — normally by purging internal caches (images, DNS, sqlite, etc.), dropping assets that can be recreated (WebGL contexts for example) and running the garbage collector and cycle collector.

When encountering a low memory condition the first memory-pressure event that will be sent will have the low-memory payload. If after a predefined time (5s) we're still in a low memory condition another memory-pressure event will be fired, but this time with the low-memory-ongoing payload. This payload is used when we continue to experience low-memory conditions and we want to flush caches and do other cheap forms of memory minimization, but know that heavy handed approaches like a GC are unlikely to succeed.

How the LMK and low memory notifications work together

Currently two low memory thresholds are used, a soft and a hard one. The soft level is set above the LMK level for background applications and is used to minimized memory usage when we start running out of memory but before applications get killed. The hard one on the other hand is used to keep alive foreground applications once all background ones have already been killed by the LMK. Since only one kernel trigger is available, the two levels are implemented by having Gecko dynamically adjust the trigger. The aggregated action of the LMK and low memory notifications is as follows when a device is running out of memory:

  1. Send memory-pressure events to all applications.
  2. If we didn't free enough memory, kill background apps in least-recently-used order.
  3. If we didn't free enough memory, send memory-pressure events to all remaining applications.
  4. If the condition persists resend a memory-pressure event every 5 seconds, but mark them as ongoing so the GC/CC doesn't react to them.
  5. Kill perceivable or high-priority background applications.
  6. Kill the keyboard app, if it is running.
  7. Kill foreground applications.
  8. Kill high priority foreground applications.
  9. Kill the preallocated process.

Document Tags and Contributors

 Contributors to this page: chrisdavidmills, gsvelto, jacksonf, pranay.kumar
 Last updated by: chrisdavidmills,