Browser storage limits and eviction criteria

Существует целый ряд веб-технологий, которые хранят данные того или иного рода на стороне клиента (то есть на локальном диске). Процесс, с помощью которого браузер решает, сколько места выделить для хранения веб-данных и что удалить, когда этот предел будет достигнут, не прост и отличается между браузерами. В этой статье описывается, как браузеры определяют, какой локальный контент следует очистить и когда, чтобы освободить необходимое локальное пространство для хранения.

Какие технологии используют браузерные хранилища данных?

В Firefox следующие технологии используют хранилище данных браузера для хранения данных, когда это необходимо. В этом контексте мы называем их "квотными клиентами":

The "last access time" of origins is updated when any of these are activated/deactivated — origin eviction will delete data for all these quota clients.

In Chrome/Opera, the Quota Management API handles quota management for AppCache, IndexedDB, WebSQL, and File System API.

Different types of data storage

Even in the same browser, using the same storage method, there are different classes of data storage to understand. This section discusses the different ones you might find in different browsers.

Storage comes in two types:

  • Persistent: This is data that is intended to be kept around for a long time. This will only be evicted if the user chooses to (for example, in Firefox you can choose to delete all stored data or only stored data from selected origins by going to Preferences and using the options under Privacy & Security > Cookies & Site Data).
  • Temporary: This is data that doesn't need to persist for a very long time. This will be evicted under a least recently used (LRU policy) when Storage limits are reached.

In Firefox, when persistent storage is used, the user is given a UI popup to alert them that this data will persist, and asks if they are happy with that. Temporary data storage does not elicit any user prompts.

Storage is temporary by default; developers can choose to use persistent storage for their sites using the StorageManager.persist() method available in the Storage API.

Where is the data stored?

Each storage type represents a separate repository. Here's the actual mapping to directories under a user's Firefox profile (other browsers may differ slightly):

  • <profile>/storage — the main top-level directory for storages maintained by the quota manager (see below)
  • <profile>/storage/permanent — persistent data storage repository
  • <profile>/storage/temporary — temporary data storage repository
  • <profile>/storage/default — default data storage repository

Storage limits

The maximum browser storage space is dynamic — it is based on your hard drive size. The global limit is calculated as 50% of free disk space. In Firefox, an internal browser tool called the Quota Manager keeps track of how much disk space each origin is using up, and deletes data if necessary.

So if your hard drive is 500 GB, then the total storage for a browser is 250 GB. If this is exceeded, a process called origin eviction comes into play, deleting an entire origin's worth of data until the storage amount goes under the limit again. There is no trimming effect put in place to delete parts of origins — deleting one database of an origin could cause problems with inconsistency.

There's also another limit called group limit — this is defined as 20% of the global limit, but it has a minimum of 10 MB and a maximum of 2 GB. Each origin is part of a group (group of origins). There's one group for each eTLD+1 domain. For example:

  • mozilla.org — group1, origin1
  • www.mozilla.org — group1, origin2
  • joe.blogs.mozilla.org — group1, origin3
  • firefox.com — group2, origin4

In this group, mozilla.org, www.mozilla.org, and joe.blogs.mozilla.org can aggregately use a maximum of 20% of the global limit. firefox.com has a separate maximum of 20%.

The two limits react differently to limits being reached:

  • The group limit is also called the "hard limit": it doesn't trigger origin eviction.
  • The global limit is a "soft limit" since there's a chance that some space will be freed and the operation can continue.

LRU policy

When the available disk space is filled up, the quota manager will start clearing out data based on an LRU policy — the least recently used origin will be deleted first, then the next one, until the browser is no longer over the limit.

We track the "last access time" for each origin using temporary storage. Once the global limit for temporary storage is reached (more on the limit later), we try to find all currently unused origins (i.e., ones with no tabs/apps open that are keeping open datastores). These are then sorted according to "last access time." The least recently used origins are then deleted until there's enough space to fulfill the request that triggered this origin eviction.

Смотрите также