Storage

Living Standard — Last Updated

Participate:
GitHub whatwg/storage (new issue, open issues)
IRC: #whatwg on Freenode
Commits:
GitHub whatwg/storage/commits
@storagestandard

Abstract

The Storage Standard defines an API for persistent storage and quota estimates, as well as the platform storage architecture.

1. Introduction

Over the years the web has grown various APIs that can be used for storage, e.g., IndexedDB, localStorage, and showNotification(). The Storage Standard consolidates these APIs by defining:

Traditionally, as the user runs out of storage space on their device, the data stored with these APIs gets lost without the user being able to intervene. However, persistent boxes cannot be cleared without consent by the user. This thus brings data guarantees users have enjoyed on native platforms to the web.

A simple way to make storage persistent is through invoking the persist() method. It simultaneously requests the end user for permission and changes the storage to be persistent once granted:

navigator.storage.persist().then(persisted => {
  if(persisted) {
    /* … */
  }
})

To not show user-agent-driven dialogs to the end user unannounced slightly more involved code can be written:

Promise.all([
  navigator.storage.persisted(),
  navigator.permissions.query({name: "persistent-storage"})
]).then((persisted, permission) => {
  if(!persisted && permission == "granted") {
    navigator.storage.persist().then( /* … */ )
  } else if(!persistent && permission == "prompt") {
    showPersistentStorageExplanation()
  }
})

The estimate() method can be used to determine whether there is enough space left to store content for an application:

function retrieveNextChunk(nextChunkInfo) {
  return navigator.storage.estimate().then(info => {
    if(info.quota - info.usage > nextChunkInfo.size)
      return fetch(nextChunkInfo.url)
    else throw new Error("insufficient space to store next chunk")
  }).then( /* … */ )
}

2. Terminology

This specification depends on the Infra Standard. [INFRA]

This specification uses terminology from the DOM, HTML, IDL, Permissions API, and URL Standards. [DOM] [HTML] [WEBIDL] [PERMISSIONS] [URL]

A schemeless origin group is a group of one of the following:

This definition will move to a more suitable location eventually.

3. Infrastructure

A user agent has various kinds of storage:

Credentials

End-user credentials, such as username and passwords submitted through HTML forms

Permissions

Permissions for various features, such as geolocation

Network

HTTP cache, cookies, authentication entries, TLS client certificates

Site
Indexed DB, Cache API, service worker registrations, localStorage, history.pushState(), application caches, notifications, etc.

This specification primarily concerns itself with site storage.

Site storage consists of zero or more site storage units.

Each origin has an associated site storage unit. A site storage unit contains a single box. [HTML]

3.1. Boxes

A box has mode which is either "best-effort" or "persistent". A persistent box is a box whose mode is "persistent". A non-persistent box is a box whose mode is not "persistent".

A box is considered to be an atomic unit. Whenever a box is cleared by the user agent, it must be cleared in its entirety.

4. Persistence permission

A box can only be turned into a persistent box if the user (or user agent on behalf of the user) has granted permission to use the "persistent-storage" feature.

When granted to an origin, the persistence permission can be used to protect storage from the user agent’s clearing policies. The user agent cannot clear storage marked as persistent without involvement from the origin or user. This makes it particularly useful for resources the user needs to have available while offline or resources the user creates locally.

The "persistent-storage" powerful feature’s permission-related flags, algorithms, and types are defaulted, except for:

permission state
"persistent-storage"'s permission state must have the same value for all environment settings objects with a given origin.
permission revocation algorithm
If "persistent-storage"'s permission state is not "granted", then set the current origin’s site storage unit’s box’s mode to "best-effort".

5. Usage and quota

The site storage usage of an origin origin is a rough estimate of the amount of bytes used in origin’s site storage unit.

This cannot be an exact amount as user agents might, and are encouraged to, use deduplication, compression, and other techniques that obscure exactly how much bytes an origin uses.

The site storage quota of an origin origin is a conservative estimate of the amount of bytes available to origin’s site storage unit. This amount should be less than the total available storage space on the device to give users some wiggle room.

User agents are strongly encouraged to provide "popular" origins with a larger site storage quota. Factors such as navigation frequency, recency of visits, bookmarking, and permission for "persistent-storage" can be used as indications of "popularity".

6. User Interface Guidelines

User agents should not distinguish between network storage and site storage in their user interface. Instead user agents should offer users the ability to remove all storage for a given schemeless origin group. This ensures to some extent that network storage cannot be used to revive site storage. This also reduces the amount users need to know about the different ways in which a schemeless origin group can store data.

Credentials storage should be separated as it might contain data the user might not be able to revive, such as an autogenerated password. Since permissions storage is mostly simple booleans it too can be separated to avoid inconveniencing the user. Credentials and permissions are also somewhat easier to understand and differentiate for users from network storage and site storage.

6.1. Storage Pressure

When the user agent notices it comes under storage pressure and it cannot free up sufficient space by clearing network storage and non-persistent boxes within site storage, then the user agent should alert the user and offer a way to clear persistent boxes.

7. API

[SecureContext,
 NoInterfaceObject,
 Exposed=(Window,Worker)]
interface NavigatorStorage {
  readonly attribute StorageManager storage;
};
Navigator implements NavigatorStorage;
WorkerNavigator implements NavigatorStorage;

Each environment settings object has an associated StorageManager object. [HTML]

The storage attribute’s getter must return context object’s relevant settings object’s StorageManager object.

[SecureContext,
 Exposed=(Window,Worker)]
interface StorageManager {
  Promise<boolean> persisted();
  [Exposed=Window] Promise<boolean> persist();

  Promise<StorageEstimate> estimate();
};

dictionary StorageEstimate {
  unsigned long long usage;
  unsigned long long quota;
};

The persisted() method, when invoked, must run these steps:

  1. Let promise be a new promise.

  2. Let settingsObject be context object’s relevant settings object.

  3. Run these substeps in parallel:

    1. Let origin be settingsObject’s origin.

    2. Let persisted be true if origin’s site storage unit’s box is a persistent box, and false otherwise.

    3. Resolve promise with persisted.

  4. Return promise.

The persist() method, when invoked, must run these steps:

  1. Let promise be a new promise.

  2. Let settingsObject be context object’s relevant settings object.

  3. Run these substeps in parallel:

    1. Let origin be settingsObject’s origin.

    2. Let permission be the result of requesting permission to use "persistent-storage".

      User agents are encouraged to not let the user answer this question twice for the same origin around the same time and this algorithm is not equipped to handle such a scenario.

    3. Let persisted be true, if origin’s site storage unit’s box is a persistent box, and false otherwise.

    4. If persisted is false and permission is "granted", then set persisted to true and set origin’s site storage unit’s box’s mode to "persistent".

    5. Queue a task to resolve promise with persisted.

  4. Return promise.

The estimate() method, when invoked, must run these steps:

  1. Let promise be a new promise.

  2. Let settingsObject be context object’s relevant settings object.

  3. Run these substeps in parallel:

    1. Let origin be settingsObject’s origin.

    2. Let usage be site storage usage for origin.

    3. Let quota be site storage quota for origin.

    4. Let dictionary be a new StorageEstimate dictionary whose usage member is usage and quota member is quota.

    5. Resolve promise with dictionary.

  4. Return promise.

Acknowledgments

With that, many thanks to Adrian Bateman, Alex Russell, Aislinn Grigas, Ali Alabbas, Ben Kelly, Ben Turner, Dale Harvey, David Grogan, Jake Archibald, Jeffrey Yasskin, Jonas Sicking, Joshua Bell, Kenji Baheux, Kinuko Yasuda, Luke Wagner, Michael Nordman, Mounir Lamouri, Shachar Zohar, 黃強 (Shawn Huang), and 簡冠庭 (Timothy Guan-tin Chien) for being awesome!

This standard is written by Anne van Kesteren (Mozilla, annevk@annevk.nl).

Per CC0, to the extent possible under law, the editor has waived all copyright and related or neighboring rights to this work.

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[DOM]
Anne van Kesteren. DOM Standard. Living Standard. URL: https://dom.spec.whatwg.org/
[HTML]
Anne van Kesteren; et al. HTML Standard. Living Standard. URL: https://html.spec.whatwg.org/multipage/
[INFRA]
Anne van Kesteren; Domenic Denicola. Infra Standard. Living Standard. URL: https://infra.spec.whatwg.org/
[PERMISSIONS]
Mounir Lamouri; Marcos Caceres. The Permissions API. URL: https://w3c.github.io/permissions/
[URL]
Anne van Kesteren. URL Standard. Living Standard. URL: https://url.spec.whatwg.org/
[WEBIDL]
Cameron McCormack; Boris Zbarsky; Tobie Langel. Web IDL. URL: https://heycam.github.io/webidl/

IDL Index

[SecureContext,
 NoInterfaceObject,
 Exposed=(Window,Worker)]
interface NavigatorStorage {
  readonly attribute StorageManager storage;
};
Navigator implements NavigatorStorage;
WorkerNavigator implements NavigatorStorage;

[SecureContext,
 Exposed=(Window,Worker)]
interface StorageManager {
  Promise<boolean> persisted();
  [Exposed=Window] Promise<boolean> persist();

  Promise<StorageEstimate> estimate();
};

dictionary StorageEstimate {
  unsigned long long usage;
  unsigned long long quota;
};