Skip to main content

Decentralized Storage Systems

Storage is an integral part of modern computer systems, and the same is true for distributed and decentralized systems like a blockchain. When interacting with the Polkadot ecosystem, it will be helpful if you familiarize yourself with the current Web 3.0 approach to decentralized storage systems, and how these systems can support your endeavors.

DCS (Decentralized cloud storage)

The key attribute that characterizes centralized cloud storage is location [of data]. In decentralized cloud storage, the key attribute becomes the data itself instead of the data's location. This can be viewed as the shift from the centralized location-centric storage approach to the decentralized content-centric approach.

IPFS (Interplanetary File System)

IPFS is a peer-to-peer distributed file system that seeks to connect all computing devices with the same system of files, by utilizing features such as content-addressing, content-signing, and enhanced security methods through encryption. IPFS aims to address the current hurdles of the HTTP-based Internet.

Brief comparison of IPFS & HTTP:

IPFSHTTP
network: peer-to-peer model (decentralized)network: client-server model (centralized)
requests: use a cryptographic hash of that datarequests: use the address on which data is hosted
accessibility: data is distributed to multiple nodes and can be accessed at any time. Bandwidth is high: nearest peer can serve the dataaccessibility: data can only be accessed if the server is live and there are no interruptions in transmission. Bandwidth is limited: clients send requests to the same server

Filecoin

Like IPFS, Filecoin is a protocol developed by Protocol Labs that offers a decentralized storage network. Filecoin's main focus is the storage itself and uses IPFS as a [complementary] back-end protocol.

Storj

Storj is a decentralized cloud storage system for developers.

Sia

Sia aims to be decentralized storage for the post-cloud world.

Both Storj & Sia encrypt all files before being uploaded and distributes them robustly and reliably.

Swarm

EthSwarm is a system of peer-to-peer networked nodes that create a decentralized storage and communication service for a sovereign digital society.

BitTorrent File System (BTFS)

BTFS is a file-sharing protocol that uses the TRON network and the BitTorrent ecosystem.

Substrate Storage

Substrate takes a layered approach to storage by using a key-value data store that is implemented as a database-backed, modified Merkle tree. Substrate's higher-layer storage abstractions are built on the key-value store.

The key-value data store is backed by RocksDB, and it also supports an experimental Parity database.

The database is used for components that require persistent storage: Substrate clients, Substrate light-clients & off-chain workers. For more information, check out Substrate Developer Hub's Storage Page.

When building on Substrate, runtime developers can take advantage of Substrate's FRAME Storage pallet which gives access to Substrate's storage APIs. These storage items support values that are encoded by Parity's SCALE (Simple Concatenated Aggregate Little-Endian) Codec.

There is a Storage Value API that is used to store single values, a Storage Map API that is used to a key-value hash map, a Storage Double Map API that creates a storage map with two keys to provide the ability to efficiently remove all entries that have a common first key, and a Storage N Map API that can be used to store a hash map with any arbitrary number of keys.

These layered APIs act as runtime storage that allows you to store data in your blockchain. More information can be found at the Runtime Storage Page on Substrate Developer Hub.

PolkadotJS Storage

The PolkadotJS API offers storage methods that are part of the default Substrate runtime. They are exposed via api.query.<module>.<method>. See the official docs for more details.