From FIL to Shelby: The Evolution of Decentralization Storage

Storage has been one of the popular tracks in the blockchain industry, and Filecoin, as the leading project of the last bull market, once had a market value exceeding $10 billion. Arweave, with its selling point of permanent storage, reached a maximum market value of $3.5 billion. With the availability of cold data storage being questioned, the necessity of permanent storage is challenged, and the development of decentralized storage has hit a bottleneck. The emergence of Walrus has brought renewed attention to the long-silent storage track, while the Shelby project, launched in collaboration between Aptos and Jump Crypto, aims to promote the application of hot data storage. This article will analyze the evolution path of decentralized storage from the development history of the four projects: Filecoin, Arweave, Walrus, and Shelby, and explore its future development prospects.

Filecoin: Building a Decentralized Data Cloud

Filecoin, as a representative project that emerged early on, has its development direction centered around Decentralization, which is also a common characteristic of early blockchain projects. Filecoin combines storage with Decentralization, attempting to solve the trust issues of centralized data storage. However, certain aspects sacrificed for the sake of achieving Decentralization later became pain points that projects like Arweave or Walrus aimed to address.

IPFS: architecture Decentralization, but limited by transmission bottlenecks

IPFS was launched in 2015, aiming to revolutionize the traditional HTTP protocol through content addressing. However, the biggest drawback of IPFS is its extremely slow retrieval speed, which makes it difficult to meet practical application needs. The underlying P2P protocol of IPFS is mainly suitable for "cold data", which refers to static content that does not change often, and it does not have significant advantages in handling hot data.

Although IPFS itself is not a blockchain, its directed acyclic graph design concept is highly compatible with many public chains and Web3 protocols, making it an ideal framework for blockchain infrastructure.

FIL's economic model

The token economic model of Filecoin mainly includes three roles: users, storage miners, and retrieval miners. Users pay fees to store data, storage miners receive token rewards for storing data, and retrieval miners provide data when users need it and receive rewards.

This model has potential vulnerabilities. Storage miners may fill in junk data to obtain rewards, and since this data will not be retrieved, its loss will not trigger a penalty mechanism. The Filecoin replication proof consensus can only ensure that user data has not been deleted, but it cannot prevent miners from filling in junk data.

The operation of Filecoin largely relies on miners' continued investment in the token economy, rather than on the real demand for distributed storage from end users. Although the project continues to iterate, at this stage, Filecoin is more aligned with the "mining coin logic" rather than the "application-driven" storage project positioning.

Arweave: The Double-Edged Sword of Long-Termism

Compared to Filecoin's construction of an incentivized and provable decentralized "data cloud", Arweave focuses on providing permanent storage capabilities. Arweave does not attempt to build a distributed computing platform; its entire system revolves around the core assumption that "important data should be stored once and preserved permanently". This extreme long-termism sets Arweave apart from Filecoin in terms of mechanisms, incentive models, hardware requirements, and narrative perspectives.

Arweave takes Bitcoin as a learning object and is dedicated to continuously optimizing its permanent storage network over the long term. The project team does not care about marketing and competitors, focusing instead on iterating the network architecture. This long-termism has made Arweave popular in the last bull market, and it also positions the project to withstand multiple bull and bear cycles. However, the value of permanent storage still needs time to be validated.

Since version 1.5 to the latest version 2.9, the Arweave mainnet has been committed to lowering the participation threshold for miners, incentivizing them to maximize data storage, and continuously improving network robustness. In unfavorable market conditions, Arweave has taken a conservative approach, not embracing the miner community, leading to stagnation in ecological development, upgrading the mainnet at minimal cost, and continuously lowering hardware requirements while ensuring network security.

Major Version Upgrade Review

Version 1.5 exposed a vulnerability where miners could rely on GPU stacking instead of real storage to optimize block generation chances. Version 1.7 introduced the RandomX algorithm, limiting the use of specialized computing power and requiring general-purpose CPUs to participate in mining, weakening computing power centralization.

Version 2.0 adopts the SPoA mechanism, transforming data proofs into a concise path of Merkle tree structure, and introduces format 2 transactions to reduce synchronization burden. This architecture alleviates network bandwidth pressure and significantly enhances node collaboration capabilities. However, some miners can still evade the responsibility of holding real data through centralized high-speed storage pool strategies.

Version 2.4 introduces the SPoRA mechanism, which incorporates global indexing and slow hash random access. Miners must genuinely hold data blocks to participate in valid block creation, thereby weakening the effect of power stacking from a mechanistic standpoint. Miners are beginning to focus on storage access speed, driving the application of high-speed read/write devices such as SSDs. Version 2.6 introduces hash chain control to regulate block creation rhythm, balancing the marginal benefits of high-performance devices and providing fair participation space for small and medium-sized miners.

Subsequent versions further enhance network collaboration capabilities and storage diversity: 2.7 adds collaborative mining and pool mechanisms, improving the competitiveness of small miners; 2.8 introduces a composite packaging mechanism, allowing large-capacity low-speed devices to participate flexibly; 2.9 introduces a new packaging process in the replica_2_9 format, significantly improving efficiency and reducing computational dependencies, completing the closed loop of the data-driven mining model.

Overall, Arweave's upgrade path clearly presents its long-term strategy oriented towards storage: while continuously resisting the trend of computing power centralization, it lowers the participation threshold to ensure the possibility of the protocol's long-term operation.

Walrus: A New Attempt at Hot Data Storage

The design philosophy of Walrus is completely different from that of Filecoin and Arweave. Filecoin is dedicated to creating a verifiable Decentralization storage system, but it is only suitable for cold data; Arweave focuses on permanent data storage, but its application scenarios are limited; Walrus aims to optimize the cost of hot data storage protocols.

RedStuff: Innovations and Limitations of the Magic Modified Error Correction Code

Walrus believes that the storage costs of Filecoin and Arweave are unreasonable. Both use a fully replicated architecture, which, while providing strong fault tolerance and node independence, requires multiple redundant copies to maintain robustness, thereby increasing storage costs. Walrus attempts to find a balance between the two, enhancing availability through a structured redundancy approach while controlling replication costs.

The RedStuff technology created by Walrus is based on Reed-Solomon ( RS ) coding, which is a traditional erasure coding algorithm. Erasure coding allows for the duplication of datasets by adding redundant fragments for the reconstruction of the original data. RS coding is widely used in fields such as CD-ROMs, satellite communications, and QR codes.

The core of RedStuff is to split data into primary slices and secondary slices. Primary slices are used to recover the original data, generated and distributed under strict constraints; secondary slices are generated through simple operations, providing elastic fault tolerance and enhancing system robustness. This structure reduces the requirements for data consistency, allowing different nodes to temporarily store different versions of data, emphasizing "eventual consistency."

RedStuff achieves effective storage in low computing power and low bandwidth environments, but essentially still belongs to a variant of erasure coding systems. It sacrifices some data read determinism in exchange for cost control and scalability in a decentralized environment. However, RedStuff has not truly broken through the coding computation bottleneck of erasure coding, but has instead avoided the high coupling points of traditional architectures through structural strategies. Its innovation is more reflected in the combinatorial optimization on the engineering side rather than a disruption at the fundamental algorithm level.

The ecological synergy between Walrus and Sui

The target scenario of Walrus is to store large binary files ( Blobs ), which are at the core of many decentralized applications. In the cryptocurrency field, this mainly refers to images and videos in NFTs and social media content.

Although Walrus also mentioned the potential uses of AI model dataset storage and data availability layer (DA), the retreat of Web3 AI projects has made the prospects for related applications unclear. As for the DA layer, whether Walrus can become an effective alternative will need to be verified after mainstream projects like Celestia regain market attention.

Therefore, Walrus's core positioning can be understood as a hot storage system for content assets such as NFTs, emphasizing dynamic invocation, real-time updates, and version management capabilities. This also explains why Walrus needs to rely on Sui: with the high-performance chain capabilities of Sui, Walrus can build a high-speed data retrieval network, significantly reducing operational costs and avoiding direct competition with traditional cloud storage services on a unit cost basis.

According to official data, Walrus's storage costs are about one-fifth of traditional cloud services. Although it is tens of times more expensive than FIL and Arweave, its goal is to build a decentralized hot storage system that can be used in real business scenarios. Walrus itself operates as a PoS network, with the core responsibility of verifying the honesty of storage nodes and providing basic security guarantees for the system.

For Sui, there is currently no urgent need for off-chain storage support. However, if there is a future desire to support complex scenarios such as AI applications, content assetization, and composable agents, the storage layer will be indispensable in providing context, context, and indexing capabilities. High-performance chains can handle complex state models, but these states need to be bound to verifiable data in order to build a trustworthy content network.

Shelby: Unlocking the Potential of Web3 Applications with Dedicated Networks

Among the technical bottlenecks faced by Web3 applications, "read performance" has always been a challenging shortcoming. Whether it's video streaming, RAG systems, real-time collaboration tools, or AI model inference engines, they all rely on low-latency and high-throughput access to hot data. Existing decentralized storage protocols have made progress in data persistence and trustlessness, but due to operating over the public internet, they cannot escape the limitations of high latency, unstable bandwidth, and uncontrollable data scheduling.

Shelby attempts to address this issue at its source. First, the Paid Reads mechanism reshapes the "read operation" dilemma in decentralized storage. In traditional systems, reading data is almost free, and the lack of effective incentives leads to service nodes being generally reluctant to respond. Shelby introduces a pay-per-read model that directly links user experience with service node revenue: the faster and more reliably nodes return data, the more rewards they can earn. This is not an incidental economic design but the core logic of Shelby's performance design.

Secondly, Shelby introduces a dedicated fiber optic network, building a high-speed channel for the instant reading of hot data in Web3. This architecture bypasses the public transport layer that Web3 systems generally rely on, directly deploying storage nodes and RPC nodes on a high-performance, low-congestion, physically isolated transport backbone. This not only significantly reduces the latency of inter-node communication but also ensures the predictability and stability of the transmission bandwidth. The underlying network structure of Shelby is closer to the dedicated line deployment model between AWS internal data centers, rather than the "uploading to a miner node" logic of other Web3 protocols.

This network-level architectural inversion makes Shelby the first truly capable decentralized hot storage protocol to support Web2-level user experiences. Users can read 4K videos, call embedding data from large language models, or trace transaction logs on Shelby, achieving sub-second response times. For service nodes, the dedicated network not only enhances service efficiency but also significantly reduces bandwidth costs, making the "pay-per-read" model economically viable, thus incentivizing the system to evolve towards higher performance rather than higher storage capacity.

In terms of data persistence and cost, Shelby uses the Efficient Coding Scheme built with Clay Codes, achieving storage redundancy as low as <2x through MSR and MDS optimal coding structures, while maintaining 11 nines of persistence and 99.9% availability. This is not only technically more efficient but also more competitive in terms of cost, providing dApp developers who value cost optimization and resource scheduling with a "cheap and fast" option.

Summary and Outlook

The evolution from Filecoin, Arweave, and Walrus to Shelby shows that the narrative of Decentralization storage has gradually shifted from a "being is justified" technological utopia to a "usability is justice" realism. Early projects were driven by economic incentives.