Introduction
Silver is a distributed file system which is designed from the ground
up to write only sequentially and never over-write by the use of a
shared log.
Abstract
File systems today are built for a storage world of the past:
they expect that users have limited storage and will want to reuse that
storage over time by overwriting data.
The trend, however, is the exact opposite: users are starting to
realize that they have nearly limitless storage
and never want to overwrite data.
Silver is a distributed file system which is designed from the ground
up to write only sequentially and never over-write by the use of a
shared log. We demonstrate Silver using a FUSE prototype and show that
such a design results in a scalable, high performant system which
supports tiering, versioning, and cloning very efficiently.
Details
File systems today are built for a storage world of the past:
they expect that users have limited storage and will want to reuse that
storage over time by overwriting data.
The trend, however, is the exact opposite: users are starting to
realize that they have nearly limitless storage
and never want to overwrite data. Users are beginning to demand not only
that their file systems offer ``time-travel'' capability, but also
fast snapshot and cloning, enabling quick copies to be made to
support applications such as docker. Ideally, a file
system should be distributed, so users can share and access their files
from anywhere, yet be highly consistent, so that users do not have to
change their expectations about the file system semantics.
In this paper, we introduce Silver, which is a distributed file system engineered
from the ground up around these observations.
Silver keeps track of all changes, enabling users to go back to any
point in time for almost free. Silver is built to be truly
append-only and never overwrites data. While other append-only file
systems exist, especially in the realm of optical media:
* Silver is a distributed, highly scalable file system with
strong consistency semantics.
* Silver supports multi-versioning with continuous, consistent snapshots:
every operation to Silver is logged and the system efficiently supports
``time-travel'' on the log.
* Silver supports efficient copy-on-write cloning of any snapshot,
which can be done at the file level or the file system level.
* Silver separates the read path from the write path, simplifying
tiering, which is implemented using a multi-level read cache.