Introduction

Silver is a distributed file system which is designed from the ground
up to write only sequentially and never over-write by the use of a
shared log.

Abstract

File systems today are built for a storage world of the past: they expect that users have limited storage and will want to reuse that storage over time by overwriting data. The trend, however, is the exact opposite: users are starting to realize that they have nearly limitless storage and never want to overwrite data. Silver is a distributed file system which is designed from the ground up to write only sequentially and never over-write by the use of a shared log. We demonstrate Silver using a FUSE prototype and show that such a design results in a scalable, high performant system which supports tiering, versioning, and cloning very efficiently.

Details

File systems today are built for a storage world of the past: they expect that users have limited storage and will want to reuse that storage over time by overwriting data. The trend, however, is the exact opposite: users are starting to realize that they have nearly limitless storage and never want to overwrite data. Users are beginning to demand not only that their file systems offer ``time-travel'' capability, but also fast snapshot and cloning, enabling quick copies to be made to support applications such as docker. Ideally, a file system should be distributed, so users can share and access their files from anywhere, yet be highly consistent, so that users do not have to change their expectations about the file system semantics. In this paper, we introduce Silver, which is a distributed file system engineered from the ground up around these observations. Silver keeps track of all changes, enabling users to go back to any point in time for almost free. Silver is built to be truly append-only and never overwrites data. While other append-only file systems exist, especially in the realm of optical media: * Silver is a distributed, highly scalable file system with strong consistency semantics. * Silver supports multi-versioning with continuous, consistent snapshots: every operation to Silver is logged and the system efficiently supports ``time-travel'' on the log. * Silver supports efficient copy-on-write cloning of any snapshot, which can be done at the file level or the file system level. * Silver separates the read path from the write path, simplifying tiering, which is implemented using a multi-level read cache.

Date

2016

Authors

Related Projects

Type

Conference

Booktitle

HotStorage '2016