[Freedombox-discuss] Tahoe-LAFS is like a Bittorrent with which you can upload as well as download

Zooko Wilcox-O'Hearn zooko at zooko.com
Fri Jun 8 04:30:04 UTC 2012

On Thu, Jun 7, 2012 at 7:29 AM, Nick Daly <nick.m.daly at gmail.com> wrote:
> What's Tahoe's required overhead?  Is it static (5KB, regardless of
> partition size), or does it grow with the reserved space (2% from 0 -
> 5 GB, 3% for > 5GB)?

Hm, I'm not exactly sure. In general, people tend to allocate large
amounts of disk space to one Tahoe-LAFS storage server, like at least
0.5 TB, and then not worry about the few KB or MB of overhead that
Tahoe-LAFS adds.

The storage server process is the one that stores ciphertext on disk
and sends and receives ciphertext over TCP on behalf of a storage
client. It stores each piece of ciphertext in a separate file in its
local filesystem, and it organizes those files into 2-level hierarchy
of directories. That's a source of overhead, since it takes a little
more than X bytes of disk space to store an X byte file in your file
system. That overhead is proportional to the number of files stored
and the exact amount is determined by the filesystem, but I would
estimate 2 KiB per file.

Another source of overhead is that the storage server maintains a
couple of small files of metadata about which files it has processed.
That is a small fixed amount that doesn't grow with the number or size
of files. It's probably around 10 KB.

Another source of overhead is per-file metadata -- leases (markings
showing that someone said "I'm interested in this file -- please don't
garbage-collect it for at least 30 more days!"), and
integrity-checking hashes. The integrity checks are proportional to
the size of each file, but small. It's about 64 bytes of hashes for
each 40 KB of file.

>   My biggest concern with this is space, as the 2GB microSD card the DreamPlug comes with is tough to manage.  If we can use the SD card slot when it's available, this'll be less of a problem, but I haven't really thought about that problem yet.

My instinct would be not to try to make use of such scarce storage
resources, but instead say "Want to contribute to the great FreedomBox
Cloud Storage grid? Then plug in a 1 TB hard drive to the USB port!".
That would actually make the resulting storage much more reliable,
because only people who felt a certain level of willingness to
contribute would run a Tahoe-LAFS storage server at all. By excluding
all the people who turned it on at some point and then got bored a few
hours later and turned it off again, we would greatly reduce the churn
and unreliability.



More information about the Freedombox-discuss mailing list