Go to file
Chris Mason d79f499eae Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block.  Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block.  This means that when we read the inode,
we've probably read in at least some checksums as well.

But, this has a few problems:

* The checksums are indexed by logical offset in the file.  When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data.  It would be faster if we could checksum
the compressed data instead.

* If we implement encryption, we'll be checksumming the plain text and
storing that on disk.  This is significantly less secure.

* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct.  This makes the raid
layer balancing and extent moving much more expensive.

* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.

* There is potentitally one copy of the checksum in each subvolume
referencing an extent.

The solution used here is to store the extent checksums in a dedicated
tree.  This allows us to index the checksums by phyiscal extent
start and length.  It means:

* The checksum is against the data stored on disk, after any compression
or encryption is done.

* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.

This makes compression significantly faster by reducing the amount of
data that needs to be checksummed.  It will also allow much faster
raid management code in general.

The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent.  This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 17:00:31 -05:00
.hgignore Add .hgignore file to ignore verison.h 2008-07-23 11:36:13 -04:00
.hgtags Added tag v0.16 for changeset 2b26e4f8c71e 2008-08-04 15:46:36 -04:00
COPYING fix COPYING 2007-06-12 15:41:08 -04:00
INSTALL Documentation update 2007-06-28 16:02:01 -04:00
Makefile Btrfs image tool 2008-11-20 09:52:48 -05:00
bcp bcp updates for single file copies 2008-05-02 16:12:02 -04:00
bit-radix.c Make btrfs-progs compile with -fstrict-aliasing (included in -O2) again. 2007-09-14 10:31:06 -04:00
bit-radix.h add GPLv2 2007-06-12 09:07:11 -04:00
btrfs-image.c Fix image tool compile warnings 2008-12-02 07:19:08 -05:00
btrfs-show.c Print the version string in a few programs 2008-07-24 12:13:30 -04:00
btrfs-vol.c Verify parent generation number on btree reads 2008-05-13 13:48:58 -04:00
btrfsck.c Add fallocate support v2 2008-10-31 12:48:02 -04:00
btrfsctl.c Modify the subvol and snapshot creation ioctls to work anywhere in the tree 2008-11-18 11:26:35 -05:00
btrfstune.c update btrfs-progs for seed device support 2008-11-18 10:40:06 -05:00
convert.c Btrfs: move data checksumming into a dedicated tree 2008-12-08 17:00:31 -05:00
crc32c.c CRC32C big endian bugs... 2008-02-15 11:20:02 -05:00
crc32c.h calculate checksums during IO 2007-06-28 16:20:29 -04:00
ctree.c update btrfs-progs for seed device support 2008-11-18 10:40:06 -05:00
ctree.h Btrfs: move data checksumming into a dedicated tree 2008-12-08 17:00:31 -05:00
debug-tree.c Remove offset field from struct btrfs_extent_ref 2008-10-09 11:55:30 -04:00
dir-item.c Unaligned access fixes 2008-02-15 11:19:58 -05:00
dir-test.c Add a readonly flag open_ctree to force RO opens 2008-05-05 09:45:26 -04:00
disk-io.c btrfs-progs: support for different csum algorithims 2008-12-02 09:58:23 -05:00
disk-io.h btrfs-progs: support for different csum algorithims 2008-12-02 09:58:23 -05:00
extent-cache.c Add support for multiple devices per filesystem 2008-03-24 15:03:18 -04:00
extent-cache.h Port extent buffer to btrfs-progs 2008-01-04 10:36:26 -05:00
extent-tree.c update btrfs-progs for seed device support 2008-11-18 10:40:06 -05:00
extent_io.c Verify parent generation number on btree reads 2008-05-13 13:48:58 -04:00
extent_io.h Verify parent generation number on btree reads 2008-05-13 13:48:58 -04:00
file-item.c btrfs-progs: support for different csum algorithims 2008-12-02 09:58:23 -05:00
hash.h Switch btrfs_name_hash() to crc32c 2008-08-20 20:11:23 -04:00
hasher.c Unaligned access fixes 2008-02-15 11:19:58 -05:00
inode-item.c btrfs-progs new dir index support 2008-07-24 12:13:32 -04:00
inode-map.c btrfs-progs: Fix printf format casting errors 2008-04-01 10:52:22 -04:00
ioctl.h Modify the subvol and snapshot creation ioctls to work anywhere in the tree 2008-11-18 11:26:35 -05:00
kerncompat.h Add mirroring support across multiple drives 2008-04-03 16:35:48 -04:00
list.h Dynamic chunk allocation 2008-03-24 15:03:58 -04:00
mkfs.c Btrfs: move data checksumming into a dedicated tree 2008-12-08 17:00:31 -05:00
print-tree.c Btrfs: move data checksumming into a dedicated tree 2008-12-08 17:00:31 -05:00
print-tree.h Update btrfs-progs to match kernel sources 2008-01-04 10:38:22 -05:00
quick-test.c Add a readonly flag open_ctree to force RO opens 2008-05-05 09:45:26 -04:00
radix-tree.c Make btrfs-progs compile with -fstrict-aliasing (included in -O2) again. 2007-09-14 10:31:06 -04:00
radix-tree.h i386 fixes from axboe 2007-06-12 11:39:09 -04:00
random-test.c add GPLv2 2007-06-12 09:07:11 -04:00
rbtree.c Stop using radix trees to record pending allocations 2007-10-15 16:25:04 -04:00
rbtree.h Stop using radix trees to record pending allocations 2007-10-15 16:25:04 -04:00
root-tree.c Add disk format requirements for subvol backward and forward refs 2008-11-18 10:34:08 -05:00
show-blocks Add debug-tree -e to print all allocated extents, and show-blocks to graph them 2008-03-03 19:55:16 -05:00
transaction.h Update btrfs-progs to match kernel sources 2008-01-04 10:38:22 -05:00
utils.c Btrfs: move data checksumming into a dedicated tree 2008-12-08 17:00:31 -05:00
utils.h Update the Ext3 converter 2008-04-22 14:06:56 -04:00
version.sh Use git for version number too 2008-08-20 20:11:22 -04:00
volumes.c Btrfs image tool 2008-11-20 09:52:48 -05:00
volumes.h update btrfs-progs for seed device support 2008-11-18 10:40:06 -05:00