Commit Graph

139 Commits (07ce7005fc81289eb4c7dde7d601be08c977b92c)

Author SHA1 Message Date
David Sterba 2b7cdab425 btrfs-progs: make getopt tables static const
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-19 13:44:49 +01:00
David Sterba ecbf1339a6 btrfs-progs: cleanup, move getop long options close to their use
Move long_option defintions just before getopt_long everywhere.

Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-19 13:30:06 +01:00
Qu Wenruo e960b3f057 btrfs-progs: Don't increase error count if the inode can be repaired.
The original check_inode_recs() will return -1 if found any error in a
inode_record. This is OK for original design since there is almost
nothing can repair at that time.

However more and more error from nlink mismatch to missing inode item
can be repaired in try_repair_inode(), check_inode_recs() should not
increase the error count if the inode can be repair.

With this patch, repair function for leaf-corruption will not return
error if all corruption inode can be recovered.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-09 19:09:47 +01:00
Qu Wenruo 0da591ddf2 btrfs-progs: Fix a extent buffer leak in count_csum_range().
The commit f495a2ac66 ("btrfs-progs: fsck: remove unfriendly BUG_ON()
for searching tree failure") is causing tons of extent buffer leak if some
csum mismatches in btrfsck.

This is caused by a misplaced btrfs_release_path(), fix it.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-09 18:34:39 +01:00
David Sterba 70607997ee btrfs-progs: fix leak of path in reset_balance
A path may leak after an unlikely error condition.

Resolves-coverity-id: 1199474
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-03 03:33:31 +01:00
David Sterba fd1e9ad6a8 btrfs-progs: check, handle error in fix_key_order
In case the buffer is corrupted and the for loop does not happen, we'd
return garbage. The caller retunrs -EIO in case of any corruption, use
that value in fix_key_order.

Resolves-coverity-id: 1246944
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-03 02:47:32 +01:00
David Sterba 01a9f4308b btrfs-progs: check, missing parens around compound block in find_normal_file_extent
Resolves-coverity-id: 1260248
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-30 15:48:55 +01:00
David Sterba 0665aaf5a3 btrfs-progs: check, fix path leak in error branch
Resolves-coverity-id: 1260250
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-30 15:48:54 +01:00
Qu Wenruo 0f38766b8f btrfs-progs: Fix a bug in reset_nlink() which may deletes the recovered file
In reset_nlink(), we believe rec->found_link as accurate number of the
valid links. However it only records the number of found DIR_ITEM, so we
can't use it as reliable value.

Before this patch, in some case, leaf corruption recovery will believe
there is a valid backref but don't add_link() since it can't find any
valid one and don't put it into the lost+found dir.
So the recovered inode will be considered as an orphan item without
orphan item and repair_inode_orphan_item() will add orphan item for it,
causing all the filename/filetype we recovered being a waste of time.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-29 17:13:41 +01:00
David Sterba 34350d11c1 btrfs-progs: check result of first_cache_extent
If the tree's empty, we'll get NULL and dereference it.

Resolves-Coverity-CID: 1248828
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-19 18:04:23 +01:00
Wang Shilong 1f728b1a51 Btrfs-progs, fsck: move root items repair after root rebuilding
If some critical roots are corrupt, reapr_root_items() will fail,
this is detected by fsck_tests.sh's extent rebuilding tests.

Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 15:42:52 +01:00
Wang Shilong b3375a7e8b Btrfs-progs: fsck: add ability to rebuild extent tree with snapshots
This patch makes us to rebuild a really corrupt extent tree with snapshots.
To implement this, we have to verify whether a block is FULL BACKREF.

This idea come from Josef Bacik:

1) We walk down the original tree, every eb we encounter has
btrfs_header_owner(eb) == root->objectid.  We add normal references
for this root (BTRFS_TREE_BLOCK_REF_KEY) for this root.  World peace
is achieved.

2) We walk down the snapshotted tree.  Say we didn't change anything
at all, it was just a clean snapshot and then boom.  So the
btrfs_header_owner(root->node) == root->objectid, so normal backref.
We walk down to the next level, where btrfs_header_owner(eb) !=
root->objectid, but the level above did, so we add normal refs for all
of these blocks.  We go down the next level, now our
btrfs_header_owner(parent) != root->objectid and
btrfs_header_owner(eb) != root->objectid.  This is where we need to
now go back and see if btrfs_header_owner(eb) currently has a ref on
eb.  If it does we are done, move on to the next block in this same
level, we don't have to go further down.

3) Harder case, we snapshotted and then changed things in the original
root.  Do the same thing as in step 2, but now we get down to
btrfs_header_owner(eb) != root->objectid && btrfs_header_owner(parent)
!= root->objectid.  We lookup the references we have for eb and notice
that btrfs_header_owner(eb) no longer refers to eb.  So now we must
set FULL_BACKREF on this extent reference and add a
SHARED_BLOCK_REF_KEY for this eb using the parent->start as the
offset.  And we need to keep walking down and doing the same thing
until we either hit level 0 or btrfs_header_owner(eb) has a ref on the
block.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 15:42:52 +01:00
Wang Shilong 41beb7d4a7 Btrfs-progs: fsck: deal with snapshot one by one when rebuilding extent tree
Previously, we deal with node block firstly and then leaf block which can
maximize readahead. However, to rebuild extent tree, we need deal with snapshot
one by one.

This patch makes us deal with snapshot one by one if we need rebuild extent
tree otherwise we drop into previous way.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 15:42:52 +01:00
Qu Wenruo 55fea7186c btrfs-progs: Add inode item rebuild function.
Add a basic inode item rebuild function for I_ERR_NO_INODE_ITEM.
The main use case is to repair btrfs which fs root has corrupted leaf,
but it is already working for case if the corrupteed fs root leaf/node
contains no inode extent_data.

The repair needs 3 elements for inode rebuild:
1. inode number
   This is quite easy, existing inode_record codes will detect it quite
   well.

2. inode type
   This is the trick part. The only reliable method is to recovery it from
   parent's dir_index/item.
   The remaining method will search for regular file extent for FILE
   type or child's backref for DIR(todo).
   Fallback will be FILE.

Inode name(inode_ref) will be recoverd by nlink repair function.

This is just a fundamental implement, some advanced recovery can be
improved later with btrfs-progs infrastructure change.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:52:28 +01:00
Qu Wenruo 1581d7e5db btrfs-progs: Recover btree by dropping corrupted leaf/node.
Current btrfsck can skip corrupted leaf and even repair some corrupted
one if their bytenr or key order is wrong.

However when it comes to csum error leaf, btrfsck can only skip them,
which is OK for read-only iteration, but is bad for write.

This patch introduce the new repair_btree() function to recover the
btree, deleting all the corrupted leaf/node including corresponding
extent, allowing later write into the btree.

This patch provides the basis for later recovery with corrupted leaves.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:50:28 +01:00
Qu Wenruo 887cfa14ad btrfs-progs: record and report leaf/node corruption in fs/subvol tree
When leaf/node is corrupted in fs/subvolume root, btrfsck can ignore it
without much pain except some stderr messages complaining about it.
But this works fine doing read-only works, if we want to do deeper
recovery like rebuild missing inodes in the b+tree, it will cause
problem.

At least, info user that there is something wrong in the btree,
and this patch provides the base for later btree repair.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:44:53 +01:00
Qu Wenruo 61d7fc4ee6 btrfs-progs: Add fixing function for inodes whose nlink dismatch
[BUG]
At least two users have already hit a bug in btrfs causing file
missing(Chromium config file).
The missing file's nlink is still 1 but its backref points to non-exist
parent inode.

This should be a kernel bug, but btrfsck fix is needed anyway.

[FIX]
For such nlink mismatch inode, we will reset all the inode_ref with its
dir_index/item (including valid one), and re-add the valids.
If there is no valid backref for it, create 'lost+found' under the root
of the subvolume and add link to the directory.

Reported-by: Mike Gavrilov <mikhail.v.gavrilov@gmail.com>
Reported-by: Ed Tomlinson <edt@aei.ca>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:44:53 +01:00
Qu Wenruo 15bc913158 btrfs-progs: Add helper function find_file_name/type for nlink and inode_item repair.
Add find_file_name() and find_file_type() function for later nlink and
inode_item repair.

Later nlink repair will use both function and and inode_item repair will
use find_file_type().

They are done by searching the backref list, dir_item/index for type
search and dir_item/index or inode_ref for name search.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:44:53 +01:00
Qu Wenruo 0cc75eddd0 btrfs-progs: Add btrfs_unlink() and btrfs_add_link() functions.
Add btrfs_unlink() and btrfs_add_link() functions in inode.c,
for the incoming btrfs_mkdir() and later inode operations functions.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:13:06 +01:00
Qu Wenruo ed3a206c8f btrfs-progs: Record highest inode number before repair.
Record highest inode number before inode repair.

This is especially important for corrupted leaf case.
Under that case, if use btrfs_find_free_objectid, it may find a ino
existing in corrupted leaf but dropped by btree_recover.
If that happens, created dir will be referenced incorrectly since there
may be inode_ref or dir_index/item refers to it.

So we must record the highest inode number according to the inode_cache.
Inode_cache is OK since when a inode_ref or dir_index/item is found even
the referenced source is not found, it will be created.
If we record the highest inode number of inode_cache, and use
highest_inode + 1 as 'lost+found' dir, it will ensure the newly created
dir not conflicting with any possible inode.

This provides the basis for nlink or inode rebuild for repairing btrfs
with leaf/node corruption.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 11:55:17 +01:00
Josef Bacik 3a98fe43f6 Btrfs-progs: allow fsck to take the tree bytenr
Sometimes we have a pretty corrupted fs but have an old tree bytenr that we
could use, add the ability to specify the tree root bytenr.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Tested-by: Ansgar Hockmann-Stolle <ansgar.hockmann-stolle@uni-osnabrueck.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-04 17:03:56 +01:00
Qu Wenruo 0ffacad290 btrfs-progs: rebuild missing block group during chunk recovery if possible
Before the patch, chunk will be considered bad if the corresponding
block group is missing, even the only uncertain data is the 'used'
member of the block group.

This patch will try to recalculate the 'used' value of the block group
and rebuild it.
So even only chunk item and dev extent item is found, the chunk can be
recovered.
Although if extent tree is damanged and needed extent item can't be
read, the block group's 'used' value will be the block group length, to
prevent any later write/block reserve damaging the block group.
In that case, we will prompt user and recommend them to use
'--init-extent-tree' to rebuild extent tree if possible.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-04 16:48:13 +01:00
Qu Wenruo 1102b3483b btrfs-progs: print root dir verbose error in fsck
Before this patch, when btrfsck found an error in root dir, it will only
output the following message "root %llu root dir %llu error" without any
detailed error.

Just add print_inode_error() to print out the whole error.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-04 14:37:44 +01:00
Josef Bacik e4e7058173 Btrfs-progs: create missing root dirid
If we just don't have the root dirid stuff go ahead and re-create it, since it
is easily recreated.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-11-14 11:01:38 +01:00
Josef Bacik 9151febadd Btrfs-progs: fix missing inode items
If we have all the other items but no inode item we can recreate it for the most
part, with the exception of the permissions and ownership.  Add this ability to
btrfsck.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-11-14 11:01:24 +01:00
Josef Bacik 2dc4c0015f Btrfs-progs: add ability to replace missing dir item/dir indexes
If we have everything except the dir item and dir index we can easily replace
them, so add this ability to btrfsck.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-11-14 11:00:43 +01:00
Qu Wenruo 0f70b67a0c btrfs-progs: Fix the argument requirement for '--subvol-extents'
The following commit changed the argument requirement for
'--subvol-extents', which causes it to call arg_strtou64() on NULL,
resulting a segfault.
d34cbe76 btrfs-progs: check: do not require argument for --subvol-extents

This patch revert the patch and change the help string and man page to
make it no longer confusing.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-11-03 18:35:31 +01:00
Filipe Manana 555b7feaeb Btrfs-progs: check, ability to detect and fix outdated snapshot root items
This change adds code to detect and fix the issue introduced in the kernel
release 3.17, where creation of read-only snapshots lead to a corrupted
filesystem if they were created at a moment when the source subvolume/snapshot
had orphan items. The issue was that the on-disk root items became incorrect,
referring to the pre orphan cleanup root node instead of the post orphan
cleanup root node.

A test filesystem can be generated with the test case recently submitted for
xfstests/fstests, which is essencially the following (bash script):

    workout()
    {
	ops=$1
	procs=$2
	num_snapshots=$3

	_scratch_mkfs >> $seqres.full 2>&1
	_scratch_mount

	snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
	snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
	run_check $FSSTRESS_PROG -p $procs \
	    -x "$snapshot_cmd" -X $num_snapshots -d $SCRATCH_MNT -n $ops
    }

    ops=10000
    procs=4
    snapshots=500
    workout $ops $procs $snapshots

Example of btrfsck's (btrfs check) behaviour against such filesystem:

  $ btrfsck /dev/loop0
  root item for root 311, current bytenr 44630016, current gen 60, current level 1, new bytenr 44957696, new gen 61, new level 1
  root item for root 1480, current bytenr 1003569152, current gen 1271, current level 1, new bytenr 1004175360, new gen 1272, new level 1
  root item for root 1509, current bytenr 1037434880, current gen 1300, current level 1, new bytenr 1038467072, new gen 1301, new level 1
  root item for root 1562, current bytenr 33636352, current gen 1354, current level 1, new bytenr 34455552, new gen 1355, new level 1
  root item for root 3094, current bytenr 1011712000, current gen 2935, current level 1, new bytenr 1008484352, new gen 2936, new level 1
  root item for root 3716, current bytenr 80805888, current gen 3578, current level 1, new bytenr 73515008, new gen 3579, new level 1
  root item for root 4085, current bytenr 714031104, current gen 3958, current level 1, new bytenr 716816384, new gen 3959, new level 1
  Found 7 roots with an outdated root item.
  Please run a filesystem check with the option --repair to fix them.

  $ echo $?
  1

  $ btrfsck --repair /dev/loop0
  enabling repair mode
  fixing root item for root 311, current bytenr 44630016, current gen 60, current level 1, new bytenr 44957696, new gen 61, new level 1
  fixing root item for root 1480, current bytenr 1003569152, current gen 1271, current level 1, new bytenr 1004175360, new gen 1272, new level 1
  fixing root item for root 1509, current bytenr 1037434880, current gen 1300, current level 1, new bytenr 1038467072, new gen 1301, new level 1
  fixing root item for root 1562, current bytenr 33636352, current gen 1354, current level 1, new bytenr 34455552, new gen 1355, new level 1
  fixing root item for root 3094, current bytenr 1011712000, current gen 2935, current level 1, new bytenr 1008484352, new gen 2936, new level 1
  fixing root item for root 3716, current bytenr 80805888, current gen 3578, current level 1, new bytenr 73515008, new gen 3579, new level 1
  fixing root item for root 4085, current bytenr 714031104, current gen 3958, current level 1, new bytenr 716816384, new gen 3959, new level 1
  Fixed 7 roots.
  Checking filesystem on /dev/loop0
  UUID: 2186e9b9-c977-4a35-9c7b-69c6609d4620
  checking extents
  checking free space cache
  cache and super generation don't match, space cache will be invalidated
  checking fs roots
  checking csums
  checking root refs
  found 618537000 bytes used err is 0
  total csum bytes: 130824
  total tree bytes: 601620480
  total fs tree bytes: 580288512
  total extent tree bytes: 18464768
  btree space waste bytes: 136939144
  file data blocks allocated: 34150318080
   referenced 27815415808
  Btrfs v3.17-rc3-2-gbbe1dd8

  $ echo $?
  0

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-17 18:24:54 +02:00
Filipe Manana d6d44779f5 Btrfs-progs: check, fix return value check of is_child_root()
The following commit:

    "btrfs-progs: fsck: remove unfriendly BUG_ON() for searching tree failure"
    f495a2ac66

introduced a regression, detected through xfstests/btrfs/054, where
previously a negative return value (-1) was used to mean a particular
root didn't had any parent root, and now, after that change, a negative
value is also used to mean that an error happened. That change also made
the only caller of is_child_root() interpret any negative return value
as an error and therefore incorrectly made the caller leave with an
error, instead of continuing.

This affects only the 3.17 release candidates (3.16 and older releases
don't have this issue).

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-16 12:01:55 +02:00
Josef Bacik e3966ec9d0 Btrfs-progs: lookup all roots that point to a corrupt block
If we have a corrupt block that multiple snapshots point to we will only fix the
guy who originally pointed to the block, and then simply loop forever because we
keep finding the same bad block.  So instead lookup all roots that point to this
block, and then search down to the block for each root and fix the block in all
snapshots.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 11:11:58 +02:00
Josef Bacik 01f868b5a9 btrfs-progs: make fsck deal with bogus items
We can deal with corrupt items by deleting them in a few cases.  Fsck can easily
recover from a missing extent item or a dir index item.  So if we notice a item
is completely bogus and it is of a key that we know we can repair then just
delete it and carry on.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 11:10:36 +02:00
Josef Bacik f738a758af btrfs-progs: check blocks when checking fs roots
Usually if we find a bad block during the extent tree stuff we will error out,
but if the bad block is in an fs tree and doens't have extents in it then fsck
may still pass even though the block was complete garbage.  So add the check
block logic to the fs root checking so we actually error out of fsck if there is
a bad block.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 11:06:31 +02:00
Josef Bacik 187b85436b btrfs-progs: add the ability to fix shifted item offsets
A user had a corrupted fs where the items had been shifted improperly.  This
patch adds the ability to fix this sort of problem within fsck.  We will simply
shift the item over to the proper offset and update the offsets to make sure
they are correct.  I tested this with a hand crafted fs that was broken in the
same way as the user, and I've included the file as a new test.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 11:04:08 +02:00
Josef Bacik c2e686d76a Btrfs-progs: deal with mismatch index between dir index and inode ref
Sometimes we have a dir index and an inode ref that don't agree on the index.
In this case just assume that the inode ref is the ultimate authority on the
subject and delete the dir index.  This means we have to not reset index if we
find a mismatched inode ref to make sure we delete the right dir index.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:47:59 +02:00
Josef Bacik bb561f5485 Btrfs-progs: add a dummy backref if our location is wrong
If our location is bogus in our dir item we were just skipping the thing.
However in this case we want to just delete the dir index, so create a dummy
inode rec using BTRFS_MULTIPLE_OBJECTIDS and just add every backref we find to
the list so we know to straight up delete all of these items.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:45:25 +02:00
Josef Bacik b25a40651d Btrfs-progs: delete bogus dir indexes
We may run across dir indexes that are corrupt in such a way that it makes them
useless, such as having a bad location key or a bad name.  In this case we can
just delete dir indexes that don't show up properly and then re-create what we
need.  When we delete dir indexes however we need to restart scanning the fs
tree as we could have greated bogus inode recs if the location key was bad, so
set it up so that if we had to delete an dir index we go ahead and free up our
inode recs and return -EAGAIN to check_fs_roots so it knows to restart the loop.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:45:03 +02:00
Josef Bacik 994ce2672d Btrfs-progs: re-search tree root if it changes
If we change something while scanning fs-roots we need to redo our search so
that we get valid root items and have valid root cache.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:41:51 +02:00
Josef Bacik a8e9c9a65b Btrfs-progs: reset chunk state if we restart check
If we hid a corrupt block that we fix and we restart the fsck loop you will get
lots of noise about duplicate block groups and such.  This is because we don't
clear the block group and chunk cache when we do this restart.  This patch fixes
that, which is a little tricky since the structs are linked together with
various linked lists, but this passed with a user who was hitting this problem.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:40:22 +02:00
Josef Bacik cdb9e22e29 Btrfs-progs: break out rbtree util functions
These were added to deal with duplicated functionality within btrfs-progs, but
we specifically copied rbtree.c from the kernel, so move these functions out
into their own file.  This will make it easier to keep rbtree.c in sync.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:39:39 +02:00
Josef Bacik 6863bcf74f Btrfs-progs: repair missing dir index
If we have an inode backref entry then we know enough to add back a missing dir
index.  When messing with the inode backrefs we need to do all of that first
before we process the inode recs themselves as we may clear errors on the inode
recs as we fix the directory indexes.  This adds the framework for fixing
backref errors and fixes missing dir index issues.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-13 23:20:44 +02:00
Alexandre Oliva 12adc878ae btrfs-progs: check: do not dereference tree_refs as data_refs
In a filesystem corrupted by a faulty memory module, btrfsck would get
very confused attempting to access backrefs that weren't data backrefs
as if they were.  Besides invoking undefined behavior for accessing
potentially-uninitialized data past the end of objects, or with
dynamic types unrelated with the static types held in the
corresponding memory, it used offsets and lengths from such fields
that did not correspond to anything in the filesystem proper.

Moving the test for full backrefs and checking that they're data
backrefs earlier avoided the crash I was running into, but that was
not enough to make the filesystem complete a successful repair.

Signed-off-by: Alexandre Oliva <oliva@gnu.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 18:11:33 +02:00
Alexandre Oliva ca1e04597a btrfs-progs: repair: remove recowed entry from the to-recow list
If we attempt to repair a filesystem with metadata blocks that need
recowing, we'll get into an infinite loop repeatedly recowing the
first entry in the list, without ever removing it from the list.
Oops.  Fixed.

Signed-off-by: Alexandre Oliva <oliva@gnu.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 18:11:32 +02:00
Wang Shilong ed1b086a30 Btrfs-progs: fsck: deal with corrupted csum root
If checksum root is corrupted, fsck will get segmentation. This
is because if we fail to load checksum root, root's node is NULL which
cause NULL pointer deferences later.

To fix this problem, we just did something like extent tree rebuilding.
Allocate a new one and clear uptodate flag. We will do sanity check
before fsck going on.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 18:11:32 +02:00
Wang Shilong 8d2abcb6cc Btrfs-progs: fsck: only allow partial opening under repair mode
The reason that we allow partial opening is that sometimes,
we may have some corrupted trees.(for example extent tree), for
fsck repair case, the broken tree may be rebuilt later.

So if users only want to do check but not repair anything, this
patch will make fsck return failure as soon as possible and
tell users that some critial roots have been corrupted.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 18:11:32 +02:00
Qu Wenruo 0cf497719c btrfs-progs: Check the consistence between the parent node and child node/leaf.
When btrfs-progs walk down the tree, it does not check whether the child
node/leaf is valid.
In fact, there is some corrupted image whose csum is all valid but
parent node points to a invalid leaf.

In my case, the parent node in fs tree point to a invalid leaf(gen 11),
whose generation(15) and first key(EXTENT_TREE ROOT_ITEM 0) is
completely invalid, and will cause BUG_ON in process_inode_item().

Unfortunately, we are unable to fix when it happens.
So we can only output meaningful error message and avoid the insane
node/leaf, which is still much better than the original BUG_ON().

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:38 +02:00
Josef Bacik 9ecbbe30ec btrfs-progs: rebuild the crc tree with --init-csum-tree
We have --init-csum-tree, which just empties the csum tree.  I'm not sure why we
would ever need this, but we definitely need to be able to rebuild the csum tree
in some cases.  This patch adds the ability to completely rebuild the crc tree
by reading all of the data and adding csum entries for them.  This patch doesn't
pay attention to NODATASUM inodes, it'll happily add csums for everything.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:34 +02:00
Filipe David Borba Manana 6c9caa4dd6 btrfs-progs: check, fix csum check in the presence of non-inlined refs
When we have non-inlined extent references, we were failing to find the
corresponding extent item for an existing csum item in the csum tree.

Reproducer:

   mkfs.btrfs -f /dev/sdd
   mount /dev/sdd /mnt

   xfs_io -f -c "falloc 780366 135302" /mnt/foo
   xfs_io -c "falloc 327680 151552" /mnt/foo
   xfs_io -c "pwrite -S 0xff -b 131072 0 131072" /mnt/foo
   sync

   for i in `seq 1 40`; do btrfs subvolume snapshot /mnt /mnt/snap$i ; done
   umount /mnt

   btrfs check /dev/sdd

The check command exited with status 1 and the following output:

   Checking filesystem on /dev/sdd
   UUID: 2416ab5f-9d71-457e-bb13-a27d4f6b399a
   checking extents
   checking free space cache
   checking fs roots
   checking csums
   There are no extents for csum range 12980224-12984320
   Csum exists for 12980224-12984320 but there is no extent record
   found 1388544 bytes used err is 1
   total csum bytes: 132
   total tree bytes: 704512
   total fs tree bytes: 573440
   total extent tree bytes: 16384
   btree space waste bytes: 564479
   file data blocks allocated: 19341312
    referenced 14606336
   Btrfs v3.14.1-94-g80597e7

After this change it no longer erroneously reports a missing extent for the
csum item and exits with a status of 0.

Also added missing btrfs_prev_leaf() return value checks, as we were ignoring
errors and non-existence of left siblings completely.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:29 +02:00
Wang Shilong 8909f11042 btrfs-progs: fsck: add ability to check reloc roots
When encountering system crash or balance enospc errors,
there maybe still some reloc roots left.

The way we store reloc root is different from fs root:

reloc root's root key(BTRFS_RELOC_TREE_OBJECTID, ROOT_ITEM, objectid)
fs root's root key(objectid, ROOT_ITEM, -1)
reloc data's root key(BTRFS_DATA_RELOC_TREE_OBJECTID, ROOT_ITEM, 0)

So this patch use right key to search corresponding root node, and
avoid using normal fs root cache for reloc roots.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:26 +02:00
Wang Shilong 1d0a3eac0c btrfs-progs: fsck: finish transaction commit if repair error out
If btrfsck fail to repair, we hit something like following:

Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
read block failed check_tree_block
found 98304 bytes used err is 1
total csum bytes: 0
total tree bytes: 0
total fs tree bytes: 0
total extent tree bytes: 0
btree space waste bytes: 0
file data blocks allocated: 0
referenced 0
Btrfs v3.14.2-rc2-63-g3944f15
btrfs: transaction.h:38: btrfs_start_transaction: Assertion `!(root->commit_root)' failed.
Aborted (core dumped)

This is because under repair mode, we will start a transaction, and if we error out,
we don't finish this transaction. So in close_ctree(), it will try
to start and commit transaction which causes the above segmentation.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:23 +02:00
Wang Shilong f495a2ac66 btrfs-progs: fsck: remove unfriendly BUG_ON() for searching tree failure
Now btrfsck would hit assertation failure for some searching tree failure.
It is true that filesystem may get some metadata block corrupted,
and btrfsck could not deal with these corruptings. But, Users really
don't want a BUG_ON() here, Instead, just return errors to caller.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:21 +02:00