Commit Graph

196 Commits (1e8246a0582bfbe34045a6dc665396f327b82d1f)

Author SHA1 Message Date
Qu Wenruo ad03f840f0 btrfs-progs: Add repair and report function for orphan file extent.
In some fs tree leaf/node corruption case, file extents may be lost, but
in extent tree, its record may still exists.

This provide the possibility for such orphan file extents to be
recovered even we can't recover its compression and other info, we can
still insert it as a normal non-compression file extent.

This patch provides the repair and report function for such orphan file
extent.
Even after such repair, user may still need to try to decompress its
data if user knows that is a compressed extent.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-02-02 14:16:38 +01:00
Qu Wenruo bb06029f9b btrfs-progs: Record and report every file extent hole.
Record every file extent discontinuous hole in inode_record using a
rb_tree member.

Before the patch, btrfsck will only record the first file extent hole by
using first_extent_gap, that's good for detecting error, but not
suitable for fixing it.

This patch provides the ability to record every file extent hole and
report it.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-02-02 14:16:38 +01:00
Qu Wenruo 0ddf63c09f btrfs-progs: Record orphan data extent ref to corresponding root.
Before this patch, when a extent's data ref points to a invalid key in
fs tree, this happens if a leaf/node of fs tree is corrupted, btrfsck
can't do any repair and just exit.

In fact, such problem can be handled in fs tree repair routines, rebuild
the inode item(if missing) and add back the extent data (with some
assumption).

So this patch records such data extent refs for later fs tree recovery
routine.

TODO:
    Restore orphan data extent refs into btrfs_root is not the best
    method. It's best to directly restore it into inode_record, however
    current extent tree and fs tree can't cooperate together, so use
    btrfs_root as a temporary storage until inode_cache is built.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-02-02 14:16:38 +01:00
Karel Zak 33f2d6b84d btrfs-progs: autoconf: use standard PACKAGE_* macros
- use standard PACKAGE_{NAME,VERSION,STRING,URL,...} autoconf macros
  rather than homemade BTRFS_BUILD_VERSION

- don't #include version.h, now the file is necessary for library API only

Note that "btrfs version" returns "btrfs-progs <version>" instead of
the original confusing "btrfs <version>".

Signed-off-by: Karel Zak <kzak@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-28 18:00:58 +01:00
Hugo Mills 82ecb9c660 btrfs-progs: check: Add --readonly flag
Particularly during support conversations, people get confused about
which options to use with btrfs check. Adding a flag, --readonly, which
implies the default read-only behaviour and which conflicts with the
read-write operations, should help make the behaviour of the tool clear.

Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-27 16:42:49 +01:00
Hugo Mills 7c126f7afc btrfs-progs: Make option parsing more robust to code modifications
The current approach to option parsing, where long-only options are
selected on the basis of their position in the long_options array is
fragile and painful to modify if options are to be inserted into the
list, rather than appended.

Instead, use the last field of struct option to return a value which
cannot be a char (and hence a short option), and simply switch on those
within the case statement.

Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-27 16:41:06 +01:00
Dimitri John Ledkov e69e015634 btrfs-progs: drop feature defines from C files, in favour of CFLAGS defines
glibc 2.10+ (5+ years old) enables all the desired features:
_XOPEN_SOURCE 700, __XOPEN2K8, POSIX_C_SOURCE, DEFAULT_SOURCE; with a
single _GNU_SOURCE define in the makefile alone. For portability to
other libc implementations (e.g. dietlibc) _XOPEN_SOURCE=700 is also
defined.

This also resolves Debian bug report filed by Michael Tautschnig -
"Inconsistent use of _XOPEN_SOURCE results in conflicting
declarations". Whilst I was not able to reproduce the results, the
reported fact is that _XOPEN_SOURCE set to 500 in one set of files
(e.g. cmds-filesystem.c) generates/defines different struct stat from
other files (cmds-replace.c).

This patch thus cleans up all feature defines, and sets them at a
consistent level.

Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=747969
Signed-off-by: Dimitri John Ledkov <dimitri.j.ledkov@intel.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-27 14:45:59 +01:00
David Sterba 2b7cdab425 btrfs-progs: make getopt tables static const
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-19 13:44:49 +01:00
David Sterba ecbf1339a6 btrfs-progs: cleanup, move getop long options close to their use
Move long_option defintions just before getopt_long everywhere.

Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-19 13:30:06 +01:00
Qu Wenruo e960b3f057 btrfs-progs: Don't increase error count if the inode can be repaired.
The original check_inode_recs() will return -1 if found any error in a
inode_record. This is OK for original design since there is almost
nothing can repair at that time.

However more and more error from nlink mismatch to missing inode item
can be repaired in try_repair_inode(), check_inode_recs() should not
increase the error count if the inode can be repair.

With this patch, repair function for leaf-corruption will not return
error if all corruption inode can be recovered.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-09 19:09:47 +01:00
Qu Wenruo 0da591ddf2 btrfs-progs: Fix a extent buffer leak in count_csum_range().
The commit f495a2ac66 ("btrfs-progs: fsck: remove unfriendly BUG_ON()
for searching tree failure") is causing tons of extent buffer leak if some
csum mismatches in btrfsck.

This is caused by a misplaced btrfs_release_path(), fix it.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Satoru Takeuchi <takeuchi_satoru@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-09 18:34:39 +01:00
David Sterba 70607997ee btrfs-progs: fix leak of path in reset_balance
A path may leak after an unlikely error condition.

Resolves-coverity-id: 1199474
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-03 03:33:31 +01:00
David Sterba fd1e9ad6a8 btrfs-progs: check, handle error in fix_key_order
In case the buffer is corrupted and the for loop does not happen, we'd
return garbage. The caller retunrs -EIO in case of any corruption, use
that value in fix_key_order.

Resolves-coverity-id: 1246944
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-03 02:47:32 +01:00
David Sterba 01a9f4308b btrfs-progs: check, missing parens around compound block in find_normal_file_extent
Resolves-coverity-id: 1260248
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-30 15:48:55 +01:00
David Sterba 0665aaf5a3 btrfs-progs: check, fix path leak in error branch
Resolves-coverity-id: 1260250
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-30 15:48:54 +01:00
Qu Wenruo 0f38766b8f btrfs-progs: Fix a bug in reset_nlink() which may deletes the recovered file
In reset_nlink(), we believe rec->found_link as accurate number of the
valid links. However it only records the number of found DIR_ITEM, so we
can't use it as reliable value.

Before this patch, in some case, leaf corruption recovery will believe
there is a valid backref but don't add_link() since it can't find any
valid one and don't put it into the lost+found dir.
So the recovered inode will be considered as an orphan item without
orphan item and repair_inode_orphan_item() will add orphan item for it,
causing all the filename/filetype we recovered being a waste of time.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-29 17:13:41 +01:00
David Sterba 34350d11c1 btrfs-progs: check result of first_cache_extent
If the tree's empty, we'll get NULL and dereference it.

Resolves-Coverity-CID: 1248828
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-19 18:04:23 +01:00
Wang Shilong 1f728b1a51 Btrfs-progs, fsck: move root items repair after root rebuilding
If some critical roots are corrupt, reapr_root_items() will fail,
this is detected by fsck_tests.sh's extent rebuilding tests.

Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 15:42:52 +01:00
Wang Shilong b3375a7e8b Btrfs-progs: fsck: add ability to rebuild extent tree with snapshots
This patch makes us to rebuild a really corrupt extent tree with snapshots.
To implement this, we have to verify whether a block is FULL BACKREF.

This idea come from Josef Bacik:

1) We walk down the original tree, every eb we encounter has
btrfs_header_owner(eb) == root->objectid.  We add normal references
for this root (BTRFS_TREE_BLOCK_REF_KEY) for this root.  World peace
is achieved.

2) We walk down the snapshotted tree.  Say we didn't change anything
at all, it was just a clean snapshot and then boom.  So the
btrfs_header_owner(root->node) == root->objectid, so normal backref.
We walk down to the next level, where btrfs_header_owner(eb) !=
root->objectid, but the level above did, so we add normal refs for all
of these blocks.  We go down the next level, now our
btrfs_header_owner(parent) != root->objectid and
btrfs_header_owner(eb) != root->objectid.  This is where we need to
now go back and see if btrfs_header_owner(eb) currently has a ref on
eb.  If it does we are done, move on to the next block in this same
level, we don't have to go further down.

3) Harder case, we snapshotted and then changed things in the original
root.  Do the same thing as in step 2, but now we get down to
btrfs_header_owner(eb) != root->objectid && btrfs_header_owner(parent)
!= root->objectid.  We lookup the references we have for eb and notice
that btrfs_header_owner(eb) no longer refers to eb.  So now we must
set FULL_BACKREF on this extent reference and add a
SHARED_BLOCK_REF_KEY for this eb using the parent->start as the
offset.  And we need to keep walking down and doing the same thing
until we either hit level 0 or btrfs_header_owner(eb) has a ref on the
block.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 15:42:52 +01:00
Wang Shilong 41beb7d4a7 Btrfs-progs: fsck: deal with snapshot one by one when rebuilding extent tree
Previously, we deal with node block firstly and then leaf block which can
maximize readahead. However, to rebuild extent tree, we need deal with snapshot
one by one.

This patch makes us deal with snapshot one by one if we need rebuild extent
tree otherwise we drop into previous way.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 15:42:52 +01:00
Qu Wenruo 55fea7186c btrfs-progs: Add inode item rebuild function.
Add a basic inode item rebuild function for I_ERR_NO_INODE_ITEM.
The main use case is to repair btrfs which fs root has corrupted leaf,
but it is already working for case if the corrupteed fs root leaf/node
contains no inode extent_data.

The repair needs 3 elements for inode rebuild:
1. inode number
   This is quite easy, existing inode_record codes will detect it quite
   well.

2. inode type
   This is the trick part. The only reliable method is to recovery it from
   parent's dir_index/item.
   The remaining method will search for regular file extent for FILE
   type or child's backref for DIR(todo).
   Fallback will be FILE.

Inode name(inode_ref) will be recoverd by nlink repair function.

This is just a fundamental implement, some advanced recovery can be
improved later with btrfs-progs infrastructure change.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:52:28 +01:00
Qu Wenruo 1581d7e5db btrfs-progs: Recover btree by dropping corrupted leaf/node.
Current btrfsck can skip corrupted leaf and even repair some corrupted
one if their bytenr or key order is wrong.

However when it comes to csum error leaf, btrfsck can only skip them,
which is OK for read-only iteration, but is bad for write.

This patch introduce the new repair_btree() function to recover the
btree, deleting all the corrupted leaf/node including corresponding
extent, allowing later write into the btree.

This patch provides the basis for later recovery with corrupted leaves.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:50:28 +01:00
Qu Wenruo 887cfa14ad btrfs-progs: record and report leaf/node corruption in fs/subvol tree
When leaf/node is corrupted in fs/subvolume root, btrfsck can ignore it
without much pain except some stderr messages complaining about it.
But this works fine doing read-only works, if we want to do deeper
recovery like rebuild missing inodes in the b+tree, it will cause
problem.

At least, info user that there is something wrong in the btree,
and this patch provides the base for later btree repair.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:44:53 +01:00
Qu Wenruo 61d7fc4ee6 btrfs-progs: Add fixing function for inodes whose nlink dismatch
[BUG]
At least two users have already hit a bug in btrfs causing file
missing(Chromium config file).
The missing file's nlink is still 1 but its backref points to non-exist
parent inode.

This should be a kernel bug, but btrfsck fix is needed anyway.

[FIX]
For such nlink mismatch inode, we will reset all the inode_ref with its
dir_index/item (including valid one), and re-add the valids.
If there is no valid backref for it, create 'lost+found' under the root
of the subvolume and add link to the directory.

Reported-by: Mike Gavrilov <mikhail.v.gavrilov@gmail.com>
Reported-by: Ed Tomlinson <edt@aei.ca>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:44:53 +01:00
Qu Wenruo 15bc913158 btrfs-progs: Add helper function find_file_name/type for nlink and inode_item repair.
Add find_file_name() and find_file_type() function for later nlink and
inode_item repair.

Later nlink repair will use both function and and inode_item repair will
use find_file_type().

They are done by searching the backref list, dir_item/index for type
search and dir_item/index or inode_ref for name search.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:44:53 +01:00
Qu Wenruo 0cc75eddd0 btrfs-progs: Add btrfs_unlink() and btrfs_add_link() functions.
Add btrfs_unlink() and btrfs_add_link() functions in inode.c,
for the incoming btrfs_mkdir() and later inode operations functions.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:13:06 +01:00
Qu Wenruo ed3a206c8f btrfs-progs: Record highest inode number before repair.
Record highest inode number before inode repair.

This is especially important for corrupted leaf case.
Under that case, if use btrfs_find_free_objectid, it may find a ino
existing in corrupted leaf but dropped by btree_recover.
If that happens, created dir will be referenced incorrectly since there
may be inode_ref or dir_index/item refers to it.

So we must record the highest inode number according to the inode_cache.
Inode_cache is OK since when a inode_ref or dir_index/item is found even
the referenced source is not found, it will be created.
If we record the highest inode number of inode_cache, and use
highest_inode + 1 as 'lost+found' dir, it will ensure the newly created
dir not conflicting with any possible inode.

This provides the basis for nlink or inode rebuild for repairing btrfs
with leaf/node corruption.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 11:55:17 +01:00
Josef Bacik 3a98fe43f6 Btrfs-progs: allow fsck to take the tree bytenr
Sometimes we have a pretty corrupted fs but have an old tree bytenr that we
could use, add the ability to specify the tree root bytenr.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Tested-by: Ansgar Hockmann-Stolle <ansgar.hockmann-stolle@uni-osnabrueck.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-04 17:03:56 +01:00
Qu Wenruo 0ffacad290 btrfs-progs: rebuild missing block group during chunk recovery if possible
Before the patch, chunk will be considered bad if the corresponding
block group is missing, even the only uncertain data is the 'used'
member of the block group.

This patch will try to recalculate the 'used' value of the block group
and rebuild it.
So even only chunk item and dev extent item is found, the chunk can be
recovered.
Although if extent tree is damanged and needed extent item can't be
read, the block group's 'used' value will be the block group length, to
prevent any later write/block reserve damaging the block group.
In that case, we will prompt user and recommend them to use
'--init-extent-tree' to rebuild extent tree if possible.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-04 16:48:13 +01:00
Qu Wenruo 1102b3483b btrfs-progs: print root dir verbose error in fsck
Before this patch, when btrfsck found an error in root dir, it will only
output the following message "root %llu root dir %llu error" without any
detailed error.

Just add print_inode_error() to print out the whole error.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-04 14:37:44 +01:00
Josef Bacik e4e7058173 Btrfs-progs: create missing root dirid
If we just don't have the root dirid stuff go ahead and re-create it, since it
is easily recreated.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-11-14 11:01:38 +01:00
Josef Bacik 9151febadd Btrfs-progs: fix missing inode items
If we have all the other items but no inode item we can recreate it for the most
part, with the exception of the permissions and ownership.  Add this ability to
btrfsck.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-11-14 11:01:24 +01:00
Josef Bacik 2dc4c0015f Btrfs-progs: add ability to replace missing dir item/dir indexes
If we have everything except the dir item and dir index we can easily replace
them, so add this ability to btrfsck.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-11-14 11:00:43 +01:00
Qu Wenruo 0f70b67a0c btrfs-progs: Fix the argument requirement for '--subvol-extents'
The following commit changed the argument requirement for
'--subvol-extents', which causes it to call arg_strtou64() on NULL,
resulting a segfault.
d34cbe76 btrfs-progs: check: do not require argument for --subvol-extents

This patch revert the patch and change the help string and man page to
make it no longer confusing.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-11-03 18:35:31 +01:00
Filipe Manana 555b7feaeb Btrfs-progs: check, ability to detect and fix outdated snapshot root items
This change adds code to detect and fix the issue introduced in the kernel
release 3.17, where creation of read-only snapshots lead to a corrupted
filesystem if they were created at a moment when the source subvolume/snapshot
had orphan items. The issue was that the on-disk root items became incorrect,
referring to the pre orphan cleanup root node instead of the post orphan
cleanup root node.

A test filesystem can be generated with the test case recently submitted for
xfstests/fstests, which is essencially the following (bash script):

    workout()
    {
	ops=$1
	procs=$2
	num_snapshots=$3

	_scratch_mkfs >> $seqres.full 2>&1
	_scratch_mount

	snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
	snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
	run_check $FSSTRESS_PROG -p $procs \
	    -x "$snapshot_cmd" -X $num_snapshots -d $SCRATCH_MNT -n $ops
    }

    ops=10000
    procs=4
    snapshots=500
    workout $ops $procs $snapshots

Example of btrfsck's (btrfs check) behaviour against such filesystem:

  $ btrfsck /dev/loop0
  root item for root 311, current bytenr 44630016, current gen 60, current level 1, new bytenr 44957696, new gen 61, new level 1
  root item for root 1480, current bytenr 1003569152, current gen 1271, current level 1, new bytenr 1004175360, new gen 1272, new level 1
  root item for root 1509, current bytenr 1037434880, current gen 1300, current level 1, new bytenr 1038467072, new gen 1301, new level 1
  root item for root 1562, current bytenr 33636352, current gen 1354, current level 1, new bytenr 34455552, new gen 1355, new level 1
  root item for root 3094, current bytenr 1011712000, current gen 2935, current level 1, new bytenr 1008484352, new gen 2936, new level 1
  root item for root 3716, current bytenr 80805888, current gen 3578, current level 1, new bytenr 73515008, new gen 3579, new level 1
  root item for root 4085, current bytenr 714031104, current gen 3958, current level 1, new bytenr 716816384, new gen 3959, new level 1
  Found 7 roots with an outdated root item.
  Please run a filesystem check with the option --repair to fix them.

  $ echo $?
  1

  $ btrfsck --repair /dev/loop0
  enabling repair mode
  fixing root item for root 311, current bytenr 44630016, current gen 60, current level 1, new bytenr 44957696, new gen 61, new level 1
  fixing root item for root 1480, current bytenr 1003569152, current gen 1271, current level 1, new bytenr 1004175360, new gen 1272, new level 1
  fixing root item for root 1509, current bytenr 1037434880, current gen 1300, current level 1, new bytenr 1038467072, new gen 1301, new level 1
  fixing root item for root 1562, current bytenr 33636352, current gen 1354, current level 1, new bytenr 34455552, new gen 1355, new level 1
  fixing root item for root 3094, current bytenr 1011712000, current gen 2935, current level 1, new bytenr 1008484352, new gen 2936, new level 1
  fixing root item for root 3716, current bytenr 80805888, current gen 3578, current level 1, new bytenr 73515008, new gen 3579, new level 1
  fixing root item for root 4085, current bytenr 714031104, current gen 3958, current level 1, new bytenr 716816384, new gen 3959, new level 1
  Fixed 7 roots.
  Checking filesystem on /dev/loop0
  UUID: 2186e9b9-c977-4a35-9c7b-69c6609d4620
  checking extents
  checking free space cache
  cache and super generation don't match, space cache will be invalidated
  checking fs roots
  checking csums
  checking root refs
  found 618537000 bytes used err is 0
  total csum bytes: 130824
  total tree bytes: 601620480
  total fs tree bytes: 580288512
  total extent tree bytes: 18464768
  btree space waste bytes: 136939144
  file data blocks allocated: 34150318080
   referenced 27815415808
  Btrfs v3.17-rc3-2-gbbe1dd8

  $ echo $?
  0

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-17 18:24:54 +02:00
Filipe Manana d6d44779f5 Btrfs-progs: check, fix return value check of is_child_root()
The following commit:

    "btrfs-progs: fsck: remove unfriendly BUG_ON() for searching tree failure"
    f495a2ac66

introduced a regression, detected through xfstests/btrfs/054, where
previously a negative return value (-1) was used to mean a particular
root didn't had any parent root, and now, after that change, a negative
value is also used to mean that an error happened. That change also made
the only caller of is_child_root() interpret any negative return value
as an error and therefore incorrectly made the caller leave with an
error, instead of continuing.

This affects only the 3.17 release candidates (3.16 and older releases
don't have this issue).

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-16 12:01:55 +02:00
Josef Bacik e3966ec9d0 Btrfs-progs: lookup all roots that point to a corrupt block
If we have a corrupt block that multiple snapshots point to we will only fix the
guy who originally pointed to the block, and then simply loop forever because we
keep finding the same bad block.  So instead lookup all roots that point to this
block, and then search down to the block for each root and fix the block in all
snapshots.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 11:11:58 +02:00
Josef Bacik 01f868b5a9 btrfs-progs: make fsck deal with bogus items
We can deal with corrupt items by deleting them in a few cases.  Fsck can easily
recover from a missing extent item or a dir index item.  So if we notice a item
is completely bogus and it is of a key that we know we can repair then just
delete it and carry on.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 11:10:36 +02:00
Josef Bacik f738a758af btrfs-progs: check blocks when checking fs roots
Usually if we find a bad block during the extent tree stuff we will error out,
but if the bad block is in an fs tree and doens't have extents in it then fsck
may still pass even though the block was complete garbage.  So add the check
block logic to the fs root checking so we actually error out of fsck if there is
a bad block.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 11:06:31 +02:00
Josef Bacik 187b85436b btrfs-progs: add the ability to fix shifted item offsets
A user had a corrupted fs where the items had been shifted improperly.  This
patch adds the ability to fix this sort of problem within fsck.  We will simply
shift the item over to the proper offset and update the offsets to make sure
they are correct.  I tested this with a hand crafted fs that was broken in the
same way as the user, and I've included the file as a new test.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 11:04:08 +02:00
Josef Bacik c2e686d76a Btrfs-progs: deal with mismatch index between dir index and inode ref
Sometimes we have a dir index and an inode ref that don't agree on the index.
In this case just assume that the inode ref is the ultimate authority on the
subject and delete the dir index.  This means we have to not reset index if we
find a mismatched inode ref to make sure we delete the right dir index.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:47:59 +02:00
Josef Bacik bb561f5485 Btrfs-progs: add a dummy backref if our location is wrong
If our location is bogus in our dir item we were just skipping the thing.
However in this case we want to just delete the dir index, so create a dummy
inode rec using BTRFS_MULTIPLE_OBJECTIDS and just add every backref we find to
the list so we know to straight up delete all of these items.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:45:25 +02:00
Josef Bacik b25a40651d Btrfs-progs: delete bogus dir indexes
We may run across dir indexes that are corrupt in such a way that it makes them
useless, such as having a bad location key or a bad name.  In this case we can
just delete dir indexes that don't show up properly and then re-create what we
need.  When we delete dir indexes however we need to restart scanning the fs
tree as we could have greated bogus inode recs if the location key was bad, so
set it up so that if we had to delete an dir index we go ahead and free up our
inode recs and return -EAGAIN to check_fs_roots so it knows to restart the loop.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:45:03 +02:00
Josef Bacik 994ce2672d Btrfs-progs: re-search tree root if it changes
If we change something while scanning fs-roots we need to redo our search so
that we get valid root items and have valid root cache.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:41:51 +02:00
Josef Bacik a8e9c9a65b Btrfs-progs: reset chunk state if we restart check
If we hid a corrupt block that we fix and we restart the fsck loop you will get
lots of noise about duplicate block groups and such.  This is because we don't
clear the block group and chunk cache when we do this restart.  This patch fixes
that, which is a little tricky since the structs are linked together with
various linked lists, but this passed with a user who was hitting this problem.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:40:22 +02:00
Josef Bacik cdb9e22e29 Btrfs-progs: break out rbtree util functions
These were added to deal with duplicated functionality within btrfs-progs, but
we specifically copied rbtree.c from the kernel, so move these functions out
into their own file.  This will make it easier to keep rbtree.c in sync.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:39:39 +02:00
Josef Bacik 6863bcf74f Btrfs-progs: repair missing dir index
If we have an inode backref entry then we know enough to add back a missing dir
index.  When messing with the inode backrefs we need to do all of that first
before we process the inode recs themselves as we may clear errors on the inode
recs as we fix the directory indexes.  This adds the framework for fixing
backref errors and fixes missing dir index issues.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-13 23:20:44 +02:00
Alexandre Oliva 12adc878ae btrfs-progs: check: do not dereference tree_refs as data_refs
In a filesystem corrupted by a faulty memory module, btrfsck would get
very confused attempting to access backrefs that weren't data backrefs
as if they were.  Besides invoking undefined behavior for accessing
potentially-uninitialized data past the end of objects, or with
dynamic types unrelated with the static types held in the
corresponding memory, it used offsets and lengths from such fields
that did not correspond to anything in the filesystem proper.

Moving the test for full backrefs and checking that they're data
backrefs earlier avoided the crash I was running into, but that was
not enough to make the filesystem complete a successful repair.

Signed-off-by: Alexandre Oliva <oliva@gnu.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 18:11:33 +02:00
Alexandre Oliva ca1e04597a btrfs-progs: repair: remove recowed entry from the to-recow list
If we attempt to repair a filesystem with metadata blocks that need
recowing, we'll get into an infinite loop repeatedly recowing the
first entry in the list, without ever removing it from the list.
Oops.  Fixed.

Signed-off-by: Alexandre Oliva <oliva@gnu.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 18:11:32 +02:00
Wang Shilong ed1b086a30 Btrfs-progs: fsck: deal with corrupted csum root
If checksum root is corrupted, fsck will get segmentation. This
is because if we fail to load checksum root, root's node is NULL which
cause NULL pointer deferences later.

To fix this problem, we just did something like extent tree rebuilding.
Allocate a new one and clear uptodate flag. We will do sanity check
before fsck going on.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 18:11:32 +02:00
Wang Shilong 8d2abcb6cc Btrfs-progs: fsck: only allow partial opening under repair mode
The reason that we allow partial opening is that sometimes,
we may have some corrupted trees.(for example extent tree), for
fsck repair case, the broken tree may be rebuilt later.

So if users only want to do check but not repair anything, this
patch will make fsck return failure as soon as possible and
tell users that some critial roots have been corrupted.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 18:11:32 +02:00
Qu Wenruo 0cf497719c btrfs-progs: Check the consistence between the parent node and child node/leaf.
When btrfs-progs walk down the tree, it does not check whether the child
node/leaf is valid.
In fact, there is some corrupted image whose csum is all valid but
parent node points to a invalid leaf.

In my case, the parent node in fs tree point to a invalid leaf(gen 11),
whose generation(15) and first key(EXTENT_TREE ROOT_ITEM 0) is
completely invalid, and will cause BUG_ON in process_inode_item().

Unfortunately, we are unable to fix when it happens.
So we can only output meaningful error message and avoid the insane
node/leaf, which is still much better than the original BUG_ON().

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:38 +02:00
Josef Bacik 9ecbbe30ec btrfs-progs: rebuild the crc tree with --init-csum-tree
We have --init-csum-tree, which just empties the csum tree.  I'm not sure why we
would ever need this, but we definitely need to be able to rebuild the csum tree
in some cases.  This patch adds the ability to completely rebuild the crc tree
by reading all of the data and adding csum entries for them.  This patch doesn't
pay attention to NODATASUM inodes, it'll happily add csums for everything.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:34 +02:00
Filipe David Borba Manana 6c9caa4dd6 btrfs-progs: check, fix csum check in the presence of non-inlined refs
When we have non-inlined extent references, we were failing to find the
corresponding extent item for an existing csum item in the csum tree.

Reproducer:

   mkfs.btrfs -f /dev/sdd
   mount /dev/sdd /mnt

   xfs_io -f -c "falloc 780366 135302" /mnt/foo
   xfs_io -c "falloc 327680 151552" /mnt/foo
   xfs_io -c "pwrite -S 0xff -b 131072 0 131072" /mnt/foo
   sync

   for i in `seq 1 40`; do btrfs subvolume snapshot /mnt /mnt/snap$i ; done
   umount /mnt

   btrfs check /dev/sdd

The check command exited with status 1 and the following output:

   Checking filesystem on /dev/sdd
   UUID: 2416ab5f-9d71-457e-bb13-a27d4f6b399a
   checking extents
   checking free space cache
   checking fs roots
   checking csums
   There are no extents for csum range 12980224-12984320
   Csum exists for 12980224-12984320 but there is no extent record
   found 1388544 bytes used err is 1
   total csum bytes: 132
   total tree bytes: 704512
   total fs tree bytes: 573440
   total extent tree bytes: 16384
   btree space waste bytes: 564479
   file data blocks allocated: 19341312
    referenced 14606336
   Btrfs v3.14.1-94-g80597e7

After this change it no longer erroneously reports a missing extent for the
csum item and exits with a status of 0.

Also added missing btrfs_prev_leaf() return value checks, as we were ignoring
errors and non-existence of left siblings completely.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:29 +02:00
Wang Shilong 8909f11042 btrfs-progs: fsck: add ability to check reloc roots
When encountering system crash or balance enospc errors,
there maybe still some reloc roots left.

The way we store reloc root is different from fs root:

reloc root's root key(BTRFS_RELOC_TREE_OBJECTID, ROOT_ITEM, objectid)
fs root's root key(objectid, ROOT_ITEM, -1)
reloc data's root key(BTRFS_DATA_RELOC_TREE_OBJECTID, ROOT_ITEM, 0)

So this patch use right key to search corresponding root node, and
avoid using normal fs root cache for reloc roots.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:26 +02:00
Wang Shilong 1d0a3eac0c btrfs-progs: fsck: finish transaction commit if repair error out
If btrfsck fail to repair, we hit something like following:

Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
Check tree block failed, want=29442048, have=0
read block failed check_tree_block
found 98304 bytes used err is 1
total csum bytes: 0
total tree bytes: 0
total fs tree bytes: 0
total extent tree bytes: 0
btree space waste bytes: 0
file data blocks allocated: 0
referenced 0
Btrfs v3.14.2-rc2-63-g3944f15
btrfs: transaction.h:38: btrfs_start_transaction: Assertion `!(root->commit_root)' failed.
Aborted (core dumped)

This is because under repair mode, we will start a transaction, and if we error out,
we don't finish this transaction. So in close_ctree(), it will try
to start and commit transaction which causes the above segmentation.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:23 +02:00
Wang Shilong f495a2ac66 btrfs-progs: fsck: remove unfriendly BUG_ON() for searching tree failure
Now btrfsck would hit assertation failure for some searching tree failure.
It is true that filesystem may get some metadata block corrupted,
and btrfsck could not deal with these corruptings. But, Users really
don't want a BUG_ON() here, Instead, just return errors to caller.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:21 +02:00
Wang Shilong adfe8b5cfb btrfs-progs: fsck: clear out log tree in repair mode
Repair mode will commit transaction which will make us
fail to load log tree anymore.

Give a warning to common users, if they really want to
coninue, we will clear out log tree.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:18 +02:00
Wang Shilong 897c98de73 btrfs-progs: fsck: avoid pinning same block several times
This can not only give some speedups but also avoid forever loop
with a really broken filesystem.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-10 10:38:16 +02:00
Qu Wenruo 2c1b155988 btrfs-progs: Check the csum tree node before go through the csum tree
[BUG]
Some fsfuzzed btrfs image will cause btrfsck segfault.

[REPRODUCER]
Run btrfsck on a csum tree block corrupted image.

[REASON]
check_csums() function call btrfs_search_slot() on csum_tree but doesn't
check whether the csum_tree contains a valid extent_buffer, which causes
the segfault.

[FIX]
Check the csum_root->node before any search.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-01 16:35:33 +02:00
Josef Bacik 3fce2fa9a1 btrfs-progs: add root to dirty list when fixing bad keys
A user reported a WARN_ON() when trying to run btrfsck --repair on his fs with
bad key ordering.  This was because the root that was broken wasn't part of the
transaction yet.  We do this open coded thing in a few other places in fsck, so
just make it a helper function and make sure all the places that need to call it
do call it.  With this patch he was able to run repair without it dying.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-01 16:35:18 +02:00
Zach Brown 977f2baf36 btrfs-progs: kill BUG_ON in readahead_tree_block()
David sent a quick patch that removed a BUG_ON().  I took a peek and
found that the function was already leaking an eb ref and only returned
0.  So this fixes the leak and makes the function void and fixes up the
callers.

Accidentally-motivated-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Zach Brown <zab@zabbo.net>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-09-14 13:19:44 +02:00
David Sterba d34cbe76c7 btrfs-progs: check: do not require argument for --subvol-extents
$ btrfs check --subvol-extents /dev/sdx
ERROR: /dev/sdx is not a valid numeric value.

Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-22 16:09:56 +02:00
Gui Hecheng f1672e50c7 btrfs-progs: use check_argc_* to check arg number for all tools
Since this patch:
	btrfs-progs: move the check_argc_* functions into utils.c

All tools including the independent tools(e.g. btrfs-image, btrfs-convert)
can share the convenience of the check_argc_* functions, so this patch
adopt the argc check functions globally.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-22 15:07:02 +02:00
Mark Fasheh bc70abad8c btrfs-progs: show extent state for a subvolume
The qgroup verification code can trivially be extended to provide
extended information on the extents which a subvolume root
references. Along with qgroup-verify, I have found this tool to be
invaluable when tracking down extent references.

The patch adds a switch to the check subcommand '--subvol-extents'
which takes as args a single subvolume id. When run with the switch,
we'll print out each extent that the subvolume references. The extent
printout gives standard extent info you would expect along with
information on which other roots reference it.

Sample output follows - this is a few lines from a run on a subvolume
I've been testing qgroup changes on:

Print extent state for subvolume 281 on /dev/vdb2
UUID: 8203ca66-9858-4e3f-b447-5bbaacf79c02
Offset		Len	Root Refs	Roots
12582912	20480	12	257 279 280 281 282 283 284 285 286 287 288 289
12603392	8192	12	257 279 280 281 282 283 284 285 286 287 288 289
12611584	12288	12	257 279 280 281 282 283 284 285 286 287 288 289
 <snip a bunch of extents to show some variety>
124583936	16384	4	281 282 283 280
125075456	16384	4	280 281 282 283
126255104	16384	11	257 280 281 282 283 284 285 286 287 288 289
4763508736	4096	3	279 280 281

In case it wasn't clear, this applies on top of my qgroup verify patch:
"btrfs-progs: add quota group verify code"

A branch with all this can be found on github:

https://github.com/markfasheh/btrfs-progs-patches/tree/qgroup-verify

Please apply,

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-22 15:04:16 +02:00
Wang Shilong 5e94319406 Btrfs-progs: fsck: reduce memory usage for extents check
Steps to reproduce:

 # mkfs.btrfs -f /dev/sda9 -b 2g
 # mount /dev/sda9 /mnt
 # dd if=/dev/zero of=/mnt/data bs=4k oflag=direct
 # btrfs file df /mnt

    Data, single: total=1.66GiB, used=1.66GiB
    System, single: total=4.00MiB, used=16.00KiB
    Metadata, single: total=200.00MiB, used=67.88MiB

For a filesystem without snapshots, 70M metadata, extent
checking eats max memory about 110M, this is a nightmare
for some system with low memory.

It is very likely that extent record can be freed quickly
for a filesystem without snapshots, improve this by trying
if it can free memory after adding data/tree backrefs.

This patch reduces max memory cost from 110M to 40M for
extents checking for the above case.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-22 14:43:11 +02:00
Hugo Mills b5fc0b9014 btrfs-progs: check: Fix wrong level access
There's no reason to assume that the bad key order is in a leaf block,
so accessing level 0 of the path is going to be an error if it's actually
a node block that's bad.

Reported-by: Chris Mason <clm@fb.com>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-22 14:43:10 +02:00
Wang Shilong 6d6f2d097e Btrfs-progs: fsck: switch to is_fstree()
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-22 14:43:09 +02:00
Wang Shilong 1bad2f2f2d Btrfs-progs: fsck: add an option to check data csums
This patch adds an option '--check-data-csum' to verify data checksums.
fsck won't check data csums unless users specify this option explictly.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-22 14:39:33 +02:00
Mark Fasheh 96ec888aad btrfs-progs: add quota group verify code
This patch adds functionality (in qgroup-verify.c) to compute bytecounts in
subvolume quota groups. The original groups are read in and stored in memory
so that after we compute our own bytecounts, we can compare them with those
on disk. A print function is provided to do this comparison and show the
results on the console.

A 'qgroup check' pass is added to btrfsck. If any subvolume quota groups
differ from what we compute, the differences for them are printed.  We also
provide an option '--qgroup-report' which will run only the quota check code
and print a report on all quota groups.  Other than making it possible to
verify that our qgroup changes work correctly, this mode can also be used in
xfstests for automated checking after qgroup tests.

This patch does not address the following:
- compressed counts are identical to non compressed, because kernel doesn't
  make the distinction yet.  Adding the code to verify compressed counts
  shouldn't be hard at all though once kernel can do this.
- It is only concerned with subvolume quota groups (like most of
  btrfs-progs).

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-22 14:39:33 +02:00
Rakesh Pandit ad17d56084 Btrfs-progs: make smatch checker happy (trivial fixes)
It complains errno never gets assigned to zero in find-root and since
errno anyway is zero at program started up, lets remove it.

Check "copy is less then zero" isn't possible because strtoull used by
arg_strtou64 wouldn't return -ve number.

Trivial space fixes.

Signed-off-by: Rakesh Pandit <rakesh@tuxera.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-05-02 17:03:45 +02:00
Rakesh Pandit 275fd93826 Btrfs-progs: fsck: while checking root refs print readable errors
Lets use "errors" instead of "error" because more then one ref errors
are possible. Also print error messages for unresolved refs in
check_root_refs.

Signed-off-by: Rakesh Pandit <rakesh@tuxera.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-05-02 17:03:27 +02:00
Filipe David Borba Manana c17d0a73c1 Btrfs-progs: update btrfs_file_extent_inline_len to match kernel version
The following kernel commit changed the definition of the inline function
btrfs_file_extent_inline_len():

    commit 514ac8ad8793a097c0c9d89202c642479d6dfa34
    Author: Chris Mason <clm@fb.com>
    Date:   Fri Jan 3 21:07:00 2014 -0800

    Btrfs: don't use ram_bytes for uncompressed inline items

    If we truncate an uncompressed inline item, ram_bytes isn't updated to reflect
    the new size.  The fixe uses the size directly from the item header when
    reading uncompressed inlines, and also fixes truncate to update the
    size as it goes.

Not having this new definition implies that the restore tool might misbehave when
restoring files with an inline extent that got truncated on a kernel older than
release 3.14.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-04-11 19:47:29 +02:00
Wang Shilong 1d2b8a3f0b Btrfs-progs: fsck: fix wrong index in pick_next_pending()
Though all tree blocks have same size, we'd better use right
index here.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-04-11 19:05:40 +02:00
Wang Shilong 8bae5a1525 Btrfs-progs: fsck: reduce memory usage of extent record struct
Two changes:
	1.use bit filed for @found_rec
	2.u32 is enough to calculate duplicate extent number.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-04-11 19:05:40 +02:00
Wang Shilong c920facc18 Btrfs-progs: fsck: fix possible memory leaks in run_next_block()
We still need free allocated cache memory in case error happens.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-04-11 19:05:40 +02:00
Wang Shilong 30e72cc70d Btrfs-progs: fsck: don't free @seen cache until we finish searching
@seen cache is used to avoid iterating same block more than once, and
we can not free them until we have finished searching.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-04-11 19:05:39 +02:00
Rakesh Pandit bf357ff320 Btrfs-progs: fsck: fix memory leak and unnecessary call to free
Free already allocated memory to item1_data if malloc fails for
item2_data in swap_values. Seems to be a typo from commit 70749a77.

Signed-off-by: Rakesh Pandit <rakesh@tuxera.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 09:51:31 -07:00
Wang Shilong 8d2359c8d9 Btrfs-progs: fsck: handle case that we can not lookup extent info
Previously, --init-extent-tree works just because btrfs_lookup_extent_info()
blindly return 0, and this make it work if there are not any *FULL BACKREF*
mode in broken filesystem.

It is just a coincidence that --init-extent-tree option works, let's
do it in the right way firstly.

For now, we have not supported to rebuild extent tree if there are
any *FULL BACKREF* mode which means if there are snapshots with broken
filesystem, avoid using --init-extent-tree option now.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:39 -07:00
Wang Shilong 706a90b122 Btrfs-progs: fsck: force to udate tree root for some cases
commit roots won't update root item in tree root if it finds
updated root's bytenr is same as before.

However, this is not right for fsck, we need update tree root in
the following case:

1.overwrite previous root node.

2.reinit reloc data tree, this is because we skip pin relo data
 tree before which means we can allocate same block as before.

Fix this by updating tree root ourselves for the above cases.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:38 -07:00
Wang Shilong 5ec1876c2c Btrfs-progs: fsck: insert root dir into reloc data tree when reiniting it
There are two bugs when resetting balance:

 1.we will skip reinitting reloc data tree if no reloc root found, however
  this is not right because we don't pin reloc data tree before.

 2.we should insert root dir into reloc data tree,otherwise we will fail
  to fsck.

Fix problems by forcely reiniting reloc data root and inserting root dir.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:37 -07:00
Wang Shilong 1c1e57eb8c Btrfs-progs: fsck: reset balance after reiniting extent root
reset balance need cow block which will insert extent item into
extent tree. If we do this before reinitting extent root, we may
encounter EEIXST.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:35 -07:00
Wang Shilong bedef66c0b Btrfs-progs: fsck: deal with really corrupted extent tree
To reinit extent root, we need find a free extent, however,
we may have a really corrupted extent tree, so we can't rely
on existed extent tree to cache block group any more.

During test, we fail to reinit extent tree which is because we
can not find a free extent so let's make block group cache ourselves
firstly.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:34 -07:00
Josef Bacik 47c18d17c5 Btrfs-progs: record generation for tree blocks in fsck
When working with a user who had a broken file system I noticed that we were
reading a bad copy of a block when the other copy was perfectly fine.  This is
because we don't keep track of the parent generation for tree blocks, so we just
read whichever copy we damned well please with no regards for which is best.
This fixes this problem by recording the parent generation of the tree block so
we can be sure to read the most correct copy before we check it, which will give
us a better chance of fixing really broken filesystems.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:30 -07:00
Wang Shilong 6262e12206 Btrfs-progs: fsck: fix wrong return value in check_block()
We found btrfsck will output backrefs mismatch while the filesystem
is defenitely ok.

The problem is that check_block() don't return right value,which
makes btrfsck won't walk all tree blocks thus we don't get a consistent
filesystem, we will fail to check extent refs etc.

Reported-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:25 -07:00
Wang Shilong acdf338c13 Btrfs-progs: fsck: avoid overwritting existed space when initting csum tree
Steps to reproduce:
 # mkfs.btrfs -f /dev/sda9
 # btrfs check /dev/sda9 --init-extent-tree --init-csum-tree
 # btrfs check /dev/sda9

During reinitting extent tree, we will pin all metadata blocks to
avoid overwritting existing metadata space. However, those space will
be unpinned after committing transaction.

If we try to reinit csum tree after reiniting extent tree, we may
overwrite existing space. Fix this problem by making reinit extent tree
and csum tree in same transaction.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:24 -07:00
Wang Shilong 64fddab8ac Btrfs-progs: switch to arg_strtou64() part3
Switch to new helper arg_strtou64(), also check if user assign
a valid super copy.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:20 -07:00
Gui Hecheng 015fba13cd btrfs-progs: fix fsck leaks on error returns
Add close_ctree()s before the "returns" on errors after open_ctree()
Also merge the err returns into the "goto + single return" pattern.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:17 -07:00
Mitch Harder 26b2f58d78 btrfs-progs: Remove superfluous BUG_ON check.
The function call that set the ret parameter evaluated in this
BUG_ON was removed in a previous commit:
11be10f71e
Btrfs-progs: make fsck fix certain file extent inconsistencies

Signed-off-by: Mitch Harder <mitch.harder@sabayonlinux.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:14 -07:00
Mitch Harder 5218123d8f btrfs-progs: Preserve process_one_leaf return value.
The return value in process_one_leaf could be over-written while
looping over the items in the leaf.

This patch will preserve a non-zero return value to the calling
function if a non-zero return value is encountered in the loop.

The return value of one (1) is consistent with non-zero values
that could be returned while processing the leaf.

The only caller of this function (walk_down_tree) would ignore
the return value anyway.  But this patch will correct the
behaviour in case future changes intend to utilize the return
value.

Signed-off-by: Mitch Harder <mitch.harder@sabayonlinux.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:13 -07:00
Anand Jain f920dbce8d btrfs-progs: btrfsck operations should be exclusive
this patch will make btrfsck operations to open disk in exclusive mode,
so that mount will fail when btrfsck is running

Signed-off-by: Anand Jain <Anand.Jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:26 -08:00
Eryu Guan b2e99e1819 Btrfs-progs: check return value of read_tree_block() in check_chunks_and_extents()
The following steps could trigger btrfs segfault:

mkfs -t btrfs -m raid5 -d raid5 /dev/loop{0..3}
losetup -d /dev/loop2
btrfs check /dev/loop0

The reason is that read_tree_block() returns NULL and
add_root_to_pending() dereferences it without checking it first.

Also replace a BUG_ON with proper error checking.

Signed-off-by: Eryu Guan <guaneryu@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:25 -08:00
Josef Bacik 147114581d Btrfs-progs: add support for the no holes incompat flag
This adds the flag to ctree.h, adds the feature option to mkfs to turn it on and
fixes fsck so it doesn't complain about missing hole extents in files when this
flag is set.

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:23 -08:00
Josef Bacik 70749a77fe Btrfs-progs: deal with invalid key orderings and bad orphan items V2
A user had a fs where the objectid of an orphan item was not the actual orphan
item objectid.  This screwed up fsck because the block has keys in the wrong
order, also the fs scanning stuff will freak out because we have an inode with
nlink 0 and no orphan item.  So this patch is pretty big but is all related.

1) Deal with bad key ordering.  We can easily fix this up, so fix the checking
stuff to tell us exactly what it found when it said there was a problem.  Then
if it's bad key ordering we can reorder the keys and restart the scan.

2) Deal with bad keys.  If we find an orphan item with the wrong objectid it's
likely to screw with stuff, so keep track of these sort of things with a
bad_item list and just run through and delete any objects that don't make sense.
So far we just do this for orphan items but we could extend this as new stuff
pops up.

3) Deal with missing orphan items.  This is easy, if we have a file with i_nlink
set to 0 and no orphan item we can just add an orphan item.

4) Add the infrastructure to corrupt actual key values.  Needed this to create a
test image to verify I was fixing things properly.

This patch fixes the corrupt image I'm adding and passes the other make test
tests.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:15 -08:00
Josef Bacik d04707f787 Btrfs-progs: reset balance before reiniting extent root
When we re-init the extent root we make it completely empty, so when we reset a
pending balance we will fail to find refs for any blocks we may cow, which will
result in errors and we will exit out.  We need to reset the balance first so
the normal cow stuff doesn't freak out and then we can re-init the extent tree.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:14 -08:00
Anand Jain b94a3003ae btrfs-progs: fix backref after init-csum-tree
btrfsck reports backref error after running init-csum-tree

btrfsck --init-csum-tree /dev/sdc
btrfsck /dev/sdc
::
ref mismatch on [29474816 16384] extent item 1, found 0
Backref 29474816 root 7 not referenced back 0x1101d30
Incorrect global backref count on 29474816 found 1 wanted 0
backpointer mismatch on [29474816 16384]
owner ref check failed [29474816 16384]
Errors found in extent allocation tree or chunk allocation
::

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:10 -08:00
Ross Kirk 7ff512ce38 btrfs-progs: Make btrfs_header_chunk_tree_uuid() return unsigned long
Internally, btrfs_header_chunk_tree_uuid() calculates an unsigned
long, but casts it to a pointer, while all callers cast it to unsigned
long again.

From btrfs commit b308bc2f05a86e728bd035e21a4974acd05f4d1e

Signed-off-by: Ross Kirk <ross.kirk@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:04 -08:00
Anand Jain 1ecefced86 btrfs-progs: define BTRFS_UUID_UNPARSE_SIZE for uuid unparse buf size
we use 37 as the allocation size to hold the uuid_unparse, here
it defines BTRFS_UUID_UNPARSE_SIZE for the same.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:03 -08:00
Miao Xie b4f4473e8a Btrfs-progs: don't output baffling message when checking a fresh fs
As we know,  a new fs doesn't have space cache, so we set the cache generation
of the super block to be -1ULL, it is not equal to the fs generation. But the
check program didn't consider this case, and output the following message

  cache and super generation don't match, space cache will be invalidated

directly, it would be baffling the users. So we should avoid outputing such
message. This patch fixes this problem.

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:21:59 -08:00
Josef Bacik 1f5e3b2825 Btrfs-progs: allow --init-extent-tree to work when extent tree is borked
Unfortunately you can't run --init-extent-tree if you can't actually read the
extent root.  Fix this by allowing partial starts with no extent root and then
have fsck only check to see if the extent root is uptodate _after_ the check to
see if we are init'ing the extent tree.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:21:58 -08:00