Commit Graph

125 Commits (c06c5eef88d11bf58afdad918683e9382b4f7acc)

Author SHA1 Message Date
Qu Wenruo c06c5eef88 btrfs-progs: check/lowmem: Add inode mode check
There is one report about invalid free space cache inode mode.
Normally free space cache inode should have mode 100600 (regular file,
no uid/gid/sticky bit, rw------ bit).

But in that report, we have free space cache inode mode as 0.

So at least btrfs check should report invalid inode mode.

This patch will at least make btrfs check lowmem mode to detect this
problem.

Please note that, this check only applies to inodes in fs/subvol trees.
It doesn't apply to free space cache inodes.

Reported-by: Thorsten Hirsch <t.hirsch@web.de>
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:04:25 +08:00
Qu Wenruo db51d8d8f6 btrfs-progs: Use @fs_info to replace @root for btrfs_check_leaf/node()
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:04:25 +08:00
Su Yanjun cedbfc2561 btrfs-progs: check: Delete file extent item with unaligned disk bytenr
For test case fsck-tests/001-bad-file-extent-bytenr, we have an
obviously hand crafted image with unaligned file extent:

        item 7 key (257 EXTENT_DATA 0) itemoff 3453 itemsize 53
                generation 6 type 1 (regular)
                extent data disk byte 755944791 nr 1048576
                extent data offset 0 nr 1048576 ram 1048576
                extent compression 0 (none)

disk bytenr 755944791 is obviously unaligned (not even).

For such obviously corrupted file extent, we should just delete the file
extent.

Signed-off-by: Su Yanjun <suyj.fnst@cn.fujitsu.com>
[Update commit message and comment]
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:04:25 +08:00
Su Yanjun 3b35deeadd btrfs-progs: check: fix wrong @offset used in find_possible_backrefs()
Function find_possible_backrefs() is used to locate the file extents
referring to an data extent.

For data extent backref, its btrfs_extent_data_ref structure has
the following members:
- root
  Which root refers to this data extent

- objectid
  Which inode refers to this data extent

- offset
  Search *hint*.
  Its value is @file_offset - @extent_offset.

While for @file_offset, it's directly recorded in (INO EXTENT_DATA
FILE_OFFSET) key.

So when searching the file extents refers to this data extent, we can't
use btrfs_extent_data_ref::offset as search key::offset.

We must search from file offset 0, and iterate all file extents until we
hit a file extent matches the data backref.

Thankfully such time consuming behavior is not triggered frequently,
it only gets called for repair, so it shouldn't affect normal check
routine.

Signed-off-by: Su Yanjun <suyj.fnst@cn.fujitsu.com>
[Update commit message]
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:04:25 +08:00
Su Yanjun b6a0d97cba Revert "btrfs-progs: Record orphan data extent ref to corresponding root."
Commit 0ddf63c09f ("btrfs-progs: Record orphan data extent ref to
corresponding root.") introduces the ability to record a file extent
even all other related info is lost (data backref, inode item).

However this patch only records such info without doing any proper
repair, further more, it could even record invalid file extents, and the
report part only happens after all check is done.

Since we will later introduce proper file extent repair functionality,
we could revert that patch.

Signed-off-by: Su Yanjun <suyj.fnst@cn.fujitsu.com>
[Update commit message, solve merge conflicts]
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:03:51 +08:00
Su Yanjun 872c116c75 Revert "btrfs-progs: Add repair and report function for orphan file extent."
Commit ad03f840f0 ("btrfs-progs: Add repair and report function for
orphan file extent.") will record and try to repair orphan file extents
by:
- Removing the orphan file extent item if no extent backref can be found
Or
- Re-insert a file extent using data backref

Especially the later case is far from ideal, as normally extent tree is
more fragile and corruption prone.
Use any data from extent tree to try to repair could easily lead to
further corruption.

So here we revert commit ad03f840f0 ("btrfs-progs: Add repair and report
function for orphan file extent.") to cleanup the space for later proper
repair in original mode.

Signed-off-by: Su Yanjun <suyj.fnst@cn.fujitsu.com>
[Update commit message, solve conflicts with DIR_ITEM hash mismatch patchset]
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:03:02 +08:00
Su Yue 0617bde3bc btrfs-progs: lowmem: delete unaligned bytes extent data under repair
If found a extent data item has unaligned part, lowmem repair
just deletes it.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:02:13 +08:00
Su Yue a170ef86ff btrfs-progs: lowmem: rename delete_extent_tree_item() to delete_item()
The function can delete items in trees besides extent tree.
Rename and move it for further use.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
[Update comment, solve merge conflicts]
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:02:09 +08:00
Su Yue 2a058a2204 btrfs-progs: lowmem: check unaligned disk_bytenr for extent_data
Add support to check unaligned disk_bytenr for extent_data.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:02:08 +08:00
Su Yue cbe66f5804 btrfs-progs: lowmem: fix false alert if extent item has been repaired
Previously, @err are assigned immediately after check but before
repair.
repair_extent_item()'s return value also confuses the caller. If
error has been repaired and returns 0, check_extent_item() will try
to continue check the nonexistent and cause flase alerts.

Here make repair_extent_item()'s return codes only represents status
of the extent item, error bits are handled in caller of the repair
function.
Change of @err after repair.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
[Solve conflicts with DIR_ITEM hash mismatch patchset]
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:02:06 +08:00
Su Yue 0ea38ae08d btrfs-progs: lowmem: move nbytes check before isize check
For files, lowmem repair will try to check nbytes and isize,
but isize check depends nbytes.

Once bytes has been repaired, then isize should be checked and
repaired.
So move nbytes check before isize check. Also set nbytes to
extent_size once repaired successfully.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:02:04 +08:00
Su Yue ee455db07e btrfs-progs: lowmem: add argument path to punch_extent_hole()
Since repair will do CoW, the outer path may be invalid.

This patch will add an argument, @path, to punch_extent_hole().
When punch_extent_hole() returns, path will still point to the same key.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
[Update comment and commit message]
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:02:01 +08:00
Lu Fengqi fa13420cc7 btrfs-progs: lowmem: fix false alert about the existence of gaps in the check_file_extent
The 'end' parameter of check_file_extent tracks the ending offset of the
last checked extent. This is used to detect gaps between adjacent extents.

Currently such gaps are wrongly detected since for regular extents only
the size of the extent is added to the 'end' parameter. This results in
wrongly considering all extents of a file as having gaps between them
when only 2 of them really have a gap as seen in the example below.

Solution:
The extent_end variable should set to the sum of the offset and the
extent_num_bytes of the file extent.

Example:
Suppose that lowmem check the following file extent of inode 257.

        item 6 key (257 EXTENT_DATA 0) itemoff 15813 itemsize 53
                generation 6 type 1 (regular)
                extent data disk byte 13631488 nr 4096
                extent data offset 0 nr 4096 ram 4096
                extent compression 0 (none)
        item 7 key (257 EXTENT_DATA 8192) itemoff 15760 itemsize 53
                generation 6 type 1 (regular)
                extent data disk byte 13631488 nr 4096
                extent data offset 0 nr 4096 ram 4096
                extent compression 0 (none)
        item 8 key (257 EXTENT_DATA 12288) itemoff 15707 itemsize 53
                generation 6 type 1 (regular)
                extent data disk byte 13631488 nr 4096
                extent data offset 0 nr 4096 ram 4096
                extent compression 0 (none)

For inode 257, check_inode_item set extent_end to 0, then call
check_file_extent to check item {6,7,8}.
item 6)
	offset(0) == extent_end(0)
	extent_end = extent_end(0) + extent_num_bytes(4096)
item 7)
	offset(8192) != extent_end(4096)
	extent_end = extent_end(4096) + extent_num_bytes(4096)
			^^^
	The old extent_end should replace by offset(8192).
item 8)
	offset(12288) != extent_end(8192)
		^^^
	But there is no gap between item {7,8}.

Fixes: d88da10ddd ("btrfs-progs: check: introduce function to check file extent")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
[Move this patch as the 1st patch, since it's an independent fix]
Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:01:53 +08:00
Qu Wenruo e76fb663a7 btrfs-progs: check: Fix false alert about uninitialized variable
GCC 8.2.1 will report the following error:

  check/main.c: In function 'try_repair_inode':
  check/main.c:2606:5: warning: 'ret' may be used uninitialized in this function [-Wmaybe-uninitialized]
    if (!ret) {
       ^
  check/main.c:2584:6: note: 'ret' was declared here
    int ret;
        ^~~

The offending code is in repair_mismatch_dir_hash():

	int ret;

	printf(
	"Deleting bad dir items with invalid hash for root %llu ino %llu\n",
		root->root_key.objectid, rec->ino);
	while (!list_empty(&rec->mismatch_dir_hash)) {
		/* do some repair */
	}
	if (!ret) { <<< Here
		/* do some fix */
	}

The truth is, to enter try_repair_inode(), we must have
I_ERR_MISMATCH_DIR_HASH bit set for rec->errors.

And just after we set I_ERR_MISMATCH_DIR_HASH, we call
add_mismatch_dir_hash() and handled its error correctly.

So it's impossible to to skip the while loop.

Fix it by initializing @ret to -EUCLEAN, so even we hit some impossible
case, repair_mismatch_dir_hash() won't falsely consider the mismatch
hash fixed.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-03-05 12:57:41 +01:00
Jeff Mahoney 8bda5132d8 btrfs-progs: fix stray error message in check
Commit e578b59bf6 ("btrfs-progs: convert strerror to implicit %m")
missed adding braces after a conditional so we will see the following
message whenever a tree block needs repair, regardless of whether repair
was successful: "Failed to repair btree: Success"

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 18:42:14 +01:00
Qu Wenruo af816ca930 btrfs-progs: check: orig: Add ability to repair dir item with invalid hash
The repair function is reusing delete_corrupted_dir_item().

Since the error can happen for root dir inode, also call
try_repair_inode() on root dir inode.

This is especially important for old filesystems, since later kernel
introduces stricter tree-checker, which could detect such hash mismatch
and refuse to read the corrupted leaf.

With this repair ability, user could repair with btrfs check --repair.

Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1111991
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 18:42:13 +01:00
Qu Wenruo 846f838797 btrfs-progs: check: orig: Use mismatch_dir_hash_record to record bad dir items
This changes reporting from current in-place, like:

  ERROR: DIR_ITEM[256 751495445] name foor.WvG1c1TdU namelen 13 filetype 1 mismatch with its hash, wanted 751495445 have 2870353892
  root 5 root dir 256 error

To new summary report at the end of the pass:

  root 5 root dir 256 error
  root 5 inode 256 errors 40000
  Dir items with mismatch hash:
	  name: foor.WvG1c1Td namelen: 13 wanted 0xab161fe4 has 0x2ccae915

Also, with mismatch_dir_hash_record structure, it provides the base for
later original mode repair.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 18:42:13 +01:00
Qu Wenruo 38aff3d1ed btrfs-progs: check: lowmem: Add ability to repair dir item with mismatched hash
For DIR_ITEM with mismatch hash, we could just remove the offending dir
item from the tree.

Lowmem mode will handle the rest, either re-create the correct dir_item
or move the orphan inode to lost+found.

This is especially important for old filesystems, since later kernel
introduces stricter tree-checker, which could detect such hash mismatch
and refuse to read the corrupted leaf.

With this repair ability, user could repair with 'btrfs check
--mode=lowmem --repair'.

Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1111991
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 18:42:13 +01:00
Qu Wenruo 39ea41f00b btrfs-progs: check: orig: Add dev_item check for used bytes and total bytes
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 18:42:13 +01:00
Qu Wenruo 607a10d71c btrfs-progs: check: lowmem: Add dev_item check for used bytes and total bytes
Obviously, used bytes can't be larger than total bytes.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 18:42:13 +01:00
Qu Wenruo 6b4ee5612e btrfs-progs: check: orig: Add ability to detect bad dev extents
Unlike lowmem mode check, we don't have good place for original mode to
check overlapping device extents.

So this patch introduces a new function, btrfs_check_dev_extents(), to
handle such extents.

Reported-by: Hans van Kranenburg <hans.van.kranenburg@mendix.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 18:42:13 +01:00
Qu Wenruo 920d475a98 btrfs-progs: check: lowmem: Add check for overlapping dev extents
Add such check to check_dev_item(), since at that time we're also
iterating dev extents for dev item accounting.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 18:42:13 +01:00
Nikolay Borisov f7717d8cdb btrfs-progs: Remove fsid/metdata_uuid fields from fs_info
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 16:20:09 +01:00
Nikolay Borisov c4aadd9af2 btrfs-progs: Add support for metadata_uuid field
Add support for a new metadata_uuid field. This is just a preparatory
commit which switches all users of the fsid field for metdata comparison
purposes to utilize the new field. This more or less mirrors the
kernel patch, additionally:

 * Update 'btrfs inspect-internal dump-super' to account for the new
 field. This involes introducing the 'metadata_uuid' line to the
 output and updating the logic for comparing the fs uuid to the
 dev_item uuid.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-12-06 12:51:36 +01:00
Su Yue dc347d0ec6 btrfs-progs: trivial fix of line break in repair_inode_nbytes_lowmem()
Move "\n" at end of the sentence to print.

Fixes: 281eec7a9d ("btrfs-progs: check: repair inode nbytes in lowmem mode")
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-12-05 15:47:06 +01:00
Josh Soref 33d375bc20 btrfs-progs: check: fix typo in device_extent_record::chunk_objectid
This header is exported to /usr/include/btrfs but there are no known
users, so the change should be safe.

Generated by https://github.com/jsoref/spelling

Issue: #154
Author: Josh Soref <jsoref@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-11-26 18:24:49 +01:00
Josh Soref 2cd4a76ea9 btrfs-progs: fix typos in user-visible strings
* error messages
* help strings

Generated by https://github.com/jsoref/spelling

Issue: #154
Author: Josh Soref <jsoref@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-11-26 18:24:49 +01:00
Josh Soref b1d39a42a4 btrfs-progs: fix typos in comments
Generated by https://github.com/jsoref/spelling

Issue: #154
Author: Josh Soref <jsoref@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-11-26 18:24:48 +01:00
Rosen Penev 01e35d9f53 btrfs-progs: treewide: Fix missing declarations
Found using -Wmissing-prototypes in GCC.  This should improve LTO
behavior.

Note that set_free_space_tree_thresholds is an unused function. Adding
inline seems to remove the unused function warning.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-11-13 13:32:41 +01:00
David Sterba e578b59bf6 btrfs-progs: convert strerror to implicit %m
Similar to the changes where strerror(errno) was converted, continue
with the remaining cases where the argument was stored in another
variable.

The savings in object size are about 4500 bytes:

 $ size btrfs.old btrfs.new
   text    data     bss     dec     hex filename
 805055   24248   19748  849051   cf49b btrfs.old
 804527   24248   19748  848523   cf28b btrfs.new

Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-31 18:24:14 +01:00
Nikolay Borisov bf60039cbe btrfs-progs: check: lowmem: Refactor extent type checks in check_file_extent
Make the checks in check_file_extent a bit more explicit. First we check
for unknown type and fail accordingly. Then we check for inline extent
and handle it in the newly introduced check_file_extent_inline. Finally
if none of the above checks triggered then we must have a regular or
prealloc extents.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-31 18:24:13 +01:00
Nikolay Borisov a9cddf632f btrfs-progs: check: lowmem: Refactor extent len test in check_file_extent_inline
Instead of having another top-level if which checks for
'extent_num_bytes != item_inline_len' only if we are !compressed, just
move the 'if' inside the 'else' branch of the first top-level if, since
it has already checked for !compressed or not. No functional changes.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-31 18:24:13 +01:00
Nikolay Borisov 2daecf07f3 btrfs-progs: check: lowmem: Factor out inline extent checking code in its own function
Since the inline extent code can be largely self-sufficient, factor
it out from check_file_extent. No functional changes.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-31 18:24:13 +01:00
Nikolay Borisov be32eb0557 btrfs-progs: check: Add support for freespace tree fixing
Now that all the prerequisite code for proper support of free space
tree repair is in, it's time to wire it in. This is achieved by first
hooking the freespace tree to the __free_extent/alloc_reserved_tree_block
functions. And then introducing a wrapper function to contains the
existing check_space_cache and the newly introduced repair code.
Finally, it's important to note that FST repair code first clears the
existing FST in case of any problem found and rebuilds it from scratch.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-25 16:11:40 +02:00
Nikolay Borisov 909357e867 btrfs-progs: Wire up delayed refs
This commit enables the delayed refs infrastructures. This entails doing
the following:

1. Replacing existing calls of btrfs_extent_post_op (which is the
   equivalent of delayed refs) with the proper btrfs_run_delayed_refs.
   As well as eliminating open-coded calls to finish_current_insert and
   del_pending_extents which execute the delayed ops.

2. Wiring up the addition of delayed refs when freeing extents
   (btrfs_free_extent) and when adding new extents (alloc_tree_block).

3. Adding calls to btrfs_run_delayed refs in the transaction commit
   path alongside comments why every call is needed, since it's not
   always obvious (those call sites were derived empirically by running
   and debugging existing tests)

4. Correctly flagging the transaction in which we are reinitialising
   the extent tree.

5. Moving btrfs_write_dirty_block_groups to
   btrfs_write_dirty_block_groups since blockgroups should be written to
   disk after the last delayed refs have been run.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:41 +02:00
Qu Wenruo b02b426a78 btrfs-progs: fix infinite loop when bad key order repair fails
An infinite loop can be triggered during fuzz/003:

  ====== RUN MAYFAIL btrfs check --repair tests/fuzz-tests/images/bko-199833-reloc-recovery-crash.raw.restored
  [1/7] checking root items
  Fixed 0 roots.
  [2/7] checking extents
  ctree.c:1650: leaf_space_used: Warning: assertion `data_len < 0` failed, value 1
  bad key ordering 18 19
  ctree.c:1650: leaf_space_used: Warning: assertion `data_len < 0` failed, value 1
  bad key ordering 18 19
  ctree.c:1650: leaf_space_used: Warning: assertion `data_len < 0` failed, value 1
  bad key ordering 18 19

[CAUSE]
In try_to_fix_bad_block() it's possible that btrfs_find_all_roots()
finds no root referring to that tree block, thus we can't do any repair.

However in that case, we still return 0 since the last caller assigning
@ret is btrfs_find_all_roots(), and the ulist while loop doesn't get run
at all.

And since try_to_fix_bad_block() returns 0, check_block() in
check/main.c will return -EAGAIN to re-check the tree block.

This leads to the infinite loop.

[FIX]
Change the default return value from 0 to -EIO in
try_to_fix_bad_block(), so if there is no tree referring to the bad tree
block, it won't cause infinite loop anymore.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:40 +02:00
Qu Wenruo 0adf626374 btrfs-progs: exit gracefully when root dir item repair fails
Another BUG_ON() during fuzz/003:

  ====== RUN MAYFAIL btrfs check --init-csum-tree tests/fuzz-tests/images/bko-161821.raw.restored
  [1/7] checking root items
  Fixed 0 roots.
  [2/7] checking extents
  parent transid verify failed on 4198400 wanted 14 found 1114126
  parent transid verify failed on 4198400 wanted 14 found 1114126
  Ignoring transid failure
  owner ref check failed [4198400 4096]
  repair deleting extent record: key [4198400,169,0]
  adding new tree backref on start 4198400 len 4096 parent 0 root 5
  Repaired extent references for 4198400
  ref mismatch on [4222976 4096] extent item 1, found 0
  backref 4222976 root 7 not referenced back 0x55e9cc694780
  incorrect global backref count on 4222976 found 1 wanted 0
  backpointer mismatch on [4222976 4096]
  owner ref check failed [4222976 4096]
  repair deleting extent record: key [4222976,169,0]
  Repaired extent references for 4222976
  [3/7] checking free space cache
  [4/7] checking fs roots
  parent transid verify failed on 4198400 wanted 14 found 1114126
  Ignoring transid failure
  Wrong generation of child node/leaf, wanted: 1114126, have: 14
  root 5 missing its root dir, recreating
  parent transid verify failed on 4198400 wanted 14 found 1114126
  Ignoring transid failure
  ERROR: child eb corrupted: parent bytenr=4222976 item=0 parent level=1 child level=2
  check/main.c:2738: check_inode_recs: BUG_ON `ret` triggered, value -5
  failed (ignored, ret=134): btrfs check --init-csum-tree tests/fuzz-tests/images/bko-161821.raw.restored
  mayfail: returned code 134 (SIGABRT), not ignored
  test failed for case 003-multi-check-unmounted

Just abort current transaction and exit gracefully in this case, the
caller handles errors.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:40 +02:00
David Sterba 4dca0ccf3d btrfs-progs: check: reorder options to sections
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:06:24 +02:00
Qu Wenruo 21d1109168 btrfs-progs: check/original: Don't overwrite return value when we failed to repair
In check_inode_recs(), for repair mode we always reset @ret to 0.  It
makes no sense and later we check @ret to determine if the repair is
successful.

Fix it by removing the offending overwrite.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:06:22 +02:00
Qu Wenruo 159f177cf9 btrfs-progs: check/original: Avoid infinite loop when failed to repair inode
Exposed by fuzz-tests/003-multi-check-unmounted/ on fuzzed image
bko-161811.raw.xz.

It's caused by the fact when check_fs_roots() finds tree root is
modified, it re-search tree root by goto again: label.
However again: label. will also reset root objectid to 0.
If we failed to repair one fs root but still modified tree root, we will
go into such infinite loop.

Fix it by recording which root we should skip for repair mode.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:06:20 +02:00
Qu Wenruo 5f594c3ce3 btrfs-progs: check: Remove the ability to rebuild root overwriting existing tree blocks
We have function btrfs_fsck_reinit_root() to reinit csum or extent tree.
However this function allows us to let it overwrite existing tree blocks
using @overwrite parameter.

Such behavior is pretty dangerous while no caller is using this feature
explicitly.

So just remove @overwrite parameter and allow btrfs_fsck_reinit_root()
to error out when it fails to allocate tree block.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:04:58 +02:00
Stéphane Lesimple 078e9a1cc9 btrfs-progs: check: enhanced progress indicator
We reuse the task_position enum and task_ctx struct of the original progress
indicator, adding more values and fields for our needs.

Then add hooks in all steps of the check to properly record progress.

Here's how the output looks like on a 22 Tb 5-disk RAID1 FS:

Opening filesystem to check...
Checking filesystem on /dev/mapper/luks-ST10000VN0004-XXXXXXXX
UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[1/7] checking extents           (0:20:21 elapsed, 950958 items checked)
[2/7] checking root items        (0:01:29 elapsed, 15121 items checked)
[3/7] checking free space cache  (0:00:11 elapsed, 4928 items checked)
[4/7] checking fs roots          (0:51:31 elapsed, 600892 items checked)
[5/7] checking csums             (0:14:35 elapsed, 754522 items checked)
[6/7] checking root refs         (0:00:00 elapsed, 232 items checked)
[7/7] checking quota groups skipped (not enabled on this FS)
found 5286458060800 bytes used, no error found

Signed-off-by: Stéphane Lesimple <stephane_btrfs@lesimple.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:23 +02:00
Su Yue d53d42fa21 btrfs-progs: lowmem: fix false alerts of referencer count mismatch for blocks relocated
Btrfs lowmem check reports such false alerts:
=========
ERROR: extent[1419709677568, 1703936] referencer count mismatch (root: 2192, owner: 327635, offset: 0) wanted: 9, have: 13
=========

While in extent tree, the extent has:
=========
        item 98 key (1419709677568 EXTENT_ITEM 1703936) itemoff 10745 itemsize 92
                refs 35 gen 16921 flags DATA
                extent data backref root 2192 objectid 327635 offset 0 count 9
                shared data backref parent 1232784752640 count 4
                shared data backref parent 1232784736256 count 13
                shared data backref parent 1232784719872 count 9
=========

The extent data 327635 starts in this leaf without BTRFS_HEADER_FLAG_RELOC:
=========
leaf 1471521390592 items 146 free space 3671 generation 18581 owner 2192
leaf 1471521390592 flags 0x1(WRITTEN) backref revision 1
...
 item 137 key (327635 EXTENT_DATA 0) itemoff 7745 itemsize 53
                generation 15770 type 1 (regular)
                extent data disk byte 1419709677568 nr 1703936
                extent data offset 0 nr 131072 ram 1703936
                extent compression 0 (none)
        item 138 key (327635 EXTENT_DATA 131072) itemoff 7692 itemsize 53
                generation 15770 type 1 (regular)
                extent data disk byte 1419709677568 nr 1703936
                extent data offset 131072 nr 131072 ram 1703936
                extent compression 0 (none)
...(Exactly 9 items here)
=========

The next leaf is flaged as relocated, so extent data backrefs are
shared not keyed:
=========
leaf 1232784752640 items 159 free space 2503 generation 16924 owner 2192
leaf 1232784752640 flags 0x3(WRITTEN|RELOC) backref revision 1
fs uuid 0f43f49d-6e63-4b1b-bc8c-c54da409872d
chunk uuid e558e6f7-4f08-4292-91fb-9a775fdd530b
        item 0 key (327635 EXTENT_DATA 1179648) itemoff 16230 itemsize 53
                generation 15770 type 1 (regular)
                extent data disk byte 1419709677568 nr 1703936
                extent data offset 1179648 nr 131072 ram 1703936
                extent compression 0 (none)
        item 1 key (327635 EXTENT_DATA 1310720) itemoff 16177 itemsize 53
                generation 15770 type 1 (regular)
                extent data disk byte 1419709677568 nr 1703936
                extent data offset 1310720 nr 131072 ram 1703936
                extent compression 0 (none)
        item 2 key (327635 EXTENT_DATA 1441792) itemoff 16124 itemsize 53
                generation 15770 type 1 (regular)
                extent data disk byte 1419709677568 nr 1703936
                extent data offset 1441792 nr 131072 ram 1703936
                extent compression 0 (none)
        item 3 key (327635 EXTENT_DATA 1572864) itemoff 16071 itemsize 53
=======

Lowmem mode starts to count extent data backrefs in first leaf, those
backrefs are keyed as wanted.
It calls btrfs_next_item to skip to next leaf.
However, the next leaf was relocated, extent data items in this leaf
should have shared backrefs shouldn't be counted.

So let lowmem mode do not count data backrefs if leaf is flaged with
BTRFS_HEADER_FLAG_RELOC.

Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:14 +02:00
Qu Wenruo c010437455 btrfs-progs: check: orig: Don't panic when unexpected root item is referring to one extent
With crafted image, expected root item can refer to certain extent, and
original mode uses BUG_ON() to handle such case.

Fix it by gracefully return error.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=200403
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:07 +02:00
Nikolay Borisov f3c7101116 btrfs-progs: check: Drop trans/root arguments from free_extent_hook
They are not really needed, what free_extent_hook wants is really a
pointer to fs_info so give it to it directly. This is in preparation
of delayed refs code.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:01:57 +02:00
David Disseldorp 7490f94dd7 btrfs-progs: check: add experimental flag for lowmem mode
The experimental flag is already carried in the manpage, but was removed
from the btrfs check usage message as part of refactoring via
87c1bd13c1. Add it back.

Signed-off-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:01:19 +02:00
Qu Wenruo eca8dc0757 btrfs-progs: check/lowmem: Repair wrong inline ram_bytes for uncompressed extent
Similar to the original mode repair.

Reported-by: Steve Leung <sjleung@shaw.ca>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:01:03 +02:00
Qu Wenruo cd28b15496 btrfs-progs: check/lowmem: Prepare check_file_extent() to handle repair
Current check_file_extent() doesn't support later repair work, since it
doesn't accept btrfs_path structure as parameter, thus it can't modify
btrfs trees, or later check will still use the old and wrong path.

Use btrfs_path to replace btrfs_key, extent_buffer and slot parameters,
so we can modify @path directly for repair, and reduce the number of
parameters for check_file_extent().

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:00:58 +02:00
Qu Wenruo e5f50be06f btrfs-progs: check/original: Detect and repair wrong inline ram_bytes
It looks like that around 2014, btrfs kernel has a regression that would
cause offset-by-one ram_bytes for inline extent.

Add the ability to repair it in original mode.

Reported-by: Steve Leung <sjleung@shaw.ca>
Tested-by: Steve Leung <sjleung@shaw.ca>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:00:44 +02:00
Qu Wenruo 3c042605db btrfs-progs: Get rid of the confusing btrfs_file_extent_inline_len()
[BUG]
If one uncompressed inline extent has incorrect ram_bytes, neither btrfs
check nor dump-tree could detect such corruption.

[CAUSE]
Every caller tries to read inline extent ram_bytes is using
btrfs_file_extent_inline_len(), other than directly calling
btrfs_file_extent_ram_bytes().

For compressed extent, it's just calling btrfs_file_extent_ram_bytes().
However for uncompressed extent, it falls back to
btrfs_file_extent_inline_item_len(), makes us unable to detect anything
wrong in ram_bytes.

[FIX]
Just get rid of such confusing btrfs_file_extent_inline_len() function.

Reported-by: Steve Leung <sjleung@shaw.ca>
Tested-by: Steve Leung <sjleung@shaw.ca>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:00:34 +02:00