Commit Graph

177 Commits (master)

Author SHA1 Message Date
Qu Wenruo b3621da146 btrfs-progs: check/original: Detect invalid extent generation
Much like what we have done in lowmem mode, also detect and report
invalid extent generation in original mode.

Unlike lowmem mode, we have extent_record::generation, which is the
higher number of generations in EXTENT_ITEM, EXTENT_DATA or tree block
header, so there is no need to check generations in different locations.

For repair, we still need to depend on --init-extent-tree, as directly
modifying extent items can easily cause conflicts with delayed refs,
thus it should be avoided.

Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:10 +01:00
Qu Wenruo 918ba847bc btrfs-progs: check/lowmem: Detect invalid EXTENT_ITEM and EXTENT_DATA generation
Since older `btrfs check --init-extent-tree` could cause invalid
EXTENT_ITEM generation for data extents, add such check to lowmem mode
check.

Also add such generation check to file extents too.

For the repair part, I don't have any good idea yet. So affected user
may depend on --init-extent-tree again.

Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:10 +01:00
Qu Wenruo 8d45dc270a btrfs-progs: check: Populate extent generation correctly for data extents
[BUG]
When using `btrfs check --init-extent-tree`, we will create incorrect
generation number for data extents in extent tree:

        item 10 key (13631488 EXTENT_ITEM 1048576) itemoff 15828 itemsize 53
                refs 1 gen 0 flags DATA
                extent data backref root FS_TREE objectid 257 offset 0 count 1

[CAUSE]
Since data extent generation is not as obvious as tree blocks, which has
header containing its generations, so for data extents, its
extent_record::generation is not really updated, resulting such 0
generation.

[FIX]
To get generation of a data extent, there are two sources we can rely:
- EXTENT_ITEM
  There is always a btrfs_extent_item::generation can be utilized.
  Although this is not possible for --init-extent-tree use case.

- EXTENT_DATA
  We have btrfs_file_extent_item::generation for regular and
  preallocated data extents.
  Since --init-extent-tree will go through subvolume trees, this would
  be the main source for extent data generation.

Then we only need to make add_data_backref() to accept @gen parameter,
and pass it down to extent_record structure.

And for the final extent item generation update, here we add extra
fallback values, if we can't find FILE_EXTENT items.
In that case, we just fall back to current transid.

With this modification, recreated data EXTENT_ITEM now has correct
generation number:

        item 10 key (13631488 EXTENT_ITEM 1048576) itemoff 15828 itemsize 53
                refs 1 gen 6 flags DATA
                extent data backref root FS_TREE objectid 257 offset 0 count 1

Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:10 +01:00
Qu Wenruo 9a85b7db02 btrfs-progs: check: Initialize extent_record::generation member
[BUG]
When using `btrfs check --init-extent-tree`, there is a pretty high
chance that the result fs can't pass tree-checker:

  BTRFS critical (device dm-3): corrupt leaf: block=5390336 slot=149 extent bytenr=20115456 len=4096 invalid generation, have 16384 expect (0, 360]
  BTRFS error (device dm-3): block=5390336 read time tree block corruption detected
  BTRFS error (device dm-3): failed to read block groups: -5
  BTRFS error (device dm-3): open_ctree failed

[CAUSE]
The result fs has a pretty screwed up EXTENT_ITEMs for data extents:

        item 148 key (20111360 EXTENT_ITEM 4096) itemoff 8777 itemsize 53
                refs 1 gen 0 flags DATA
                extent data backref root FS_TREE objectid 841 offset 0 count 1
        item 149 key (20115456 EXTENT_ITEM 4096) itemoff 8724 itemsize 53
                refs 1 gen 16384 flags DATA
                extent data backref root FS_TREE objectid 906 offset 0 count 1

Kernel tree-checker will accept 0 generation, but that 16384 generation
is definitely going to trigger the alarm.

Looking into the code, it's add_extent_rec_nolookup() allocating a new
extent_rec, but not copying all members from parameter @tmpl, resulting
generation not properly initialized.

[FIX]
Just copy tmpl->generation in add_extent_rec_nolookup(). And since all
call sites have set all members of @tmpl to 0 before
add_extent_rec_nolookup(), we shouldn't get garbage values.

For the 0 generation problem, it will be solved in another patch.

Issue: #225 (Not the initial report, but extent tree rebuild result)
Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:10 +01:00
Qu Wenruo 3dcce48fd7 btrfs-progs: check/original: Do extra verification on file extent item
[BUG]
For certain fuzzed image, `btrfs check` will fail with the following
call trace:

  Checking filesystem on issue_213.raw
  UUID: 99e50868-0bda-4d89-b0e4-7e8560312ef9
  [1/7] checking root items
  [2/7] checking extents
  Program received signal SIGABRT, Aborted.
  0x00007ffff7c88f25 in raise () from /usr/lib/libc.so.6
  (gdb) bt
  #0  0x00007ffff7c88f25 in raise () from /usr/lib/libc.so.6
  #1  0x00007ffff7c72897 in abort () from /usr/lib/libc.so.6
  #2  0x00005555555abc3e in run_next_block (...) at check/main.c:6398
  #3  0x00005555555b0f36 in deal_root_from_list (...) at check/main.c:8408
  #4  0x00005555555b1a3d in check_chunks_and_extents (fs_info=0x5555556a1e30) at check/main.c:8690
  #5  0x00005555555b1e3e in do_check_chunks_and_extents (fs_info=0x5555556a1e30) a
  #6  0x00005555555b5710 in cmd_check (cmd=0x555555696920 <cmd_struct_check>, argc
  #7  0x0000555555568dc7 in cmd_execute (cmd=0x555555696920 <cmd_struct_check>, ar
  #8  0x0000555555569713 in main (argc=2, argv=0x7fffffffde70) at btrfs.c:386

[CAUSE]
This fuzzed images has a corrupted EXTENT_DATA item in data reloc tree:
        item 1 key (256 EXTENT_DATA 256) itemoff 16111 itemsize 12
                generation 0 type 2 (prealloc)
                prealloc data disk byte 16777216 nr 0
                prealloc data offset 0 nr 0

There are several problems with the item:
- Bad item size
  12 is too small.
- Bad key offset
  offset of EXTENT_DATA type key represents file offset, which should
  always be aligned to sector size (4K in this particular case).

[FIX]
Do extra item size and key offset check for original mode, and remove
the abort() call in run_next_block().

And to show off how robust lowmem mode is, lowmem can handle it without
any hiccup.

With this fix, original mode can detect the problem properly:
  Checking filesystem on issue_213.raw
  UUID: 99e50868-0bda-4d89-b0e4-7e8560312ef9
  [1/7] checking root items
  [2/7] checking extents
  ERROR: invalid file extent item size, have 12 expect (21, 16283]
  ERROR: errors found in extent allocation tree or chunk allocation
  [3/7] checking free space cache
  [4/7] checking fs roots
  root 18446744073709551607 root dir 256 error
  root 18446744073709551607 inode 256 errors 62, no orphan item, odd file extent, bad file extent
  ERROR: errors found in fs roots
  found 131072 bytes used, error(s) found
  total csum bytes: 0
  total tree bytes: 131072
  total fs tree bytes: 32768
  total extent tree bytes: 16384
  btree space waste bytes: 124774
  file data blocks allocated: 0
   referenced 0

Issue: #213
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:09 +01:00
Qu Wenruo 36128dff44 btrfs-progs: check/lowmem: Fix a false alert on uninitialized value
[BUG]
When compiling the devel branch with commit fb8f05e40b458
("btrfs-progs: check: Make repair_imode_common() handle inodes in
subvolume trees"), the following warning will be reported:

  check/mode-common.c: In function ‘detect_imode’:
  check/mode-common.c|1071 col 23| warning: ‘imode’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  1071 |   *imode_ret = (imode | 0700);
       |                ~~~~~~~^~~~~~~

This only occurs for regular build. If compiled with D=1, the warning
just disappears.

[CAUSE]
Looks like a bug in gcc optimization.
The code will only set @imode_ret when @found is true.
And for every "found = true" assignment we have assigned @imode.
So this is just a false alert.

[FIX]
I hope I can fix the problem of GCC, but obviously I can't (at least for
now).

So let's assign an initial value 0 to @imode to suppress the false
alert.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-22 19:09:51 +01:00
Johannes Thumshirn e388bf386b btrfs-progs: check: warn users about the possible dangers of --repair
The manual page of btrfsck clearly states 'btrfs check --repair' is a
dangerous operation.

Although this warning is in place users do not read the manual page
and/or are used to the behaviour of fsck utilities which repair the
filesystem, and thus potentially cause harm.

Similar to 'btrfs balance' without any filters, add a warning and a
countdown, so users can bail out before eventual corrupting the
filesystem more than it already is.

To override the timeout, let --force skip it and continue.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-22 19:07:20 +01:00
Qu Wenruo 989a99b5f8 btrfs-progs: Replace btrfs_block_group_cache::item with dedicated members
We access btrfs_block_group_cache::item mostly for @used and @flags.

@flags is already a dedicated member in btrfs_block_group_cache, only
@used doesn't have a dedicated member.

This patch will remove btrfs_block_group_cache::item and add
btrfs_block_group_cache::used.

It's the btrfs-progs equivalent of the following kernel patches:
btrfs: move block_group_item::used to block group
btrfs: move block_group_item::flags to block group
btrfs: remove embedded block_group_cache::item

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:09 +01:00
Qu Wenruo e46281d6fb btrfs-progs: Refactor excluded extent functions to use fs_info
The following functions are just using @root to reach fs_info:
- exclude_super_stripes
- free_excluded_extents
- add_excluded_extent

Refactor them to use fs_info directly.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:08 +01:00
Qu Wenruo b9ea7c1b23 btrfs-progs: check/original: Add check and repair for invalid inode generation
There are at least two bug reports of kernel tree-checker complaining
about invalid inode generation.

All offending inodes seem to be caused by old kernel around 2014, with
inode generation overflow.

So add such check and repair ability to lowmem mode check first.

This involves:

- Calculate the inode generation upper limit
  Unlike the lowmem mode context, we don't have anyway to determine if
  this inode belongs to log tree.
  So we use super_generation + 1 as upper limit, just like what we did
  in kernel tree checker.

- Check if the inode generation is larger than the upper limit

- Repair by resetting inode generation to current transaction
  generation
  The difference is, in original mode, we have a common trans handle for
  all repair and reset path for each repair.

Reported-by: Charles Wright <charles.v.wright@gmail.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo 0a6e40bdc1 btrfs-progs: check/lowmem: Add check and repair for invalid inode generation
There are at least two bug reports of kernel tree-checker complaining
about invalid inode generation.

All offending inodes seem to be caused by old kernel around 2014, with
inode generation overflow.

So add such check and repair ability to lowmem mode check first.

This involves:

- Calculate the inode generation upper limit
  If it's an inode from log tree, then the upper limit is
  super_generation + 1, otherwise it's super_generation.

- Check if the inode generation is larger than the upper limit

- Repair by resetting inode generation to current transaction
  generation

Reported-by: Charles Wright <charles.v.wright@gmail.com>
Tested-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo eae0c8e32f btrfs-progs: check/original: Fix inode mode in subvolume trees
To make original mode to repair imode error in subvolume trees, this
patch will do:

- Remove the show-stopper checks for root->objectid.
  Now repair_imode_original() will accept inodes in subvolume trees.

- Export detect_imode() for original mode
  Due to the call requirement, original mode must use an existing trans
  handler to do the repair, thus we need to re-implement most of the
  work done in repair_imode_common().

- Make repair_imode_original() to use detect_imode().

- Free the path after reset_imode()
  reset_imode() keeps the path, as lowmem mode uses path to locate its
  current check position.
  But for original mode, the unreleased path can cause later repair to
  report warning, so we need to manually release the path.

- Update rec->imode after imode reset
  So later repair depending on rec->imode can get correct value.

- Move the repair before repair_inode_nlinks()
  repair_inode_nlinks() needs correct imode to add DIR_INDEX/DIR_ITEM.
  So moving the repair before repair_inode_nlinks() makes the latter
  repair happier.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo 69eac9de0c btrfs-progs: check/lowmem: Repair bad imode early
For lowmem mode, if we hit a bad inode mode, normally it is reported
when we checking the DIR_INDEX/DIR_ITEM of the parent inode.

If we didn't repair at that time, the error will be recorded even if we
fixed it later.

So this patch will check for INODE_ITEM_MISMATCH error type, and if it's
really caused by invalid imode, repair it and clear the error.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo 7f8383b7a6 btrfs-progs: check: Make repair_imode_common() handle inodes in subvolume trees
[[PROBLEM]]
Before this patch, repair_imode_common() can only handle two types of
inodes:

- Free space cache inodes
- ROOT DIR inodes

For inodes in subvolume trees, the core complexity is how to determine
the correct imode, thus it was not implemented.

However there are more reports of incorrect imode in subvolume trees, we
need to support such fix.

[[ENHANCEMENT]]
So this patch adds a new function, detect_imode(), to detect imode for
inodes in subvolume trees.  The policy here is, try our best to find a
valid imode to recovery.  If no convicing info can be found, fail out.

That function will determine imode by:

1) Search for INODE_REF of the inode
   If we have INODE_REF, we will then try to find DIR_ITEM/DIR_INDEX.
   As long as one valid DIR_ITEM or DIR_INDEX can be found, we convert
   the BTRFS_FT_* to imode, then call it a day.
   This should be the most accurate way.

2) Search for DIR_INDEX/DIR_ITEM belongs to this inode
   If above search fails, we falls back to locate the DIR_INDEX/DIR_ITEM
   just after the INODE_ITEM.
   Thus this only works for non-empty directory.
   If any can be found, it's definitely a directory.

3) Search for EXTENT_DATA belongs to this inode
   If EXTENT_DATA can be found, it's either REG or LNK.
   Thus this only works for non-empty file or soft link.
   For this case, we default to REG, as user can inspect the file to
   determine if it's a file or just a path.

4) Use rdev to detect BLK/CHR
   If all above fails, but INODE_ITEM has non-zero rdev, then it's either
   a BLK or CHR file. Then we default to BLK.

5) Fail out if none of above methods succeeded
   No educated guess to make things worse.

[[SHORTCOMING]]
The above search is not perfect, there are cases where we can't repair:
E.g. orphan empty regular inode.  Since it's already orphan, it has no
INODE_REF. And it's regular empty file, it has no DIR_INDEX nor
EXTENT_DATA nor rdev. Thus we can't recover.  Although for this case, it
really doesn't matter as it's already orphan and will be deleted anyway.

Furthermore, due to the DIR_ITEM/DIR_INDEX/INODE_REF repair code which
can happen before imode repair, it's possible that DIR_ITEM search code
may not be executed.  If there is only DIR_ITEM remaining, repair code
will remove the DIR_ITEM completely and move the inode to lost+found,
leaving us no info to rebuild imode.  If there is DIR_INDEX missing,
repair code will re-insert the DIR_INDEX, then imode repair code will go
DIR_INDEX directly.

But overall, the repair code should handle the invalid imode caused by
older kernels without problem.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo ac9e07a780 btrfs-progs: check: find imode using info from INODE_REF item
Introduce a function, find_file_type(), to find filetype using info from
INODE_REF, including dir_id from key index/name from inode_ref_item.

This function will:

- Search DIR_INDEX first
  DIR_INDEX is easier since there is only one item in it.

- Validate the DIR_INDEX item
  If the DIR_INDEX is valid, use the filetype and call it a day.

- Search DIR_ITEM then
  It needs extra iteration since it's possible to have hash collision.

- Validate the DIR_ITEM
  If valid, call it a day. Or return -ENOENT;

This would be used as the primary method to determine the imode in later
imode repair code.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Qu Wenruo 87207654f1 btrfs-progs: check: Export btrfs_type_to_imode
This function will be later used by common mode code, so export it.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-18 19:21:07 +01:00
Rosen Penev 5d72055066 btrfs-progs: Fix printf formats
Discovered with cppcheck. Fix signed/unsigned int mismatches, sizeof and
long formats.

Pull-request: #197
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:31:05 +02:00
Johannes Thumshirn ed33908b44 btrfs-progs: update checksumming api
Update the checksumming API to be able to cope with more checksum types
than just CRC32C. The finalization call is merged into btrfs_csum_data.

There are some fixme's and asserts added that need to be resolved.

Co-developed-by: David Sterba <dsterba@suse.com>
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:28:34 +02:00
Johannes Thumshirn 7b4f1035a6 btrfs-progs: pass checksum type to btrfs_csum_data()/btrfs_csum_final()
In preparation to supporting new checksum algorithm pass the checksum type
to btrfs_csum_data/btrfs_csum_final, this allows us to encapsulate any
differences in processing into the respective functions

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:28:28 +02:00
Johannes Thumshirn de68086e35 btrfs-progs: don't assume checksums are always 4 bytes
Pass pointer to a generic buffer instead of fixed size that crc32c
currently uses.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:28:25 +02:00
David Sterba 0889161127 btrfs-progs: check: move device_record to main.c
The structure is used only inside main.c, no need to export it.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:27:41 +02:00
David Sterba 705f4ecf02 btrfs-progs: check: remove flat include switch from common.h
The header will be used only internally, so the FLAT_INCLUDES is not
needed.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:27:39 +02:00
David Sterba 1b42e71053 btrfs-progs: copy btrfsck.h to check/common.h
We want just one header for the check API (similar to what mkfs does)
but as btrfsck.h is exported header (libbtrfs), it needs some
deprecation beriod before it's moved through there are probably no users
of that header file in particular.

Copy the header to check, all modifications and cleanups won't affect
the public header.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:27:37 +02:00
David Sterba 8e68287d88 btrfs-progs: move qgroup-verify.[ch] to check/
The only user of the functionality is in check.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:27:35 +02:00
Qu Wenruo c31bd8db54 btrfs-progs: check/lowmem: Skip nbytes check for orphan inodes
For orphan inodes, kernel won't update its nbytes and size since it's a
waste of time.

So lowmem check can report false alert on some orphan inodes.

Fix it by checking if the inode is an orphan before
complaining/repairing its nbytes.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-09-04 16:08:56 +02:00
Josef Bacik 73ff25c65d btrfs-progs: check: don't check nbytes on unlinked files
We don't update the inode when evicting it, so the nbytes will be wrong
in between transaction commits.  This isn't a problem, stop complaining
about it to make generic/269 stop randomly failing. The orphan outdated
inodes can be still present but check will not skip them.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-09-04 16:08:11 +02:00
Qu Wenruo 1baf729743 btrfs-progs: check/original: Check and repair root item geneartion
Add such ability to original mode to fix root generation mismatch, which
can be rejected by kernel.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-09-04 16:06:52 +02:00
Qu Wenruo ebf29e39c3 btrfs-progs: check/lowmem: Check and repair root generation
Since kernel is going to reject any root item which is newer than super
block generation, we need to provide a way to fix such problem in
btrfs-check.

This patch addes the ability to report and repair root generation in
lowmem mode.

This is done by cowing the root node, so we will update the root
generation along with the root node generation at commit transaction
time.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-09-04 16:06:50 +02:00
Adam Borowski 90a87e817d btrfs-progs: check: fix option parsing for -E
The option -E has a mandatory argument that was missing from the short
option spec, thus it always crashed.

Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-09-04 16:06:24 +02:00
Naohiro Aota 8ca6c0c3c7 btrfs-progs: check: initialize qgroup_item_count in earlier stage
"btrfsck -Q" segfaults because it does not call qgroup_set_item_count_ptr()
properly:

  # btrfsck -Q /dev/sdk
  Opening filesystem to check...
  Checking filesystem on /dev/sdk
  UUID: 34a35bbc-43f8-40f0-8043-65ed33f2e6c3
  Print quota groups for /dev/sdk
  UUID: 34a35bbc-43f8-40f0-8043-65ed33f2e6c3
  Segmentation fault (core dumped)

Since "struct task_ctx ctx" is global, we can just move
qgroup_set_item_count_ptr() much earlier stage in the check process to
avoid to forget initializing it.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-26 17:46:42 +02:00
Josef Bacik 42a1aaeec4 btrfs-progs: deal with drop_progress properly in fsck
While testing snapshot deletion with dm-log-writes I saw that I was
failing the fsck sometimes when the fs was actually in the correct
state.  This is because we only skip blocks on the same level of
root_item->drop_level.  If the drop_level < the root level then we could
very well walk into nodes that we wouldn't actually walk into on fs
mount, because the drop_progress is further ahead in the slot of the
root.  Instead only process the slots of the nodes that are above the
drop_progress key.  With this patch in place we no longer improperly
fail to check fs'es that have a drop_progress set with a drop_level <
root level.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-26 17:46:42 +02:00
David Sterba c8bea2b84b btrfs-progs: move rbtree-utils.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba c07960c8be btrfs-progs: move utils.[ch] to common/
Update include paths and remove some duplicates.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba aac564aca6 btrfs-progs: move commonh to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba f93b471143 btrfs-progs: move help.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba d0970a05cd btrfs-progs: move task-utils.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba d1efe50d0a btrfs-progs: move messages.[ch] to common/
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
David Sterba f63f29e9e9 btrfs-progs: move internal.h to common/
Create directory for all sources that can be used by anything that's not
rellated to a relevant kernel part, all common functions, helpers,
utilities that do not fit any other specific category.

The traditional location would be probably lib/ with all things that are
statically linked to the main binaries, but we have libbtrfs and
libbtrfsutil so this would be confusing.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:03 +02:00
Nikolay Borisov 32f75c87a0 btrfs-progs: check: Remove duplicated and commented functions
Commit 756105181e ("btrfs-progs: check: supplement extent backref
list with rbtree") changed the backref implementation to use rb tree
and also commented the old implementations. It's been almost 2 years
since that change and it's unlikely the old version will ever be used,
so just remove it.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 13:31:16 +02:00
Jeff Mahoney 91a1476d53 btrfs-progs: pass cmd_struct to usage()
Now that every call site has a cmd_struct, we can just pass the cmd_struct
to usage to print the usager information.  This allows us to interpret
the format flags we'll add later in this series to inform the user of
which output formats any given command supports.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 13:31:15 +02:00
Jeff Mahoney b44445131f btrfs-progs: pass cmd_struct to command callback function
This patch passes the cmd_struct to the command callback function.  This
has several purposes: It allows the command callback to identify which
command was used to call it.  It also gives us direct access to the
usage associated with that command.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 13:31:15 +02:00
Jeff Mahoney 82d48463ea btrfs-progs: use cmd_struct as command entry point
Rather than having global command usage and callbacks used to create
cmd_structs in the command array, establish the cmd_struct structures
separately and use those.  The next commit in the series passes the
cmd_struct to the command callbacks such that we can access flags
and determine which of several potential command we were called as.

This establishes several macros to more easily define the commands
within each command's source.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 13:31:15 +02:00
Qu Wenruo 48248693cd btrfs-progs: check/lowmem: Reset path in repair mode to avoid incorrect item from being passed to lowmem check.
In lowmem mode, we check fs roots and free space cache by iterating
each root item and inode item, using btrfs_next_item() and a path
pointing to the root tree.

However in repair mode, check_fs_root() can modify the fs root, thus
CoWs the tree root, and the old path in check_fs

It could lead to strange behavior, e.g. after repairing a fs tree, the
path can point to a fs tree.
Since no ROOT_ITEM exists in fs tree, all remaining trees are skipped in
repair mode.

This bug exists from the early time of lowmem mode repair, and is only
exposed by recent free space inode check code. (Fs tree inodes are
passed to free space inode check, causing false alerts and repair
failure).

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-06-05 18:02:05 +02:00
Qu Wenruo 085445e793 btrfs-progs: Cleanup BTRFS_COMPAT_EXTENT_TREE_V0
BTRFS_COMPAT_EXTENT_TREE_V0 is introduced for a short time in kernel,
and it's over 10 years ago.

Nowadays there should be no user for that feature, and kernel has remove
this support in Jun, 2018. There is no need for btrfs-progs to support
it.

This patch will remove EXTENT_TREE_V0 related code and replace those
BUG_ON() to a more graceful error message.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-06-05 18:00:07 +02:00
Qu Wenruo d73427b76d btrfs-progs: check/original: Add checks for compressed extent without csum
There is one report of compressed extent happens in btrfs, but has no
csum and then leads to possible decompress error screwing up kernel
memory.

Although it's a kernel bug, and won't cause problem until compressed
data get corrupted, let's catch such problem in advance.

This patch will catch any unexpected compressed extent with:

1) 0 or less than expected csum

2) nodatasum flag set in the inode item

This is for original mode.

Reported-by: James Harvey <jamespharvey20@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-05-17 12:32:38 +02:00
Qu Wenruo cc7a35f642 btrfs-progs: check/lowmem: Add checks for compressed extent without csum
There is one report of compressed extent happens in btrfs, but has no
csum and then leads to possible decompress error screwing up kernel
memory.

Although it's a kernel bug, and won't cause problem until compressed
data get corrupted, let's catch such problem in advance.

This patch will catch any unexpected compressed extent with:

1) missing csum

2) nodatasum flag set in the inode item

This is for lowmem mode.

Reported-by: James Harvey <jamespharvey20@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-05-17 12:32:38 +02:00
Jeff Mahoney 50faf73f22 btrfs-progs: check: fixup_extent_flags needs to deal with non-skinny metadata
When repairing a file system created by a very old kernel, I ran into
issues fixing up the extent flags since fixup_extent_flags assumed
that a METADATA_ITEM would be present if the record was for metadata.

Since METADATA_ITEMs don't exist without skinny metadata, we need to
fall back to EXTENT_ITEMs.  This also falls back to EXTENT_ITEMs even
with skinny metadata enabled as other parts of the tools do.

Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-05-17 12:32:38 +02:00
Qu Wenruo 427990ad74 btrfs-progs: check/original: Check and repair free space cache inode item
Just like lowmem mode, also check and repair free space cache inode
item.

And since we don't really have a good timing/function to check free
space chace inodes, we use the same common mode
check_repair_free_space_inode() when iterating root tree.

Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:04:25 +08:00
Qu Wenruo 77fe19ba16 btrfs-progs: check/lowmem: Check and repair free space cache inode mode
Unlike inodes in fs roots, we don't really check the inode items in root
tree, in fact we just skip everything other than ROOT_ITEM and ROOT_REF.

This makes invalid inode items sneak into root tree.
For example:
        item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
                generation 30 transid 30 size 65536 nbytes 1507328
                block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
				   ^ Should be 100600
                sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
                atime 0.0 (1970-01-01 08:00:00)
                ctime 1553491158.189771625 (2019-03-25 13:19:18)
                mtime 0.0 (1970-01-01 08:00:00)
                otime 0.0 (1970-01-01 08:00:00)

There is a report of such problem in the mail list.

This patch will check and repair inode items of free space cache inodes in
lowmem mode.

Since free space cache inodes doesn't have INODE_REF but still has 1
link, we can't use check_inode_item() directly.
Instead we only check the inode mode, as that's the important part.

The check and repair function: check_repair_free_space_inode() is also
exported for original mode.

Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:04:25 +08:00
Qu Wenruo 11fd6cff82 btrfs-progs: check/original: Repair invalid inode mode in root tree
This patch will reuse the mode independent repair_imode() function, to
repair invalid inode mode.

Signed-off-by: Qu Wenruo <wqu@suse.com>
2019-04-16 09:04:25 +08:00