For dump-tree/dump-super the completion uses default filedir -d, which
is far from convenient. Use filedir for
dump-tree/dump-super/inode-resolve just like rootid.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For developers it's pretty common to call "btrfs check" on a raw image
dump other than real block device. It's also possible to end users to
do some tests on loop devices.
So current _btrfs_devs() is really making things worse. Use _filedir()
to replace _btrfs_devs() so it can complete any filenames, no matter if
it's just a file or a real block device.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There are cases that btrfs_commit_transaction() itself can fail, mostly
due to ENOSPC when allocating space.
Don't panic out in this case.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When the 'btrfsune -u' command is interrupted, the final filesystem fsid
is not written to the superblock and it cannot be mounted. Too bad that
'btrfstune' cannot continue to finish the UUID change as it should.
This patch fixes that and passes the relaxed flags for superblock and
only warns when it detects the fsid mismatch. As this is something that
should be noted in case it would be needed for further debugging, it's
not just silent.
Signed-off-by: David Sterba <dsterba@suse.com>
Reset ret value to zero after snprintf(), which returns the number
of written chars. Otherwise non-zero value returns after command
success with -P option. Also set return value from __ino_to_path_fd() to
reflect the final status for default behavior.
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
During fstests/btrfs/166, it's possible to hit a certain case where
qgroup is just enabled but rescan hasn't kicked in.
Since at qgroup enable time, we set the flag INCONSISTENT, and let later
rescan clear that flag. If power loss occurs before the rescan starts,
it's possible we get a qgroup status item with ON|INCONSISTENT but
without the RESCAN flag.
And in that case, it will definitely cause difference in qgroup
accounting as all numbers in the qgroup tree are 0.
Fix this false alert by also checking rescan progress from
btrfs_status_item. And if we find rescan progress is still 0,
INCONSISTENT flag set and no RESCAN flag set, we won't treat it as an
error.
Reported-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In check_inode_recs(), for repair mode we always reset @ret to 0. It
makes no sense and later we check @ret to determine if the repair is
successful.
Fix it by removing the offending overwrite.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Exposed by fuzz-tests/003-multi-check-unmounted/ on fuzzed image
bko-161811.raw.xz.
It's caused by the fact when check_fs_roots() finds tree root is
modified, it re-search tree root by goto again: label.
However again: label. will also reset root objectid to 0.
If we failed to repair one fs root but still modified tree root, we will
go into such infinite loop.
Fix it by recording which root we should skip for repair mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We have function btrfs_fsck_reinit_root() to reinit csum or extent tree.
However this function allows us to let it overwrite existing tree blocks
using @overwrite parameter.
Such behavior is pretty dangerous while no caller is using this feature
explicitly.
So just remove @overwrite parameter and allow btrfs_fsck_reinit_root()
to error out when it fails to allocate tree block.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The build system mentioned in the previous commit builds libraries in
both PIC and non-PIC mode. Shared libraries don't work in PIC mode, so
it expects a --disable-shared configure option, which most open source
libraries using autoconf have. Let's add it, too.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We have a build system internally which only needs to build and install
the libraries out of a repository, not any binaries. There's no easy way
to do this in btrfs-progs currently. Add --disable-programs to
./configure to support this.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Prevent unnecessary error from failing fsync(), if opened read only.
Performed 'grep "writeable = " *.h *.c' to make sure there were no odd
situations where fsync() might still be desired here. They're all straight-
forward. The only situation where writeable will be 0 is if btrfs_open_devices
is given flags without O_RDWR. There is no situation where a writeable volume
temporarily becomes unwriteable, or anything like that. Given that it's being
opened O_RDWR, there's no reason to attempt fsync().
utils.c
int btrfs_add_to_fsid() {
...
device->writeable = 1;
volumes.c
int btrfs_close_devices() {
...
while (!list_empty(&fs_devices->devices)) {
...
// just after the fsync() being patched
267: device->writeable = 0;
...
int btrfs_open_devices() {
...
list_for_each_entry(device, &fs_devices->devices, dev_list) {
...
if (flags & O_RDWR)
332: device->writeable = 1
kernel btrfs_close_devices() does not have a corresponding fsync() that I see.
Signed-off-by: James Harvey <jamespharvey20@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
btrfs_next_extent_item() looks for BTRFS_EXTENT_ITEM_KEY and
BTRFS_METADATA_KEY, which are the types we're looking for.
Signed-off-by: James Harvey <jamespharvey20@gmail.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We reuse the task_position enum and task_ctx struct of the original progress
indicator, adding more values and fields for our needs.
Then add hooks in all steps of the check to properly record progress.
Here's how the output looks like on a 22 Tb 5-disk RAID1 FS:
Opening filesystem to check...
Checking filesystem on /dev/mapper/luks-ST10000VN0004-XXXXXXXX
UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
[1/7] checking extents (0:20:21 elapsed, 950958 items checked)
[2/7] checking root items (0:01:29 elapsed, 15121 items checked)
[3/7] checking free space cache (0:00:11 elapsed, 4928 items checked)
[4/7] checking fs roots (0:51:31 elapsed, 600892 items checked)
[5/7] checking csums (0:14:35 elapsed, 754522 items checked)
[6/7] checking root refs (0:00:00 elapsed, 232 items checked)
[7/7] checking quota groups skipped (not enabled on this FS)
found 5286458060800 bytes used, no error found
Signed-off-by: Stéphane Lesimple <stephane_btrfs@lesimple.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
This is a single-line fix on the preexisting task_period_start function.
Signed-off-by: Stéphane Lesimple <stephane_btrfs@lesimple.fr>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For trees have been balanced, leaves are with flag BTRFS_HEADER_FLAG_RELOC and
extent data backrefs are shared. Like:
=====================
item 0 key (11927552 EXTENT_ITEM 524288) itemoff 3932 itemsize 63
refs 129 gen 7 flags DATA
shared data backref parent 35897344 count 41
shared data backref parent 35426304 count 37
shared data backref parent 35422208 count 51
=====================
Then make the leaf which owns the extent data cowed. The shared data
backref was to transferred to keyed data ref, but remaining backrefs are
still shared. Like:
=====================
item 0 key (11927552 EXTENT_ITEM 524288) itemoff 3887 itemsize 108
refs 129 gen 7 flags DATA
extent data backref root 5 objectid 258 offset 0 count 40
extent data backref root 5 objectid 257 offset 0 count 1
shared data backref parent 35426304 count 37
shared data backref parent 35422208 count 51
=====================
However lowmem mode used to iterate the whole inode to find all
references, and doesn't care if a reference is already counted by the
shared tree block.
Add the test case to check it.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Btrfs lowmem check reports such false alerts:
=========
ERROR: extent[1419709677568, 1703936] referencer count mismatch (root: 2192, owner: 327635, offset: 0) wanted: 9, have: 13
=========
While in extent tree, the extent has:
=========
item 98 key (1419709677568 EXTENT_ITEM 1703936) itemoff 10745 itemsize 92
refs 35 gen 16921 flags DATA
extent data backref root 2192 objectid 327635 offset 0 count 9
shared data backref parent 1232784752640 count 4
shared data backref parent 1232784736256 count 13
shared data backref parent 1232784719872 count 9
=========
The extent data 327635 starts in this leaf without BTRFS_HEADER_FLAG_RELOC:
=========
leaf 1471521390592 items 146 free space 3671 generation 18581 owner 2192
leaf 1471521390592 flags 0x1(WRITTEN) backref revision 1
...
item 137 key (327635 EXTENT_DATA 0) itemoff 7745 itemsize 53
generation 15770 type 1 (regular)
extent data disk byte 1419709677568 nr 1703936
extent data offset 0 nr 131072 ram 1703936
extent compression 0 (none)
item 138 key (327635 EXTENT_DATA 131072) itemoff 7692 itemsize 53
generation 15770 type 1 (regular)
extent data disk byte 1419709677568 nr 1703936
extent data offset 131072 nr 131072 ram 1703936
extent compression 0 (none)
...(Exactly 9 items here)
=========
The next leaf is flaged as relocated, so extent data backrefs are
shared not keyed:
=========
leaf 1232784752640 items 159 free space 2503 generation 16924 owner 2192
leaf 1232784752640 flags 0x3(WRITTEN|RELOC) backref revision 1
fs uuid 0f43f49d-6e63-4b1b-bc8c-c54da409872d
chunk uuid e558e6f7-4f08-4292-91fb-9a775fdd530b
item 0 key (327635 EXTENT_DATA 1179648) itemoff 16230 itemsize 53
generation 15770 type 1 (regular)
extent data disk byte 1419709677568 nr 1703936
extent data offset 1179648 nr 131072 ram 1703936
extent compression 0 (none)
item 1 key (327635 EXTENT_DATA 1310720) itemoff 16177 itemsize 53
generation 15770 type 1 (regular)
extent data disk byte 1419709677568 nr 1703936
extent data offset 1310720 nr 131072 ram 1703936
extent compression 0 (none)
item 2 key (327635 EXTENT_DATA 1441792) itemoff 16124 itemsize 53
generation 15770 type 1 (regular)
extent data disk byte 1419709677568 nr 1703936
extent data offset 1441792 nr 131072 ram 1703936
extent compression 0 (none)
item 3 key (327635 EXTENT_DATA 1572864) itemoff 16071 itemsize 53
=======
Lowmem mode starts to count extent data backrefs in first leaf, those
backrefs are keyed as wanted.
It calls btrfs_next_item to skip to next leaf.
However, the next leaf was relocated, extent data items in this leaf
should have shared backrefs shouldn't be counted.
So let lowmem mode do not count data backrefs if leaf is flaged with
BTRFS_HEADER_FLAG_RELOC.
Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This fuzzed image will not only cause kernel BUG_ON(), but also btrfs
check BUG_ON() for original mode.
Checking filesystem on /home/adam/btrfs/crafted_images/runtime/0.img
UUID: 3381d111-94a3-4ac7-8f39-611bbbdab7e6
checking extents
check/main.c:3677: check_owner_ref: BUG_ON `rec->is_root` triggered, value 1
btrfs(+0x572c2)[0x562d65da72c2]
btrfs(+0x6098d)[0x562d65db098d]
btrfs(+0x60bb6)[0x562d65db0bb6]
btrfs(+0x6179b)[0x562d65db179b]
btrfs(cmd_check+0x1199)[0x562d65db5589]
btrfs(main+0x88)[0x562d65d62768]
/usr/lib/libc.so.6(__libc_start_main+0xeb)[0x7f4fcbb1b06b]
btrfs(_start+0x2a)[0x562d65d6288a]
Link: https://bugzilla.kernel.org/show_bug.cgi?id=200403
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
With crafted image, expected root item can refer to certain extent, and
original mode uses BUG_ON() to handle such case.
Fix it by gracefully return error.
Link: https://bugzilla.kernel.org/show_bug.cgi?id=200403
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
BUG_ON() can be triggered if some image contains overlappin chunks.
volumes.c:1930: read_one_chunk: BUG_ON `ret` triggered, value -17
btrfs(+0x2cf12)[0x5601efa17f12]
btrfs(+0x2fd8b)[0x5601efa1ad8b]
btrfs(btrfs_read_chunk_tree+0x2bf)[0x5601efa1b30f]
btrfs(btrfs_setup_chunk_tree_and_device_map+0xe8)[0x5601efa07718]
btrfs(+0x1c944)[0x5601efa07944]
btrfs(open_ctree_fs_info+0x90)[0x5601efa07b90]
btrfs(cmd_check+0x4d7)[0x5601efa4f8c7]
btrfs(main+0x88)[0x5601ef9fd768]
/usr/lib/libc.so.6(__libc_start_main+0xeb)[0x7f3c7787306b]
btrfs(_start+0x2a)[0x5601ef9fd88a]
Extent cache code can already detect it without problems, we only need
to remove the BUG_ON() and exit gracefully.
Reported-by: Xu Wen <wen.xu@gatech.edu>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=200409
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a boolean to record whether the extent tree is being re-initialised
in the current transaction. This is going to be needed by the
delayed refs code.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently some instances of btrfs_free_extent are called with the
last parameter ("offset") being set to 1. This makes no sense, since
offset is used for data extents. I suspect this is a left-over from
95d3f20b51 ("Mixed back reference (FORWARD ROLLING FORMAT CHANGE)")
since this commit changed the signature of the function from :
-int btrfs_free_extent(struct btrfs_trans_handle *trans, struct btrfs_root
- *root, u64 bytenr, u64 num_bytes, u64 parent,
- u64 root_objectid, u64 ref_generation,
- u64 owner_objectid, int pin);
to
+int btrfs_free_extent(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root,
+ u64 bytenr, u64 num_bytes, u64 parent,
+ u64 root_objectid, u64 owner, u64 offset);
I.e the last parameter was "pin" and not offset. So these are just
leftovers with no semantic meaning. Fix this by passing 0.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This is not really needed, since we can reference the fs_info from the
passed transaction. This is in preparation for delayed-refs support.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This argument is no longer used in this function so remove it.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
They are not really needed, what free_extent_hook wants is really a
pointer to fs_info so give it to it directly. This is in preparation
of delayed refs code.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This is in preparation of delayed refs code.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Instead of updating this during update_block_group, move the updating
code at the places where we free/allocate a block. This resembles the
current state of the kernel code. This is in prep for delayed refs.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Pull the necessary function, excluding locking. Required to enable
integration of delayed refs.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
It's not needed, since we can obtain a reference to fs_info from the
passed transaction handle. This is needed by delayed refs code.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This argument is used to obtain a reference to fs_info, which can
already be done from the passed trans handle, so use that instead.
This is in preparation for delayed refs support.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
It does not 't build anymore and we don't have any use for it.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When btrfs check detects a freespace tree extent which ends beyond the
blockgroup containing it a misleading error messages is printed. For
example if we have the following extent in the freespace tree:
item 5 key (30408704 FREE_SPACE_INFO 1073741824) itemoff 16259 itemsize 8
free space info extent count 3 flags 0
item 6 key (30425088 FREE_SPACE_EXTENT 49152) itemoff 16259 itemsize 0
free space extent
item 7 key (30507008 FREE_SPACE_EXTENT 65536) itemoff 16259 itemsize 0
free space extent
item 8 key (30654464 FREE_SPACE_EXTENT 14524648038063310901) itemoff 16259 itemsize 0
Clearly the last extent is corrupted so we should print something
along the lines of:
free space extent ends at 14524648038063310901, beyond end of block group 30408704-1104150528
Instead currently this is printed:
free space extent ends at 30654464, beyond end of block group 30408704-1104150528
So instead of printing the actual erroneous end, we print the beginning
of the extent. Fix this by printing the actual corrupted end.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For easier debugging, let print_tree_block_error() print bytenr of tree
block.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Update the information to reflect the status of 4.18
Main Updates:
- Add explanation of improved compression heuristic algorithm
- Add explanation that norecovery == nologreplay
- Add explanation of nossd_spread mount option
- Add explanation of rmdir_subovl feature
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
[ minor updates ]
Signed-off-by: David Sterba <dsterba@suse.com>
Don't panic for btrfs_read_chunk_tree() if one device or chunk is
corrupted.
Caller can already handle it pretty well.
Link: https://bugzilla.kernel.org/show_bug.cgi?id=199839
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The experimental flag is already carried in the manpage, but was removed
from the btrfs check usage message as part of refactoring via
87c1bd13c1. Add it back.
Signed-off-by: David Disseldorp <ddiss@suse.de>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Similar to the original mode repair.
Reported-by: Steve Leung <sjleung@shaw.ca>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Current check_file_extent() doesn't support later repair work, since it
doesn't accept btrfs_path structure as parameter, thus it can't modify
btrfs trees, or later check will still use the old and wrong path.
Use btrfs_path to replace btrfs_key, extent_buffer and slot parameters,
so we can modify @path directly for repair, and reduce the number of
parameters for check_file_extent().
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
It looks like that around 2014, btrfs kernel has a regression that would
cause offset-by-one ram_bytes for inline extent.
Add the ability to repair it in original mode.
Reported-by: Steve Leung <sjleung@shaw.ca>
Tested-by: Steve Leung <sjleung@shaw.ca>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
If one uncompressed inline extent has incorrect ram_bytes, neither btrfs
check nor dump-tree could detect such corruption.
[CAUSE]
Every caller tries to read inline extent ram_bytes is using
btrfs_file_extent_inline_len(), other than directly calling
btrfs_file_extent_ram_bytes().
For compressed extent, it's just calling btrfs_file_extent_ram_bytes().
However for uncompressed extent, it falls back to
btrfs_file_extent_inline_item_len(), makes us unable to detect anything
wrong in ram_bytes.
[FIX]
Just get rid of such confusing btrfs_file_extent_inline_len() function.
Reported-by: Steve Leung <sjleung@shaw.ca>
Tested-by: Steve Leung <sjleung@shaw.ca>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When using decompress() in copy_one_inline(), we're passing the
decompressed extent size (ram_bytes) into decompress().
However we only has @inline_item_len read out, the pending data will be
uninitialized data.
Thankfully, all compression methods supported have some extra data in
its header, thus it won't cause real problem.
Whatever fixing it is never a bad idea.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In btrfs_add_free_space(), if the free space to be added is already
here, we trigger ASSERT() which is just another BUG_ON().
Let's remove such BUG_ON() at all.
Reported-by: Lewis Diamond <me@lewisdiamond.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo