Gcc 9 adds the flag to default warnings and this produces a lot of
warnings that don't seem to be harmful as we know the address is
aligned, but this could be hidden in the function call chain.
It's still available under W=1.
Issue: #180
Signed-off-by: David Sterba <dsterba@suse.com>
Switch the function to take a key instead of objectid and offset
separately, gcc 9 complains with enabled -Waddress-of-packed-member:
[CC] ctree.o
ctree.c: In function ‘btrfs_uuid_tree_lookup’:
ctree.c:3133:26: error: taking address of packed member of ‘struct btrfs_key’ may result in an unaligned pointer value [-Werror=address-of-packed-member]
3133 | btrfs_uuid_to_key(uuid, &key.objectid, &key.offset);
Signed-off-by: David Sterba <dsterba@suse.com>
GCC9 9.1.1 with -Waddress-of-packed-member warns about passing an
unaligned pointer from btrfs_dev_stats_values. It is up to the caller to
access the array properly, which does happen in print_dev_stats.
In file included from print-tree.c:24:
ctree.h: In function ‘btrfs_dev_stats_values’:
ctree.h:2408:9: warning: taking address of packed member of ‘struct btrfs_dev_stats_item’ may result in an unaligned pointer value [-Waddress-of-packed-member]
2408 | return p->values;
Drop the helper as print-tree.c is the only user and access the dev stat
values using the unaligned helper.
Signed-off-by: David Sterba <dsterba@suse.com>
Cleanups:
- add quotation
- use _log for messages
- unify spelling of metadata_uuid as feature and incompat bit
- add -- separators
- minor whitespace adjustments
Signed-off-by: David Sterba <dsterba@suse.com>
The directories created under default path needs a fallback in case it's
on NFS and root cannot create the directory. Update the test to create
only one additional mount point and use the TEST_MNT as the toplevel
one.
The error message we're looking for is now a bit weaker due to the less
specific path.
Signed-off-by: David Sterba <dsterba@suse.com>
Lots of test opencode the mkfs phase with no special needs, add a helper
that forcibly creates fileystem on TEST_DEV. Any options can be added,
except devices.
Signed-off-by: David Sterba <dsterba@suse.com>
In github issues, one user reports unexpected ENOSPC error if enabling
datasum druing convert. After some investigation, it looks like that
during ext2_saved/image creation, we could create large file extent
whose size can be 128M (max data extent size).
In that case, its csum block will be at least 128K. Under certain case
we need to allocate extra metadata chunks to fulfill such space
requirement.
However we only do metadata prealloc if we're reserving extents for fs
trees. (we use btrfs_root::ref_cows to determine whether we should do
metadata prealloc, and that member is only set for fs trees).
There is no explaination on why we only do metadata prealloc for file
trees, but from my educated guess, it could be related to avoid nested
extent/chunk tree modication.
At least extent reservation for csum tree shouldn't be a problem with
metadata block group preallocation.
So adding new condition for metadata preallocate to avoid unexpected
ENOSPC problem.
Issue: #123
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is a indirect recursion which can reach the extent reservation:
btrfs_reserve_extent() <--|
|- do_chunk_alloc() |
|- btrfs_alloc_chunk() |
|- btrfs_insert_item() |
|- btrfs_reserve_extent() <--|
Currently, we're using root->ref_cows to determine whether we should do
chunk prealloc to avoid such loop.
But that's still a hidden trap. Instead of solving it using some hidden
tricks, this patch will make chunk/block group allocation exclusive.
Now if do_chunk_alloc() determines to alloc chunk, it will set a flag in
transaction handle so new call of do_chunk_alloc() will refuse to
allocate new chunk until current chunk allocation finishes.
The chunks get over-allocated by 2M so there's enough space in case the
recursive call asks for a different type of blockgroup.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This ensures that 'btrfs filesystem show' can correctly identify a
filesystem on a newly created local file.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When btrfs' 'filesystem' subcommand is passed path to an image file it
currently fails since the code expects the image file is going to be
recognised by libblkid (called from btrfs_scan_devices()). This is not
the case since libblkid only scan well-known locations under /dev.
Fix this by explicitly calling open_ctree which will correctly open the
image and add it to the correct btrfs_fs_devices struct. This allows
subsequent cmd_filesystem_show logic to correctly show requested
information.
Issue: #169
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In lowmem mode, we check fs roots and free space cache by iterating
each root item and inode item, using btrfs_next_item() and a path
pointing to the root tree.
However in repair mode, check_fs_root() can modify the fs root, thus
CoWs the tree root, and the old path in check_fs
It could lead to strange behavior, e.g. after repairing a fs tree, the
path can point to a fs tree.
Since no ROOT_ITEM exists in fs tree, all remaining trees are skipped in
repair mode.
This bug exists from the early time of lowmem mode repair, and is only
exposed by recent free space inode check code. (Fs tree inodes are
passed to free space inode check, causing false alerts and repair
failure).
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
BTRFS_COMPAT_EXTENT_TREE_V0 is introduced for a short time in kernel,
and it's over 10 years ago.
Nowadays there should be no user for that feature, and kernel has remove
this support in Jun, 2018. There is no need for btrfs-progs to support
it.
This patch will remove EXTENT_TREE_V0 related code and replace those
BUG_ON() to a more graceful error message.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
As the link reported, btrfs fi sh may crash while a device is removing.
valgrind reported:
======================================================================
...
==883== Invalid write of size 8
==883== at 0x13C99A: get_device_info (in /usr/bin/btrfs)
==883== by 0x13D715: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883== Address 0x4d8c7a0 is 0 bytes after a block of size 12,288 alloc'd
==883== at 0x483877F: malloc (vg_replace_malloc.c:299)
==883== by 0x13D861: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883==
==883== Invalid write of size 8
==883== at 0x13C99D: get_device_info (in /usr/bin/btrfs)
==883== by 0x13D715: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883== Address 0x4d8c7a8 is 8 bytes after a block of size 12,288 alloc'd
==883== at 0x483877F: malloc (vg_replace_malloc.c:299)
==883== by 0x13D861: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883==
==883== Syscall param ioctl(generic) points to unaddressable byte(s)
==883== at 0x4CA9CBB: ioctl (in /usr/lib/libc-2.29.so)
==883== by 0x13C9AB: get_device_info (in /usr/bin/btrfs)
==883== by 0x13D715: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883== Address 0x4d8c7a0 is 0 bytes after a block of size 12,288 alloc'd
==883== at 0x483877F: malloc (vg_replace_malloc.c:299)
==883== by 0x13D861: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883==
--883-- VALGRIND INTERNAL ERROR: Valgrind received a signal 11 (SIGSEGV) - exiting
--883-- si_code=1; Faulting address: 0x284D8C7B8; sp: 0x1002eb5e50
valgrind: the 'impossible' happened:
Killed by fatal signal
host stacktrace:
==883== at 0x5805261C: get_bszB_as_is (m_mallocfree.c:303)
==883== by 0x5805261C: get_bszB (m_mallocfree.c:315)
==883== by 0x5805261C: vgPlain_arena_malloc (m_mallocfree.c:1799)
==883== by 0x58005AD2: vgMemCheck_new_block (mc_malloc_wrappers.c:372)
==883== by 0x58005AD2: vgMemCheck_malloc (mc_malloc_wrappers.c:407)
==883== by 0x580A7373: do_client_request (scheduler.c:1925)
==883== by 0x580A7373: vgPlain_scheduler (scheduler.c:1488)
==883== by 0x580F57A0: thread_wrapper (syswrap-linux.c:103)
==883== by 0x580F57A0: run_a_thread_NORETURN (syswrap-linux.c:156)
sched status:
running_tid=1
Thread 1: status = VgTs_Runnable (lwpid 883)
==883== at 0x483877F: malloc (vg_replace_malloc.c:299)
==883== by 0x1534AA: ??? (in /usr/bin/btrfs)
==883== by 0x153C49: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
client stack range: [0x1FFEFFA000 0x1FFF000FFF] client SP: 0x1FFEFFDCE0
valgrind stack range: [0x1002DB6000 0x1002EB5FFF] top usage: 7520 of 1048576
======================================================================
The above log says that invalid write to allocated @di_args happened
in get_device_info() called in get_fs_info().
The size of @di_args is allocated according by fi_args->num_devices.
And fi_args->num_devices is *the number of dev_items in chunk_tree*.
However, in the loop to get devices info, btrfs-progs calls ioctl
BTRFS_IOC_DEV_INFO which just finds device in
fs_info->fs_devices->devices.
Let's look at kernel side.
In btrfs_rm_device(), btrfs_rm_dev_item() causes removal of
related dev_items in chunk_tree. *Do something*.
Then delete the device from device->fs_devices.
So the case is:
Userspace kernel
get_fs_info() btrfs_rm_device()
...
btrfs_rm_dev_item()
determine fi_args->num_devices and
fi_args->max_id by seraching chunk_tree.
malloc() ...
Loop(Crashed): call get_device_info() by devid
from 1 to fi_args->max_id.
mutex_lock(&fs_devices->device_list_mutex);
list_del_rcu(&device->dev_list);
...
In the loop of get_device_info(), get_device_info() still can get info
of the removing device since it's still in fs_info->fs_devices->devices.
Then the iterator value @ndev increaments causes invalid access out of
bounds.
Solved it by adding the check of @ndev while looping.
Reported-by: Peter Hjalmarsson <kanelxake@gmail.com>
Link: https://bugzilla.redhat.com/show_bug.cgi?id=1711787
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Although moderm hardware is fast enough and crc32c calculation is not a
hotspot, doing such optimization won't hurt anyway.
Issue: #175
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is a bug report of BUG_ON() which is caused by __free_extent()
failed to lookup a backref extent:
Failed to find [1429288337408, 168, 16384]
btrfs unable to find ref byte nr 1429288583168 parent 0 root 2 owner 0 offset 0
convert/source-ext2.c:834: ext2_copy_inodes: BUG_ON ret triggered, value -5
./btrfs-convert[0x410941]
./btrfs-convert(main+0x1fdc)[0x40d3b8]
/lib64/libc.so.6(__libc_start_main+0xf3)[0x7f93bb7d2f33]
./btrfs-convert(_start+0x2e)[0x40a96e]
It's still unclear how this bug can be triggered, but adding such debug
output will provide more info for us to debug.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test misc-tests/035-receive-common-mount-point-prefix does another
mount inside TEST_MNT but current 'make test-clean' will not properly
undo the nested mount and this will break subsequent tests. The
recursive unmount can handle that.
Signed-off-by: David Sterba <dsterba@suse.com>
In convert we use trans->block_reserved >= 4096 as a threshold to commit
transaction, where block_reserved is the number of new tree blocks
allocated inside a transaction.
The problem is, we still have a hidden bug in delayed ref implementation
in btrfs-progs, when we have a large enough transaction, delayed ref may
failed to find certain tree blocks in extent tree and cause transaction
abort.
This fix will workaround it by committing transaction at a much lower
threshold.
The old 4096 means 4096 new tree blocks, when using default (16K)
nodesize, it's 64M, which can contain over 12k inlined data extent or
csum for around 60G, or over 800K file extents.
The new threshold will limit the size of new tree blocks to 2M, aligning
with the chunk preallocator threshold, and reducing the possibility to
hit that delayed ref bug.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
libbtrfs.so already has user's LDFLAGS applied. The change also applies
those to libbtrfsutil.so. A separate variable is used for that though it
currently only copies LDFLAGS. This is to make it obvious that
libbtrfsutils is a standalone library.
Reported-by: Michał Górny
Bug: https://bugs.gentoo.org/686284
Pull-request: #172
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
Adds Make variables EXTRA_PYTHON_CFLAGS and EXTRA_PYTHON_LDFLAGS which
can be used to pass CFLAGS and LDFLAGS respectively when building the
Python library.
This is required to support reproducible builds, as there are often
compiler and linker flags that must be passed in order to generate
reproducible output (e.g. -fdebug-prefix-map)
Pull-request: #176
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is one report of compressed extent happens in btrfs, but has no
csum and then leads to possible decompress error screwing up kernel
memory.
Although it's a kernel bug, and won't cause problem until compressed
data get corrupted, let's catch such problem in advance.
This patch will catch any unexpected compressed extent with:
1) 0 or less than expected csum
2) nodatasum flag set in the inode item
This is for original mode.
Reported-by: James Harvey <jamespharvey20@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is one report of compressed extent happens in btrfs, but has no
csum and then leads to possible decompress error screwing up kernel
memory.
Although it's a kernel bug, and won't cause problem until compressed
data get corrupted, let's catch such problem in advance.
This patch will catch any unexpected compressed extent with:
1) missing csum
2) nodatasum flag set in the inode item
This is for lowmem mode.
Reported-by: James Harvey <jamespharvey20@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The command
$ printf 'btrfs-stream\0\0\0\0\0' | btrfs receive --dump
can loop as the stream is not valid, but the maximum error limit is not
set properly for --dump. The command line parameter -E applies here too,
so it's still possible to dump partially damanged stream.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=200085
Author: Alexander Kovtunenko <akovtunenko@slice.com>
Signed-off-by: David Sterba <dsterba@suse.com>
blkid_get_cache() returns error code which is -errno. So we can use them
directly.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently, when operating in a more verbose mode (-vv), the receive
command does not mention any write or clone commands, unlike other
commands.
This change adds debug messages for the write and clone operations, that
do not include data but only offsets and lengths, as this is actually
very useful to debug a send stream and I use it frequently.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When repairing a file system created by a very old kernel, I ran into
issues fixing up the extent flags since fixup_extent_flags assumed
that a METADATA_ITEM would be present if the record was for metadata.
Since METADATA_ITEMs don't exist without skinny metadata, we need to
fall back to EXTENT_ITEMs. This also falls back to EXTENT_ITEMs even
with skinny metadata enabled as other parts of the tools do.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When running fuzz-tests/003 and fuzz-tests/009, btrfs-progs will crash
due to BUG_ON().
[CAUSE]
We abused BUG_ON() in btrfs_commit_transaction(), which is one of the
most error prone function for fuzzed images.
Currently to cleanup the aborted transaction, we only need to clean up
the only per-transaction data: delayed refs.
This patch will introduce a new function, btrfs_destroy_delayed_refs()
to cleanup delayed refs when we failed to commit transaction.
With that function, we will gently destroy per-trans delayed ref, and
remove the BUG_ON()s in btrfs_commit_transaction().
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch will refactor btrfs_finish_extent_commit():
- Make it return void
There is no failure pattern for btrfs_finish_extent_commit(), thus it
always return 0. And the caller doesn't care about the return value.
So no need to return int.
- Remove @root and @unpin parameters
@root is only used to extract fs_info, which can be extracted from
transaction handler already.
@unpin is always fs_info->pinned_extents.
All these parameters can be extracted from @trans, no need to pass
them.
The function signature now matches the kernel counterpart.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
cleanup_ref_head() will only return 0 or 1, no way to return a negative
value. So remove the dead branch.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The temporary files are not accessible if the testsuite is hosted on
NFS, pre-create them and allow writes.
Signed-off-by: David Sterba <dsterba@suse.com>
The caller owns the fd passed to btrfs_util_subvolume_id_fd(), so we
shouldn't close it on error. Fix it, add a regression test, and bump the
library patch version.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The branch passes all selftests, except:
- misc/035
Known bug as the fix is reverted.
- fuzz/003
- fuzz/009
Not a regression, as stable tags also triggers them.
BUG_ON() in commit_transaction get triggered due to ENOSPC.
These two bugs will be addressed soon. but not in this pull.
This pull request include the following features:
Core change:
- check --repair
* Flush/FUA support to avoid breaking metadata CoW
Now btrfs-progs crashing or transaction aborted won't cause
new transid error.
Fixes and Enhancement:
- generic
* Try best copy when reading tree blocks.
* Skip unnecessary retry when one tree block copy fails.
* Avoid back tree block to populate tree block cache.
* Don't BUG_ON() when failed to flush/write super blocks
- check
* File extents repair no longer relies data in extent tree.
* New ability to check and repair free space cache invalid inode mode.
* Update backup roots when commit transaction.
- Misc
* fs_info <-> root parameters cleanup for btrfs_check_leaf/node()
Commit 7a12d8470e ("btrfs-progs: Do metadata preallocation as long as
we're not modifying extent tree") tries to fix#123, however due to the
fact that chunk tree also has root->ref_cows set, we will call
do_chunk_alloc() until call stack explodes.
So revert that offending patch until we have a much better comment on
root->ref_cows and find a better solution to this problem.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[BUG]
Since commit "btrfs-progs: disk-io: Flush to ensure super block write is
FUA" mkfs-tests/017 will fail like:
====== RUN MUSTFAIL /home/adam/btrfs-progs/mkfs.btrfs -K -f /dev/mapper/btrfs-progs-thin-vol
ERROR: failed to write super block for devid 1: flush error: Input/output error
disk-io.c:1810: write_all_supers: BUG_ON `ret` triggered, value -5
/home/adam/btrfs-progs/mkfs.btrfs(+0x1e5c1)[0x557a2c83e5c1]
/home/adam/btrfs-progs/mkfs.btrfs(+0x1e65f)[0x557a2c83e65f]
/home/adam/btrfs-progs/mkfs.btrfs(write_all_supers+0x1ce)[0x557a2c843a8a]
/home/adam/btrfs-progs/mkfs.btrfs(write_ctree_super+0x12d)[0x557a2c843be2]
/home/adam/btrfs-progs/mkfs.btrfs(btrfs_commit_transaction+0x250)[0x557a2c887c56]
/home/adam/btrfs-progs/mkfs.btrfs(+0xc0b1)[0x557a2c82c0b1]
/home/adam/btrfs-progs/mkfs.btrfs(main+0x1049)[0x557a2c82e929]
/usr/lib/libc.so.6(__libc_start_main+0xf3)[0x7f6689e99223]
/home/adam/btrfs-progs/mkfs.btrfs(_start+0x2e)[0x557a2c82b86e]
failed (expected): /home/adam/btrfs-progs/mkfs.btrfs -K -f /dev/mapper/btrfs-progs-thin-vol
[CAUSE]
Just one BUG_ON() in write_all_supers().
[FIX]
Just remove the BUG_ON(). Callers of write_all_supers() are already
checking the return value.
Also since write_all_supers() can return error, make write_ctree_super()
callers, btrfs_commit_transaction() and close_ctree_fs_info() to
handle the error correctly.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
[BUG]
There are tons of reports of btrfs-progs screwing up the fs, the most
recent one is "btrfs check --clear-space-cache v1" triggered BUG_ON()
and then leaving the fs with transid mismatch problem.
[CAUSE]
In kernel, we have block layer handing the flush work, even on devices
without FUA support (like most SATA device using default libata
settings), kernel handles FUA write by flushing the device, then normal
write, and finish it with another flush.
The pre-flush, write, post-flush works pretty well to implement FUA
write.
However in btrfs-progs we just use pwrite(), there is nothing keeping
the write order.
So even for basic v1 free space cache clearing, we have different vision
on the write sequence from kernel bio layer (by dm-log-writes) and user
space pwrite() calls.
In btrfs-progs, with extra debug output in write_tree_block() and
write_dev_supers(), we can see btrfs-progs follows the right write
sequence:
Opening filesystem to check...
Checking filesystem on /dev/mapper/log
UUID: 3feb3c8b-4eb3-42f3-8e9c-0af22dd58ecf
write tree block start=1708130304 gen=39
write tree block start=1708146688 gen=39
write tree block start=1708163072 gen=39
write super devid=1 gen=39
write tree block start=1708179456 gen=40
write tree block start=1708195840 gen=40
write super devid=1 gen=40
write tree block start=1708130304 gen=41
write tree block start=1708146688 gen=41
write tree block start=1708228608 gen=41
write super devid=1 gen=41
write tree block start=1708163072 gen=42
write tree block start=1708179456 gen=42
write super devid=1 gen=42
write tree block start=1708130304 gen=43
write tree block start=1708146688 gen=43
write super devid=1 gen=43
Free space cache cleared
But from dm-log-writes, the bio sequence is a different story:
replaying 1742: sector 131072, size 4096, flags 0(NONE)
replaying 1743: sector 128, size 4096, flags 0(NONE) <<< Only one sb write
replaying 1744: sector 2828480, size 4096, flags 0(NONE)
replaying 1745: sector 2828488, size 4096, flags 0(NONE)
replaying 1746: sector 2828496, size 4096, flags 0(NONE)
replaying 1787: sector 2304120, size 4096, flags 0(NONE)
......
replaying 1790: sector 2304144, size 4096, flags 0(NONE)
replaying 1791: sector 2304152, size 4096, flags 0(NONE)
replaying 1792: sector 0, size 0, flags 8(MARK)
During the free space cache clearing, we committed 3 transaction but
dm-log-write only caught one super block write.
This means all the 3 writes were merged into the last super block write.
And the super block write was the 2nd write, before all tree block
writes, completely screwing up the metadata CoW protection.
No wonder crashed btrfs-progs can make things worse.
[FIX]
Fix this super serious problem by implementing pre and post flush for
the primary super block in btrfs-progs.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
When we failed to write super blocks, we just output something like:
WARNING: failed to write sb: I/O error
Or
WARNING: failed to write all sb data
There is no info about which device failed and there are two different
error message for the same write error.
This patch will change it to something more detailed:
ERROR: failed to write super block for devid 1: write error: I/O error
This provides the basis for later super block flush error handling.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
The image has one free space cache inode with invalid mode (0).
item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
generation 30 transid 30 size 65536 nbytes 1507328
block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
atime 0.0 (1970-01-01 08:00:00)
ctime 1553491158.189771625 (2019-03-25 13:19:18)
mtime 0.0 (1970-01-01 08:00:00)
otime 0.0 (1970-01-01 08:00:00)
Both lowmem and original mode should be able to detect and fix it.
The extracted test image is pretty big (1G extracted), as kernel won't
cache small chunks.
Even with SSD, such test may still take some seconds just extracting the
image.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Just like lowmem mode, also check and repair free space cache inode
item.
And since we don't really have a good timing/function to check free
space chace inodes, we use the same common mode
check_repair_free_space_inode() when iterating root tree.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Unlike inodes in fs roots, we don't really check the inode items in root
tree, in fact we just skip everything other than ROOT_ITEM and ROOT_REF.
This makes invalid inode items sneak into root tree.
For example:
item 9 key (256 INODE_ITEM 0) itemoff 13702 itemsize 160
generation 30 transid 30 size 65536 nbytes 1507328
block group 0 mode 0 links 1 uid 0 gid 0 rdev 0
^ Should be 100600
sequence 23 flags 0x1b(NODATASUM|NODATACOW|NOCOMPRESS|PREALLOC)
atime 0.0 (1970-01-01 08:00:00)
ctime 1553491158.189771625 (2019-03-25 13:19:18)
mtime 0.0 (1970-01-01 08:00:00)
otime 0.0 (1970-01-01 08:00:00)
There is a report of such problem in the mail list.
This patch will check and repair inode items of free space cache inodes in
lowmem mode.
Since free space cache inodes doesn't have INODE_REF but still has 1
link, we can't use check_inode_item() directly.
Instead we only check the inode mode, as that's the important part.
The check and repair function: check_repair_free_space_inode() is also
exported for original mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
David Sterba