The only mechanism we have in the progs for searching qgroups is to load
all of them and filter the results. This works for qgroup show but
to add quota information to 'btrfs subvoluem show' it's pretty wasteful.
This patch splits out setting up the search and performing the search so
we can search for a single qgroupid more easily. Since TREE_SEARCH
will give results that don't strictly match the search terms, we add
a filter to match only the results we care about.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We use structures to pass the info and limit from the kernel as items
but store the individual values separately in btrfs_qgroup. We already
have a btrfs_qgroup_limit structure that's used for setting the limit.
This patch introduces a btrfs_qgroup_info structure and uses that and
btrfs_qgroup_limit in btrfs_qgroup.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When compiling btrfs-progs with GCC 9 (9.1.0), we got the following
warnings:
In file included from utils.h:30,
from cmds-receive.c:45:
cmds-receive.c: In function 'process_subvol':
messages.h:42:3: warning: '%s' directive argument is null [-Wformat-overflow=]
42 | __btrfs_error((fmt), ##__VA_ARGS__); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cmds-receive.c:178:3: note: in expansion of macro 'error'
178 | error("subvol: another one already started, path buf: %s",
| ^~~~~
[CC] cmds-inspect-tree-stats.o
cmds-receive.c: In function 'process_snapshot':
messages.h:42:3: warning: '%s' directive argument is null [-Wformat-overflow=]
42 | __btrfs_error((fmt), ##__VA_ARGS__); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cmds-receive.c:248:3: note: in expansion of macro 'error'
248 | error("snapshot: another one already started, path buf: %s",
| ^~~~~
[FIX]
We're using wrong member for the error output.
Fix the member to output.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
gcc 9.1 reports:
utils.c: In function ‘test_num_disk_vs_raid’:
utils.c:1931:3: warning: attribute ‘fallthrough’ not preceding a case label or default label
1931 | __attribute__ ((fallthrough));
| ^~~~~~~~~~~~~
As there's no other label to fall to (default: is the first one), remove the
annotation.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
GCC 9.1.0 will report the following error when compiling btrfs-progs:
In file included from print-tree.c:24:
ctree.h: In function 'btrfs_dev_stats_values':
ctree.h:2408:9: warning: taking address of packed member of 'struct btrfs_dev_stats_item' may result in an unaligned pointer value [-Waddress-of-packed-member]
2408 | return p->values;
| ^
[FIX]
Follow the kernel way of accessing dev stats by using
btrfs_dev_stats_value(eb, ptr, index).
So that we don't need to bother accessing the packed member.
This also unifies the helper function in kernel and btrfs-progs.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add const prefix for the following parameters:
- @eb of memcmp_extent_buffer()
- @eb of read_extent_buffer()
This backports kernel commit 1cbb1f454e53 ("btrfs: struct-funcs,
constify readers") to btrfs-progs.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When btrfs-progs is compiled with D=asan, it can't pass even the very
basic fsck tests due to btrfs-image has memory leak:
=== START TEST /home/adam/btrfs/btrfs-progs/tests//fsck-tests/001-bad-file-extent-bytenr
restoring image default_case.img
=================================================================
==7790==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 104 byte(s) in 1 object(s) allocated from:
#0 0x7f1d3b738389 in __interceptor_malloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cc:86
#1 0x560ca6b7f4ff in btrfs_add_delayed_tree_ref /home/adam/btrfs/btrfs-progs/delayed-ref.c:569
#2 0x560ca6af2d0b in btrfs_free_extent /home/adam/btrfs/btrfs-progs/extent-tree.c:2155
#3 0x560ca6ac16ca in __btrfs_cow_block /home/adam/btrfs/btrfs-progs/ctree.c:319
#4 0x560ca6ac1d8c in btrfs_cow_block /home/adam/btrfs/btrfs-progs/ctree.c:383
#5 0x560ca6ac6c8e in btrfs_search_slot /home/adam/btrfs/btrfs-progs/ctree.c:1153
#6 0x560ca6ab7e83 in fixup_device_size image/main.c:2113
#7 0x560ca6ab9279 in fixup_chunks_and_devices image/main.c:2333
#8 0x560ca6ab9ada in restore_metadump image/main.c:2455
#9 0x560ca6abaeba in main image/main.c:2723
#10 0x7f1d3b148ce2 in __libc_start_main (/usr/lib/libc.so.6+0x23ce2)
... tons of similar leakage for delayed_tree_ref ...
Direct leak of 96 byte(s) in 1 object(s) allocated from:
#0 0x7f1d3b738389 in __interceptor_malloc /build/gcc/src/gcc/libsanitizer/asan/asan_malloc_linux.cc:86
#1 0x560ca6b7f5fb in btrfs_add_delayed_tree_ref /home/adam/btrfs/btrfs-progs/delayed-ref.c:583
#2 0x560ca6af5679 in alloc_tree_block /home/adam/btrfs/btrfs-progs/extent-tree.c:2503
#3 0x560ca6af57ac in btrfs_alloc_free_block /home/adam/btrfs/btrfs-progs/extent-tree.c:2524
#4 0x560ca6ac115b in __btrfs_cow_block /home/adam/btrfs/btrfs-progs/ctree.c:290
#5 0x560ca6ac1d8c in btrfs_cow_block /home/adam/btrfs/btrfs-progs/ctree.c:383
#6 0x560ca6b7bb15 in commit_tree_roots /home/adam/btrfs/btrfs-progs/transaction.c:98
#7 0x560ca6b7c525 in btrfs_commit_transaction /home/adam/btrfs/btrfs-progs/transaction.c:192
#8 0x560ca6ab92be in fixup_chunks_and_devices image/main.c:2337
#9 0x560ca6ab9ada in restore_metadump image/main.c:2455
#10 0x560ca6abaeba in main image/main.c:2723
#11 0x7f1d3b148ce2 in __libc_start_main (/usr/lib/libc.so.6+0x23ce2)
... tons of similar leakage for delayed_ref_head ...
SUMMARY: AddressSanitizer: 1600 byte(s) leaked in 16 allocation(s).
failed to restore image ./default_case.img
[CAUSE]
Commit c6039704c5 ("btrfs-progs: Add delayed refs infrastructure")
introduces delayed ref infrastructure for free space tree, however the
refcount_dec_and_test() from kernel code is wrongly backported.
refcount_dec_and_test() will return true if the refcount reaches 0.
So kernel code will free the allocated space as expected:
if (refcount_dec_and_test(&ref->refs)) {
kmem_cache_free();
}
However btrfs-progs backport is using the opposite condition:
if (--ref->refs) {
kfree();
}
This will not free the memory for the last user, but for refs >= 2.
Causing both use-after-free and memory leak for any offline write
operation.
[FIX]
Fix the (--ref->refs) condition to (--ref->refs == 0) to fix the
backport error.
Fixes: c6039704c5 ("btrfs-progs: Add delayed refs infrastructure")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
There is a bug report of unexpected ENOSPC from btrfs-convert, issue #123.
After some debugging, even when we have enough unallocated space, we
still hit ENOSPC at btrfs_reserve_extent().
[CAUSE]
Btrfs-progs relies on chunk preallocator to make enough space for
data/metadata.
However after the introduction of delayed-ref, it's no longer reliable
to rely on btrfs_space_info::bytes_used and
btrfs_space_info::bytes_pinned to calculate used metadata space.
For a running transaction with a lot of allocated tree blocks,
btrfs_space_info::bytes_used stays its original value, and will only be
updated when running delayed ref.
This makes btrfs-progs chunk preallocator completely useless. And for
btrfs-convert/mkfs.btrfs --rootdir, if we're going to have enough
metadata to fill a metadata block group in one transaction, we will hit
ENOSPC no matter whether we have enough unallocated space.
[FIX]
This patch will introduce btrfs_space_info::bytes_reserved to track how
many space we have reserved but not yet committed to extent tree.
To support this change, this commit also introduces the following
modification:
- More comment on btrfs_space_info::bytes_*
To make code a little easier to read
- Export update_space_info() to preallocate empty data/metadata space
info for mkfs.
For mkfs, we only have a temporary fs image with SYSTEM chunk only.
Export update_space_info() so that we can preallocate empty
data/metadata space info before we start a transaction.
- Proper btrfs_space_info::bytes_reserved update
The timing is the as kernel (except we don't need to update
bytes_reserved for data extents)
* Increase bytes_reserved when call alloc_reserved_tree_block()
* Decrease bytes_reserved when running delayed refs
With the help of head->must_insert_reserved to determine whether we
need to decrease.
Issue: #123
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch will export disk-io.c::check_super() as btrfs_check_super()
and use it in btrfs-image for extra verification.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When there are over 32 (in my example, 35) online CPUs, btrfs-image -c9
will just hang.
[CAUSE]
Btrfs-image has a hard coded limit (32) on how many threads we can use.
For the "-t" option we do the up limit check.
But when we don't specify "-t" option and speicified "-c" option, then
btrfs-image will try to auto detect the number of online CPUs, and use
it without checking if it's over the up limit.
And for num_threads larger than the up limit, we will over write the
adjust members of metadump_struct/mdrestore_struct, corrupting
pthread_mutex_t and pthread_cond_t, causing synchronising problem.
Nowadays, with SMT/HT and higher cpu core counts, it's not hard to go
beyond 32 threads, and hit the bug.
[FIX]
Just do extra num_threads check before using the number from sysconf().
Reviewed-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Gcc 9 adds the flag to default warnings and this produces a lot of
warnings that don't seem to be harmful as we know the address is
aligned, but this could be hidden in the function call chain.
It's still available under W=1.
Issue: #180
Signed-off-by: David Sterba <dsterba@suse.com>
Switch the function to take a key instead of objectid and offset
separately, gcc 9 complains with enabled -Waddress-of-packed-member:
[CC] ctree.o
ctree.c: In function ‘btrfs_uuid_tree_lookup’:
ctree.c:3133:26: error: taking address of packed member of ‘struct btrfs_key’ may result in an unaligned pointer value [-Werror=address-of-packed-member]
3133 | btrfs_uuid_to_key(uuid, &key.objectid, &key.offset);
Signed-off-by: David Sterba <dsterba@suse.com>
GCC9 9.1.1 with -Waddress-of-packed-member warns about passing an
unaligned pointer from btrfs_dev_stats_values. It is up to the caller to
access the array properly, which does happen in print_dev_stats.
In file included from print-tree.c:24:
ctree.h: In function ‘btrfs_dev_stats_values’:
ctree.h:2408:9: warning: taking address of packed member of ‘struct btrfs_dev_stats_item’ may result in an unaligned pointer value [-Waddress-of-packed-member]
2408 | return p->values;
Drop the helper as print-tree.c is the only user and access the dev stat
values using the unaligned helper.
Signed-off-by: David Sterba <dsterba@suse.com>
Cleanups:
- add quotation
- use _log for messages
- unify spelling of metadata_uuid as feature and incompat bit
- add -- separators
- minor whitespace adjustments
Signed-off-by: David Sterba <dsterba@suse.com>
The directories created under default path needs a fallback in case it's
on NFS and root cannot create the directory. Update the test to create
only one additional mount point and use the TEST_MNT as the toplevel
one.
The error message we're looking for is now a bit weaker due to the less
specific path.
Signed-off-by: David Sterba <dsterba@suse.com>
Lots of test opencode the mkfs phase with no special needs, add a helper
that forcibly creates fileystem on TEST_DEV. Any options can be added,
except devices.
Signed-off-by: David Sterba <dsterba@suse.com>
In github issues, one user reports unexpected ENOSPC error if enabling
datasum druing convert. After some investigation, it looks like that
during ext2_saved/image creation, we could create large file extent
whose size can be 128M (max data extent size).
In that case, its csum block will be at least 128K. Under certain case
we need to allocate extra metadata chunks to fulfill such space
requirement.
However we only do metadata prealloc if we're reserving extents for fs
trees. (we use btrfs_root::ref_cows to determine whether we should do
metadata prealloc, and that member is only set for fs trees).
There is no explaination on why we only do metadata prealloc for file
trees, but from my educated guess, it could be related to avoid nested
extent/chunk tree modication.
At least extent reservation for csum tree shouldn't be a problem with
metadata block group preallocation.
So adding new condition for metadata preallocate to avoid unexpected
ENOSPC problem.
Issue: #123
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is a indirect recursion which can reach the extent reservation:
btrfs_reserve_extent() <--|
|- do_chunk_alloc() |
|- btrfs_alloc_chunk() |
|- btrfs_insert_item() |
|- btrfs_reserve_extent() <--|
Currently, we're using root->ref_cows to determine whether we should do
chunk prealloc to avoid such loop.
But that's still a hidden trap. Instead of solving it using some hidden
tricks, this patch will make chunk/block group allocation exclusive.
Now if do_chunk_alloc() determines to alloc chunk, it will set a flag in
transaction handle so new call of do_chunk_alloc() will refuse to
allocate new chunk until current chunk allocation finishes.
The chunks get over-allocated by 2M so there's enough space in case the
recursive call asks for a different type of blockgroup.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This ensures that 'btrfs filesystem show' can correctly identify a
filesystem on a newly created local file.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When btrfs' 'filesystem' subcommand is passed path to an image file it
currently fails since the code expects the image file is going to be
recognised by libblkid (called from btrfs_scan_devices()). This is not
the case since libblkid only scan well-known locations under /dev.
Fix this by explicitly calling open_ctree which will correctly open the
image and add it to the correct btrfs_fs_devices struct. This allows
subsequent cmd_filesystem_show logic to correctly show requested
information.
Issue: #169
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In lowmem mode, we check fs roots and free space cache by iterating
each root item and inode item, using btrfs_next_item() and a path
pointing to the root tree.
However in repair mode, check_fs_root() can modify the fs root, thus
CoWs the tree root, and the old path in check_fs
It could lead to strange behavior, e.g. after repairing a fs tree, the
path can point to a fs tree.
Since no ROOT_ITEM exists in fs tree, all remaining trees are skipped in
repair mode.
This bug exists from the early time of lowmem mode repair, and is only
exposed by recent free space inode check code. (Fs tree inodes are
passed to free space inode check, causing false alerts and repair
failure).
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
BTRFS_COMPAT_EXTENT_TREE_V0 is introduced for a short time in kernel,
and it's over 10 years ago.
Nowadays there should be no user for that feature, and kernel has remove
this support in Jun, 2018. There is no need for btrfs-progs to support
it.
This patch will remove EXTENT_TREE_V0 related code and replace those
BUG_ON() to a more graceful error message.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
As the link reported, btrfs fi sh may crash while a device is removing.
valgrind reported:
======================================================================
...
==883== Invalid write of size 8
==883== at 0x13C99A: get_device_info (in /usr/bin/btrfs)
==883== by 0x13D715: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883== Address 0x4d8c7a0 is 0 bytes after a block of size 12,288 alloc'd
==883== at 0x483877F: malloc (vg_replace_malloc.c:299)
==883== by 0x13D861: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883==
==883== Invalid write of size 8
==883== at 0x13C99D: get_device_info (in /usr/bin/btrfs)
==883== by 0x13D715: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883== Address 0x4d8c7a8 is 8 bytes after a block of size 12,288 alloc'd
==883== at 0x483877F: malloc (vg_replace_malloc.c:299)
==883== by 0x13D861: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883==
==883== Syscall param ioctl(generic) points to unaddressable byte(s)
==883== at 0x4CA9CBB: ioctl (in /usr/lib/libc-2.29.so)
==883== by 0x13C9AB: get_device_info (in /usr/bin/btrfs)
==883== by 0x13D715: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883== Address 0x4d8c7a0 is 0 bytes after a block of size 12,288 alloc'd
==883== at 0x483877F: malloc (vg_replace_malloc.c:299)
==883== by 0x13D861: get_fs_info (in /usr/bin/btrfs)
==883== by 0x153B5F: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
==883==
--883-- VALGRIND INTERNAL ERROR: Valgrind received a signal 11 (SIGSEGV) - exiting
--883-- si_code=1; Faulting address: 0x284D8C7B8; sp: 0x1002eb5e50
valgrind: the 'impossible' happened:
Killed by fatal signal
host stacktrace:
==883== at 0x5805261C: get_bszB_as_is (m_mallocfree.c:303)
==883== by 0x5805261C: get_bszB (m_mallocfree.c:315)
==883== by 0x5805261C: vgPlain_arena_malloc (m_mallocfree.c:1799)
==883== by 0x58005AD2: vgMemCheck_new_block (mc_malloc_wrappers.c:372)
==883== by 0x58005AD2: vgMemCheck_malloc (mc_malloc_wrappers.c:407)
==883== by 0x580A7373: do_client_request (scheduler.c:1925)
==883== by 0x580A7373: vgPlain_scheduler (scheduler.c:1488)
==883== by 0x580F57A0: thread_wrapper (syswrap-linux.c:103)
==883== by 0x580F57A0: run_a_thread_NORETURN (syswrap-linux.c:156)
sched status:
running_tid=1
Thread 1: status = VgTs_Runnable (lwpid 883)
==883== at 0x483877F: malloc (vg_replace_malloc.c:299)
==883== by 0x1534AA: ??? (in /usr/bin/btrfs)
==883== by 0x153C49: ??? (in /usr/bin/btrfs)
==883== by 0x11B0C1: main (in /usr/bin/btrfs)
client stack range: [0x1FFEFFA000 0x1FFF000FFF] client SP: 0x1FFEFFDCE0
valgrind stack range: [0x1002DB6000 0x1002EB5FFF] top usage: 7520 of 1048576
======================================================================
The above log says that invalid write to allocated @di_args happened
in get_device_info() called in get_fs_info().
The size of @di_args is allocated according by fi_args->num_devices.
And fi_args->num_devices is *the number of dev_items in chunk_tree*.
However, in the loop to get devices info, btrfs-progs calls ioctl
BTRFS_IOC_DEV_INFO which just finds device in
fs_info->fs_devices->devices.
Let's look at kernel side.
In btrfs_rm_device(), btrfs_rm_dev_item() causes removal of
related dev_items in chunk_tree. *Do something*.
Then delete the device from device->fs_devices.
So the case is:
Userspace kernel
get_fs_info() btrfs_rm_device()
...
btrfs_rm_dev_item()
determine fi_args->num_devices and
fi_args->max_id by seraching chunk_tree.
malloc() ...
Loop(Crashed): call get_device_info() by devid
from 1 to fi_args->max_id.
mutex_lock(&fs_devices->device_list_mutex);
list_del_rcu(&device->dev_list);
...
In the loop of get_device_info(), get_device_info() still can get info
of the removing device since it's still in fs_info->fs_devices->devices.
Then the iterator value @ndev increaments causes invalid access out of
bounds.
Solved it by adding the check of @ndev while looping.
Reported-by: Peter Hjalmarsson <kanelxake@gmail.com>
Link: https://bugzilla.redhat.com/show_bug.cgi?id=1711787
Signed-off-by: Su Yue <Damenly_Su@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Although moderm hardware is fast enough and crc32c calculation is not a
hotspot, doing such optimization won't hurt anyway.
Issue: #175
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is a bug report of BUG_ON() which is caused by __free_extent()
failed to lookup a backref extent:
Failed to find [1429288337408, 168, 16384]
btrfs unable to find ref byte nr 1429288583168 parent 0 root 2 owner 0 offset 0
convert/source-ext2.c:834: ext2_copy_inodes: BUG_ON ret triggered, value -5
./btrfs-convert[0x410941]
./btrfs-convert(main+0x1fdc)[0x40d3b8]
/lib64/libc.so.6(__libc_start_main+0xf3)[0x7f93bb7d2f33]
./btrfs-convert(_start+0x2e)[0x40a96e]
It's still unclear how this bug can be triggered, but adding such debug
output will provide more info for us to debug.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test misc-tests/035-receive-common-mount-point-prefix does another
mount inside TEST_MNT but current 'make test-clean' will not properly
undo the nested mount and this will break subsequent tests. The
recursive unmount can handle that.
Signed-off-by: David Sterba <dsterba@suse.com>
In convert we use trans->block_reserved >= 4096 as a threshold to commit
transaction, where block_reserved is the number of new tree blocks
allocated inside a transaction.
The problem is, we still have a hidden bug in delayed ref implementation
in btrfs-progs, when we have a large enough transaction, delayed ref may
failed to find certain tree blocks in extent tree and cause transaction
abort.
This fix will workaround it by committing transaction at a much lower
threshold.
The old 4096 means 4096 new tree blocks, when using default (16K)
nodesize, it's 64M, which can contain over 12k inlined data extent or
csum for around 60G, or over 800K file extents.
The new threshold will limit the size of new tree blocks to 2M, aligning
with the chunk preallocator threshold, and reducing the possibility to
hit that delayed ref bug.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
libbtrfs.so already has user's LDFLAGS applied. The change also applies
those to libbtrfsutil.so. A separate variable is used for that though it
currently only copies LDFLAGS. This is to make it obvious that
libbtrfsutils is a standalone library.
Reported-by: Michał Górny
Bug: https://bugs.gentoo.org/686284
Pull-request: #172
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
Adds Make variables EXTRA_PYTHON_CFLAGS and EXTRA_PYTHON_LDFLAGS which
can be used to pass CFLAGS and LDFLAGS respectively when building the
Python library.
This is required to support reproducible builds, as there are often
compiler and linker flags that must be passed in order to generate
reproducible output (e.g. -fdebug-prefix-map)
Pull-request: #176
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is one report of compressed extent happens in btrfs, but has no
csum and then leads to possible decompress error screwing up kernel
memory.
Although it's a kernel bug, and won't cause problem until compressed
data get corrupted, let's catch such problem in advance.
This patch will catch any unexpected compressed extent with:
1) 0 or less than expected csum
2) nodatasum flag set in the inode item
This is for original mode.
Reported-by: James Harvey <jamespharvey20@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is one report of compressed extent happens in btrfs, but has no
csum and then leads to possible decompress error screwing up kernel
memory.
Although it's a kernel bug, and won't cause problem until compressed
data get corrupted, let's catch such problem in advance.
This patch will catch any unexpected compressed extent with:
1) missing csum
2) nodatasum flag set in the inode item
This is for lowmem mode.
Reported-by: James Harvey <jamespharvey20@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The command
$ printf 'btrfs-stream\0\0\0\0\0' | btrfs receive --dump
can loop as the stream is not valid, but the maximum error limit is not
set properly for --dump. The command line parameter -E applies here too,
so it's still possible to dump partially damanged stream.
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=200085
Author: Alexander Kovtunenko <akovtunenko@slice.com>
Signed-off-by: David Sterba <dsterba@suse.com>
blkid_get_cache() returns error code which is -errno. So we can use them
directly.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Currently, when operating in a more verbose mode (-vv), the receive
command does not mention any write or clone commands, unlike other
commands.
This change adds debug messages for the write and clone operations, that
do not include data but only offsets and lengths, as this is actually
very useful to debug a send stream and I use it frequently.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When repairing a file system created by a very old kernel, I ran into
issues fixing up the extent flags since fixup_extent_flags assumed
that a METADATA_ITEM would be present if the record was for metadata.
Since METADATA_ITEMs don't exist without skinny metadata, we need to
fall back to EXTENT_ITEMs. This also falls back to EXTENT_ITEMs even
with skinny metadata enabled as other parts of the tools do.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
When running fuzz-tests/003 and fuzz-tests/009, btrfs-progs will crash
due to BUG_ON().
[CAUSE]
We abused BUG_ON() in btrfs_commit_transaction(), which is one of the
most error prone function for fuzzed images.
Currently to cleanup the aborted transaction, we only need to clean up
the only per-transaction data: delayed refs.
This patch will introduce a new function, btrfs_destroy_delayed_refs()
to cleanup delayed refs when we failed to commit transaction.
With that function, we will gently destroy per-trans delayed ref, and
remove the BUG_ON()s in btrfs_commit_transaction().
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Mahoney