ETA is calculated in a wrong way. It should be just current time in
seconds + sec_left, independently if the job was resumed or not.
Pull-request: #190
Signed-off-by: Grzegorz Kowal <grzegorz@amuncode.org>
Signed-off-by: David Sterba <dsterba@suse.com>
When creating a filesystem with mixed block groups, we are creating two
space info objects to track used/reserved/pinned space, one only for data
and another one only for metadata.
This is making fstests test case generic/416 fail, with btrfs' check
reporting over an hundred errors about bad extents:
(...)
bad extent [17186816, 17190912), type mismatch with chunk
bad extent [17195008, 17199104), type mismatch with chunk
bad extent [17203200, 17207296), type mismatch with chunk
(...)
Because, surprisingly, this results in block groups that do not have the
BTRFS_BLOCK_GROUP_DATA flag set but have data extents allocated in them.
This is a regression introduced in btrfs-progs v5.2.
So fix this by making sure we only create one space info object, for both
metadata and data, when mixed block groups are enabled.
Fixes: c31edf610c ("btrfs-progs: Fix false ENOSPC alert by tracking used space correctly")
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When running misc-test/034, we got unexpected log output:
[TEST/misc] 033-filename-length-limit
[TEST/misc] 034-metadata-uuid
Checking btrfstune logic
Checking dump-super output
Checking output after fsid change
Checking for incompat textual representation
Checking setting fsid back to original
Testing btrfs-image restore
This is caused by commit 2570cff076 ("btrfs-progs: test: cleanup misc-tests/034")
which uses _log facility which also populates stdout.
Just change _log() to echo "$*" >> "$RESULTS" to fix it.
Unlike the initial commit, there is no other user of _log, so it
shouldn't affect other tests.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In process_clone(), we're not checking the return value of strdup().
But, there's no reason to strdup() in the first place: we just pass the
path into path_cat_out(). Get rid of the strdup().
Fixes: f1c24cd80d ("Btrfs-progs: add btrfs send/receive commands")
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
"btrfsck -Q" segfaults because it does not call qgroup_set_item_count_ptr()
properly:
# btrfsck -Q /dev/sdk
Opening filesystem to check...
Checking filesystem on /dev/sdk
UUID: 34a35bbc-43f8-40f0-8043-65ed33f2e6c3
Print quota groups for /dev/sdk
UUID: 34a35bbc-43f8-40f0-8043-65ed33f2e6c3
Segmentation fault (core dumped)
Since "struct task_ctx ctx" is global, we can just move
qgroup_set_item_count_ptr() much earlier stage in the check process to
avoid to forget initializing it.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The old parameters, @ref_generation and @owner_objectid, are pretty
confusing when using auto-completion.
Unify the parameters as a quick fix.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
- add first line of the long description
- list the type values in all commands (set, get, list)
- enhance
- split option description
Signed-off-by: David Sterba <dsterba@suse.com>
While testing snapshot deletion with dm-log-writes I saw that I was
failing the fsck sometimes when the fs was actually in the correct
state. This is because we only skip blocks on the same level of
root_item->drop_level. If the drop_level < the root level then we could
very well walk into nodes that we wouldn't actually walk into on fs
mount, because the drop_progress is further ahead in the slot of the
root. Instead only process the slots of the nodes that are above the
drop_progress key. With this patch in place we no longer improperly
fail to check fs'es that have a drop_progress set with a drop_level <
root level.
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
[BUG]
Btrfs-progs sometimes fails to find certain extent backref when
committing transaction.
The most reliable way to reproduce it is fsck-test/013 on 64K page sized
system:
[...]
adding new data backref on 315859712 root 287 owner 292 offset 0 found 1
btrfs unable to find ref byte nr 31850496 parent 0 root 2 owner 0 offset 0
Failed to find [30867456, 168, 65536]
Also there are some github bug reports related to this problem.
[CAUSE]
Commit 909357e867 ("btrfs-progs: Wire up delayed refs") introduced
delayed refs in btrfs-progs.
However in that commit, delayed refs are not run at correct timing.
That commit calls btrfs_run_delayed_refs() before
btrfs_write_dirty_block_groups(), which needs to update
BLOCK_GROUP_ITEMs in extent tree, thus could cause new delayed refs.
This means each time we commit a transaction, we may screw up the extent
tree by dropping some pending delayed refs, like:
Transaction 711:
btrfs_commit_transaction()
|- btrfs_run_delayed_refs()
| Now all delayed refs are written to extent tree
|
|- btrfs_write_dirty_block_groups()
| Needs to update extent tree root
| ADD_DELAYED_REF to 315859712.
| Delayed refs are attached to current trans handle.
|
|- __commit_transaction()
|- write_ctree_super()
|- btrfs_finish_extent_commit()
|- kfree(trans)
Now delayed ref for 315859712 are lost
Transaction 712:
Tree block 315859712 get dropped
btrfs_commit_transaction()
|- btrfs_run_delayed_refs()
|- run_one_delayed_ref()
|- __free_extent()
As previous ADD_DELAYED_REF to 315859712 is lost, extent tree
doesn't have any backref for 315859712, causing the bug
In fact, commit c31edf610c ("btrfs-progs: Fix false ENOSPC alert by
tracking used space correctly") detects the tree block leakage, but in
the reproducer we have too much noise, thus nobody notices the leakage
warning.
[FIX]
We can't just move btrfs_run_delayed_refs() after
btrfs_write_dirty_block_groups(), as during btrfs_run_delayed_refs(), we
can re-dirty block groups.
Thus we need to exhaust both delayed refs and dirty blocks.
This patch will call btrfs_write_dirty_block_groups() and
btrfs_run_delayed_refs() in a loop until both delayed refs and dirty
blocks are exhausted. Much like what we do in commit_cowonly_roots() in
kernel.
Also, to prevent such problem from happening again (and not to debug
such problem again), add extra check on delayed refs before freeing the
transaction handle.
Reported-by: Klemens Schölhorn <klemens@schoelhorn.eu>
Issue: #187
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
At least in Debian, default build flags include -Werror=format-security,
for good reasons in most cases. Here, the string comes from strftime --
and though I don't suspect any locale would be crazy enough to have %X
include a '%' char, the compiler has no way to know that.
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
The test cli-tests/008-subvolume-get-set-default fails when compiled
with 'D=ubsan', the access to search header items does not follow the
type alignment, so use the accessors.
The error:
subvolume get-default: default id is not 256, but
libbtrfsutil/subvolume.c:361:13: runtime error: member access within
misaligned address 0x7ffc147e4b6f for type 'const struct
btrfs_ioctl_search_header', which requires 8 byte alignment
Note that using the accessors does not fix the ubsan warning, as it
warns on taking the address of a member whose _base_ type is unaligned,
ie. it's the 'sh'.
Fixing that would need to play tricks with pointers to do &sh->type
manually, but to avoid triggering ubsan.
Signed-off-by: David Sterba <dsterba@suse.com>
Add helpers that do proper unaligned access of search heade items. This
is done in the non-libbtrfsutil code already, use the same helpers here
too. We can't use the get_unaligned_* helpers that are defined in
kerncompat, so use plain memcpy that will work everywhere.
Signed-off-by: David Sterba <dsterba@suse.com>
'make clean' followed by 'make test-clean' will fail due to the sanity
check for 'btrfs' binary. We don't need that for cleaning the test, so
turn the error into a warning.
Signed-off-by: David Sterba <dsterba@suse.com>
Add support for changing all integer items in the superblock. Sort them
in the order of defintion in the structure.
Signed-off-by: David Sterba <dsterba@suse.com>
The field type bit width is now only used to describe the members, but
let's add all of the integer ones.
Signed-off-by: David Sterba <dsterba@suse.com>
The CI build prints a lot of warnings
[CC] btrfs.o
In file included from volumes.h:22,
from btrfs.c:22:
kerncompat.h:39: warning: "__always_inline" redefined
#define __always_inline __inline __attribute__ ((__always_inline__))
so define the macro conditionally.
Signed-off-by: David Sterba <dsterba@suse.com>
The shell keyword function is not necessary and not used in many tests,
remove it from the few places that use it right now.
Signed-off-by: David Sterba <dsterba@suse.com>
The comman 'btrfs inspect dump-tree <dev>' will scan all the devices
from the filesystem by defaul.
So as of now you can not inspect each mirrored device independently.
This patch adds option --noscan, which when used won't scan the system
for the partner devices, instead it just uses the devices provided in
the argument.
For example:
btrfs inspect dump-tree --noscan <dev> [<dev>..]
This helps to debug degraded raid1 and raid10.
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Build several standalone tools into one binary and switch the function
by name (symlink or hardlink).
* btrfs
* mkfs.btrfs
* btrfs-image
* btrfs-convert
* btrfstune
The static target is also supported. The name of resulting boxed
binaries is btrfs.box and btrfs.box.static . All the binaries can be
built at the same time without prior configuration.
text data bss dec hex filename
822454 27000 19724 869178 d433a btrfs
927314 28816 20812 976942 ee82e btrfs.box
2067745 58004 44736 2170485 211e75 btrfs.static
2627198 61724 83800 2772722 2a4ef2 btrfs.box.static
File sizes:
857496 btrfs
968536 btrfs.box
2141400 btrfs.static
2704472 btrfs.box.static
Standalone utilities:
512504 btrfs-convert
495960 btrfs-image
471224 btrfstune
491864 mkfs.btrfs
1747720 btrfs-convert.static
1411416 btrfs-image.static
1304256 btrfstune.static
1361696 mkfs.btrfs.static
So the shared 900K binary saves ~2M, or ~5.7M for static build.
Signed-off-by: David Sterba <dsterba@suse.cz>
The documentation lacks clarity about depth to which recursive
'fi du' goes, and was pointed out by a user.
Add test that creates another mount inside a filesystem and verifies
that 'fi du' does not go there.
Issue: #185
Signed-off-by: David Sterba <dsterba@suse.com>
State that defrag by default does not descend to subvolumes, this has
been hidden in implementation details.
Issue: #185
Signed-off-by: David Sterba <dsterba@suse.com>
The documentation lacks clarity about depth to which recursive
defragmentation go, and was pointed out by a user.
The problem here is that the subvolume behaves the same as mount point
regarding path traversal. The nftw stops on mount boundary (FTW_MOUNT).
Add test that verifies this behaviour. Defrag has to be updated to allow
descending to subvolumes (and not mountpoints).
Issue: #185
Signed-off-by: David Sterba <dsterba@suse.com>
fgets consumes n-1 bytes from input buffer.
When a user types y\n, the newline is left in the buffer. As a result,
the next fgets uses that \n as answer without waiting for the user to
type.
This patch also fix a bug that dereference the ret without checking if
it's NULL.
* Consumes the `\n` from stdin buffer
* Avoid NULL pointer dereference: treat EOF as default value
Pull-request: #182
Author: pjw91 <mail6543210@yahoo.com.tw>
Signed-off-by: David Sterba <dsterba@suse.com>
The travis-ci fails at test misc-tests/021-image-multi-devices because
the 'btrfs check' is not run with root permissions, unlike all the other
commands. The check is read-only by default, so that should be safe.
Signed-off-by: David Sterba <dsterba@suse.com>
The transfer lines from dd bloat the logs and other lines may not fit.
Disable xfer in all dd commands but still allow errors to be caught.
Signed-off-by: David Sterba <dsterba@suse.com>
Rename the directory for continuous integration scripts to a more
generic name as we're going to use more than one. The base image on
travis has an old kernel. It's not possible to use a newer one and some
tests fail making the coverage unreliable.
Signed-off-by: David Sterba <dsterba@suse.com>
Gcc version 9.1.1 reports:
In file included from /usr/include/string.h:494,
from kerncompat.h:25,
from ctree.h:26,
from send-utils.c:26:
In function ‘memset’,
inlined from ‘btrfs_read_root_item’ at send-utils.c:165:3,
inlined from ‘subvol_uuid_search2’ at send-utils.c:494:8:
/usr/include/bits/string_fortified.h:71:10: warning: ‘__builtin_memset’ offset [248, 439] from the object at ‘root_item’ is out of the bounds of referenced subobject ‘generation_v2’ with type ‘long long unsigned int’ at offset 239 [-Warray-bounds]
71 | return __builtin___memset_chk (__dest, __ch, __len, __bos0 (__dest));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
That's correct in case the intent is to overwrite just the generation_v2
member, but we want to zero the rest of the root item structure starting
from the generation_v2. No typecasts can obscure that from gcc so the
starting address is calculated as base pointer + member offset.
Signed-off-by: David Sterba <dsterba@suse.com>
Grzegorz Kowal