This allows us to change the use-visible UUID on filesytems from
userspace if desired, by copying the existing UUID to the new location
for metadata comparisons. If this is done, an incompat flag must be
set to prevent older filesystems from mounting the filesystem, but
the original UUID can be restored, and the incompat flag removed.
This introduces the new -m|-M UUID options similar to current
-u|-U UUID ones with the difference that we don't rewrite the fsid but
just copy the old uuid and set a new one. Additionally running with
[-M old-uuid] clears the incompat flag and retains only fsid on-disk.
Additionally it's not allowed to intermix -m/-u/-U/-M options in a
single invocation of btrfstune, nor is it allowed to change the uuid
while there is a uuid rewrite in-progress. Also changing the uuid of a
seed device is not currently allowed (can change in the future).
Example:
btrfstune -m /dev/loop1
btrfs inspect-internal dump-super /dev/loop1
superblock: bytenr=65536, device=/dev/loop1
---------------------------------------------------------
csum_type 0 (crc32c)
csum_size 4
csum 0x4b7ea749 [match]
<ommitted for brevity>
fsid 0efc41d3-4451-49f3-8108-7b8bdbcf5ae8
metadata_uuid 352715e7-62cf-4ae0-92ee-85a574adc318
<ommitted for brevity>
incompat_flags 0x541
( MIXED_BACKREF |
EXTENDED_IREF |
SKINNY_METADATA |
METADATA_UUID )
<omitted for brevity>
dev_item.uuid 0610deee-dfc3-498b-9449-a06533cdec98
dev_item.fsid 352715e7-62cf-4ae0-92ee-85a574adc318 [match]
<ommitted for brevity>
mount /dev/loop1 btrfs-mnt/
btrfs fi show btrfs-mnt/
Label: none uuid: 0efc41d3-4451-49f3-8108-7b8bdbcf5ae8
Total devices 1 FS bytes used 128.00KiB
devid 1 size 5.00GiB used 536.00MiB path /dev/loop1
In this case a new btrfs filesystem was created and the original uuid
was 352715e7-62cf-4ae0-92ee-85a574adc318, then btrfstune was run which
copied that value over to metadata_uuid field and set the current fsid
to 0efc41d3-4451-49f3-8108-7b8bdbcf5ae8. And as far as userspace is
concerned this is the fsid of the fs.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add support for a new metadata_uuid field. This is just a preparatory
commit which switches all users of the fsid field for metdata comparison
purposes to utilize the new field. This more or less mirrors the
kernel patch, additionally:
* Update 'btrfs inspect-internal dump-super' to account for the new
field. This involes introducing the 'metadata_uuid' line to the
output and updating the logic for comparing the fs uuid to the
dev_item uuid.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Test case misc/021 is testing if we could mount a single disk btrfs
image recovered from multi disk fs.
The problem is, current kernel has extra check for block group, chunk
and dev extent. This means no image can pass btrfs check for chunk
tree and the filesystem will not mount.
So do extra btrfs check before mount, this will also help us to locate
the problem in btrfs-image easier.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We have btrfs_alloc_dev_extent() accepting @convert flag to toggle
special handling for convert.
However the @convert flag only determines whether we call
find_free_dev_extent(), and we may later need to insert dev extents
without searching dev tree.
So refactor btrfs_alloc_dev_extent() into 2 functions,
- btrfs_alloc_dev_extent(), which will try to find free dev extent, and
- btrfs_insert_dev_extent(), which will just inserts a dev extent
For implementation, btrfs_alloc_dev_extent() will call
btrfs_insert_dev_extent() anyway, so there's no duplicated code.
This removes the need of @convert parameter, and make
btrfs_insert_dev_extent() public for later usage.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since btrfs-image is just restoring tree blocks without really checking
if that tree block contents makes sense, for multi-device image, block
group items will keep the incorrect block group flags.
For example, for a metadata RAID1 data RAID0 btrfs recovered to a single
disk, its chunk tree will look like:
item 1 key (FIRST_CHUNK_TREE CHUNK_ITEM 22020096)
length 8388608 owner 2 stripe_len 65536 type SYSTEM
item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 30408704)
length 1073741824 owner 2 stripe_len 65536 type METADATA
item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 1104150528)
length 1073741824 owner 2 stripe_len 65536 type DATA
All chunks have correct type (SINGLE).
While its block group items will look like:
item 1 key (22020096 BLOCK_GROUP_ITEM 8388608)
block group used 16384 chunk_objectid 256 flags SYSTEM|RAID1
item 3 key (30408704 BLOCK_GROUP_ITEM 1073741824)
block group used 114688 chunk_objectid 256 flags METADATA|RAID1
item 11 key (1104150528 BLOCK_GROUP_ITEM 1073741824)
block group used 1572864 chunk_objectid 256 flags DATA|RAID0
All block group items still have the wrong profiles.
And btrfs check (lowmem mode for better output) will report error for
such image:
ERROR: chunk[22020096 30408704) related block group item flags mismatch, wanted: 2, have: 18
ERROR: chunk[30408704 1104150528) related block group item flags mismatch, wanted: 4, have: 20
ERROR: chunk[1104150528 2177892352) related block group item flags mismatch, wanted: 1, have: 9
This patch will do an extra repair for block group items to fix the
profile of them.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Current fixup_devices() will only remove DEV_ITEMs and reset DEV_ITEM
size. Later we need to do more fixup works, so change the name to
fixup_chunks_and_devices() and refactor the original device size fixup
to fixup_device_size().
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Move "\n" at end of the sentence to print.
Fixes: 281eec7a9d ("btrfs-progs: check: repair inode nbytes in lowmem mode")
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For reloc tree, despite of its short lifespan, it's still the backref,
where reloc tree root backref points back to itself, makes it special.
So it's more approriate to put them into 020-extent-ref-cases.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The header is exported as /usr/include/btrfs-list.h but as it's only a
delcaration, it's not used for anything, so the change is safe.
Generated by https://github.com/jsoref/spelling
Issue: #154
Author: Josh Soref <jsoref@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This header is exported to /usr/include/btrfs but there are no known
users, so the change should be safe.
Generated by https://github.com/jsoref/spelling
Issue: #154
Author: Josh Soref <jsoref@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Besides the comments, there's a slight change as the file config.log
will be deleted by the 'clean-gen' rule.
Generated by https://github.com/jsoref/spelling
Issue: #154
Author: Josh Soref <jsoref@users.noreply.github.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There's a newer version of zstd, without any obvious changes that would
affect our build testing coverage.
Signed-off-by: David Sterba <dsterba@suse.com>
btrfsutil.h and the Python docstrings are thorough, but I've gotten a
couple of requests for a high-level overview of the available interfaces
and example usages. Add them to README.md.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
With the previous few fixes and features, we should bump the minor
version.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We can use the new BTRFS_IOC_GET_SUBVOL_ROOTREF and
BTRFS_IOC_INO_LOOKUP_USER ioctls to allow non-root users to list
subvolumes.
This is based on a patch from Misono Tomohiro but takes a different
approach (mainly, this approach is more similar to the existing tree
search approach).
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Attempt to use the BTRFS_IOC_GET_SUBVOL_INFO ioctl (added in kernel
4.18) for subvolume_info() if not root. Also, rename
get_subvolume_info_root() -> get_subvolume_info_privileged() for
consistency with further changes.
This is based on a patch from Misono Tomohiro.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Some upcoming tests will need to create a second Btrfs filesystem, so
add support for this to the test helpers.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
These will be used for testing some upcoming changes which allow
unprivileged operations.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We're leaking file descriptors, which makes it impossible to clean up
the temporary mount point created by the test.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This has been supported since day one, but it wasn't documented.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
async became a keyword in Python 3.7, so, e.g., create_subvolume('foo',
async=True) is now a syntax error. Fix it with the Python convention of
adding a trailing underscore to the keyword (async -> async_). This is
what several other Python libraries did to handle this.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Right now, we're defaulting to top=5 (i.e, all subvolumes). The
documented default is top=0 (i.e, only beneath the given path). This is
the expected behavior. Fix it and make the test cases cover it.
Reported-by: Jonathan Lemon <bsd@fb.com>
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Reported on IRC that the inode number limit appears to be 264, while the
actual value is 2^64. Fix that for the manual page backend by redefining
the format.
Signed-off-by: David Sterba <dsterba@suse.com>
The 'swap' is too generic and clashes with some userspace tools that
compile against btrfs, eg. snapper when including
boost::smart_ptr::scoped_array after kerncompat.h:
/usr/include/boost/smart_ptr/scoped_array.hpp:127:13: error: macro "swap" requires 2 arguments, but only 1 given
a.swap(b);
Signed-off-by: David Sterba <dsterba@suse.com>
Found using -Wmissing-prototypes in GCC. This should improve LTO
behavior.
Note that set_free_space_tree_thresholds is an unused function. Adding
inline seems to remove the unused function warning.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
pthread_t is an opaque type, not an integer one. The > 0 makes no sense
and compiler throws a warning.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Replaced bswap with _ variants bswap_32 etc. While it's a glibc
extension, all of the popular libc implementations (glibc, uClibc, musl,
BIONIC) seem to support it.
Added static inline to two functions to match little endian variants. This
fixes a linking error experienced when compiling on gcc 7.3.0 with LTO,
possibly a bug that was fixed later.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a new section, QUOTA RESCAN, to explain why and when we need a full
quota rescan when assigning/removing qgroup relationship.
Also, since 'remove' shares the same options of 'assign', add reference
to 'assign' options for 'remove' subcommand.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update ]
Signed-off-by: David Sterba <dsterba@suse.com>
Even though we have --rescan option, it still needs the user to manually
specify it or call quota rescan command.
This does not make much sense for the most common usecase and only
confuses users. When the rescan is not desired, the --no-rescan option
can be used, eg. when a mass qgruop assignment happens and the repeated
rescan would slow things down. This is considered a marginal usecase.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Rename the function so its name falls in line with the rest of btrfs
nomenclature. So when we change the uuid of the header it's in fact of
the buffer header. Also remove the root argument since it's used solely
to take a reference to fs_info which can be done via the mandatory eb
argument. No functional changes.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This function already takes an extent buffer that contains a reference
to the fs_info. Use that and reduce argument count. No functional
changes.
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Originally print-tree uses depth first search to print trees.
This works fine until we reach 3 level trees (root node level is 2).
For tall trees whose root level is 2 or higher, DFS will mix nodes and
leaves like the following example:
Level 2 A
/ \
Level 1 B C
/ | | \
Level 0 D E F G
DFS will cause the following output sequence:
A B D E C F G
Which in term of node/leave is:
N N L L N L L
Such mixed node/leave result is sometimes hard for human to read.
This patch will introduce 2 new options, --bfs and --dfs.
--dfs: keeps the original output sequence, and to keep things
compatible it's the default behavior.
--bfs: uses breadth-first search, and will cause better output
sequence like:
A B C D E F G
Which in term of node/leave is:
N N N L L L L
Much better sorted and may become default in the future.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nikolay Borisov