Currently we are doing a pretty slow search for system chunks before
restoring real data.
The current behavior is to search all clusters for chunk tree root
first, then search all clusters again and again for every chunk tree
block.
This causes recursive calls and pretty slow start up, the only good news
is since chunk tree are normally small, we don't need to iterate too
many times, thus overall it's acceptable.
To address such bad behavior, we could take usage of system chunk array
in the super block.
By recording all system chunks ranges, we could easily determine if an
extent belongs to chunk tree, thus do one loop simple linear search for
chunk tree leaves.
This patch only introduces the code base for later patches.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
There is no need to allocate 2 * max_pending_size (which can be 256M) if
we're just extracting super block.
We only need to prepare BTRFS_SUPER_INFO_SIZE as buffer size.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We can easily get confusing error message like:
ERROR: restore failed: Success
This is caused by wrong "%m" usage, as we normally use ret to indicate
error, without populating errno.
This patch will fix it by output the return value directly as normally
we have extra error message to show more meaning message than the return
value.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The removed paragraph in btrfs-man5.asciidoc says the same as the
previous one.
Signed-off-by: Merlin Büge <merlin.buege@tuhh.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Add definition, crypto wrappers and support to mkfs for blake2 for
checksumming. There are 2 aliases either blake2 or blake2b.
Signed-off-by: David Sterba <dsterba@suse.com>
Upstream commit 997fa5ba1e14b52c554fb03ce39e579e6f27b90c,
git repository: git://github.com/BLAKE2/BLAKE2
The reference implemetation added in this patch is unchanged and will be
modified only to compile in current code base and with minimal other
modifications in case of future sync with upstream code. IOW, the coding
style should stay as-is and does not conform to the other btrfs-progs
code. This is an exception for xxhash and sha256 code as well.
Signed-off-by: David Sterba <dsterba@suse.com>
Add the definition to the checksum types and let mkfs accept it.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
A simple tool to microbenchmark performance of the hashes. Uses rdtsc
for timing, so works only on x86_64.
$ make hash-speedtest
$ ./hash-speedtest [iterations]
Block size: 4096
Iterations: 100000
NULL-NOP: cycles: 56061823, c/i 560
NULL-MEMCPY: cycles: 61296469, c/i 612
CRC32C: cycles: 179961796, c/i 1799
XXHASH: cycles: 138434590, c/i 1384
Signed-off-by: David Sterba <dsterba@suse.com>
The table won't change at runtime and the string name can be in a buffer
avoiding the pointer indirection. Make one entry aligned to 16 bytes,
plenty of space to store reasonably long csum names.
Signed-off-by: David Sterba <dsterba@suse.com>
The SHA256 is going to be used in the future, so this makes it a second
user and we also have the appropriate directory now.
Signed-off-by: David Sterba <dsterba@suse.com>
With the introduction of xxhash64 to btrfs-progs we created a crypto/
directory for all the hashes used in btrfs (although no
cryptographically secure hash is there yet).
Move the crc32c implementation from kernel-lib/ to crypto/ as well so we
have all hashes consolidated.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Copy of xxhash.[ch] from git://github.com/Cyan4973/xxHash, version
v0.7.1. The include xxh3.h has been commented out as we don't have it
here, otherwise the copy is unchnaged.
Signed-off-by: David Sterba <dsterba@suse.com>
The libbtrfs-test simulated build happens outside of the source
repository, but sometimes the system library is used instead of the repo
one. When -rpath does not work, force the correct library by LD_PRELOAD.
Signed-off-by: David Sterba <dsterba@suse.com>
Several people reported build breakage of snapper, due to missing
symbols in libbtrfs.so. Move the objects to the library objects, now we
don't have to worry about the new exports as the libbtrfs.sym is
unchanged. And there are no new .h files being exported though there are
the .o files in the library.
Issue: #214
Link: https://github.com/openSUSE/snapper/issues/500
Signed-off-by: David Sterba <dsterba@suse.com>
The shared library exports many functions that are not supposed to be
public, like rb-tree, crc32c or internal helpers but as this has been
potentially in use we should at least make a list. There's only a
subset being used by the snapper project.
Export majority of current symbols visible in libbtrfs so any future
additions to libbtrfs objects are automatically hidden and don't pollute
the namespace further.
Note that all projects should switch to libbtrfsutil rather than
libbtrfs that exists for historical reasons and will be deprecated in
the future.
Signed-off-by: David Sterba <dsterba@suse.com>
Make use of GitLab-CI nested virutal environment to start QEMU instance
inside containers and perform btrfs-progs build, execute unit test cases
and save the logs.
This allows to run the progs testsuite on a recent kernel, newer than
what CI instances usually provide. As this is is emulated, the runtime
is longer.
Issue: #171
Signed-off-by: Lakshmipathi.G <lakshmipathi.ganapathi@collabora.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Fix compiler warnings and errors in btrfs-sb-mod due to incorrect
conversion with the checksum updates.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
If we generate testsuite tarball by 'make testsuite', there is no
clean-test.sh in it. But some tests need cleanup if a testcase failed.
Signed-off-by: An Long <lan@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Discovered with cppcheck. Fix signed/unsigned int mismatches, sizeof and
long formats.
Pull-request: #197
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
A user reports that some symbols are missing from libbtrfs, eg.
radix_tree_init. This is correct and there are few more. The headers
exported through the library all need the respective object files.
The sources are GPL so is libbtrfs, which is known
Issue: #205
Signed-off-by: David Sterba <dsterba@suse.com>
As of 5.1, btrfs now supports compression levels for zstd. Let users
know about this in the man page.
Pull-request: #204
Signed-off-by: Dennis Zhou <dennis@kernel.org>
Signed-off-by: David Sterba <dsterba@suse.com>
uClibc does not provide backtrace() nor <execinfo.h>. When building
btrfs-progs, passing --disable-backtrace is enough to make it build with
uClibc. But once btrfs-progs is installed and another program/library
includes kerncompat.h, it fails to build with uClibc, because
BTRFS_DISABLE_BACKTRACE is not defined.
The most correct fix for this would be to have kerncompat.h generated
from kerncompat.h.in during the btrfs-progs build process, and tuned
depending on autoconf/automake variables. But as a quick fix that
follows the current strategy, we simply tweak the existing __GLIBC__
conditional. Indeed, uClibc pretends to be glibc and defines __GLIBC__,
but it does not replace it completely, hence the need to define
BTRFS_DISABLE_BACKTRACE when __GLIBC__ is not defined *or* when
__UCLIBC__ is defined.
Pull-request: #206
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Retrieved from: https://git.buildroot.net/buildroot/tree/package/btrfs-progs/0002-kerncompat.h-define-BTRFS_DISABLE_BACKTRACE-when-bui.patch]
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
As mkfs will grow new checksums, print the used checksum in it's
versbose output.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Adding this table will make extending btrfs-progs with new checksum types
easier.
Also add accessor functions to access the table fields.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a helper to check if we have a valid csum type from the super block.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Add an option to mkfs to specify which checksum algorithm will be used
for the filesystem. Currently only crc32c is supported.
The option name is -c, presumably one of the comonly used options so it
gets the lowercase option.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Update the checksumming API to be able to cope with more checksum types
than just CRC32C. The finalization call is merged into btrfs_csum_data.
There are some fixme's and asserts added that need to be resolved.
Co-developed-by: David Sterba <dsterba@suse.com>
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
update_block_csum() in btrfs-sb-mod.c is always called with the 'is_sb'
argument set to 1.
Get rid of the special case for is_sb == 0.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In preparation to supporting new checksum algorithm pass the checksum type
to btrfs_csum_data/btrfs_csum_final, this allows us to encapsulate any
differences in processing into the respective functions
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Pass pointer to a generic buffer instead of fixed size that crc32c
currently uses.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Add the checksum type to csum_tree_block_size(), __csum_tree_block_size()
and verify_tree_block_csum_silent().
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Cache the super-block's checksum type field in 'struct recover_control'.
This will be needed for further refactoring the checksum handling.
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
The callers of csum_tree_block_size() blindly assume we're only having
crc32c as a possible checksum and thus pass in
btrfs_csum_sizes[BTRFS_CSUM_TYPE_CRC32] for the size argument of
csum_tree_block_size().
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Add checksum type to the definition structure for a new filesystem, this
will be used in following patches.
Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
Switch backend to docbook5 for asciidoctor and disable validation as v5
does not use DTBs.
Pull-request: #203
Issue: #201
Signed-off-by: Dimitri John Ledkov <xnox@ubuntu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
As misc-tests/021 image dump is restored on the same original loop
device, this overlaps with the stale data and makes the test pass
falsely.
Fix this by using a new device for restore.
And also, the btrfs-image dump and restore doesn't restore the file
data, so any read on the files should be avoided. So instead of file
data use file stat data for the checksum.
Reported-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since commit 04be0e4b19 ("btrfs-progs: corrupt-block: Correctly
handle -r when passing -I") the 'r' switch is used with both -I and -d
options. So remove the wrong clarificatoin that -r is used only with -d
option.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Use the return value of listxattr instead of tokenizing.
The end of the extended attribute list is indicated by the return value,
not an empty list item (two consecutive NULs). Using strtok in this way
thus sometimes caused add_xattr_item to reuse stack data in xattr_list
from the previous invocation, thus querying attributes that are not
actually in the file's xattr list.
Issue: #194
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Vladimir Panteleev <git@vladimir.panteleev.md>
Signed-off-by: David Sterba <dsterba@suse.com>
Qu Wenruo