Add test case which we have NO_HOLES incompat flag while still have
hole file extent.
This can be created by enabling NO_HOLES feature on an existing
filesystem, which lowmem mode would cause false alert for it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ minor adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
Test that we are able to create an image from a multiple devices fs, that
we are able to restore that image into a single device and finally that we
are able to mount it.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
[ added shell quotation and chmod a+w so testsuite on NFS works ]
Signed-off-by: David Sterba <dsterba@suse.com>
Patches "btrfs-progs: tests: correctly receive clones to mounted subvol"
(8eaf63bc9a) and followup are missing last
unmount which leads to failure of misc/020.
Signed-off-by: David Sterba <dsterba@suse.com>
tests/fssum.c:599:13: warning: In the GNU C Library, "major" is defined
by <sys/sysmacros.h>. For historical compatibility, it is
currently defined by <sys/types.h> as well, but we plan to
remove this soon. To use "major", include <sys/sysmacros.h>
directly. If you did not intend to use a system-defined macro
"major", you should undefine it after including <sys/types.h>.
sum_add_u64(&cs, major(st.st_rdev));
Signed-off-by: David Sterba <dsterba@suse.com>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
[ enhance tests to take extra options and use for -e ]
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new image, which contains external SHARED_DATA_REF items to
trigger a lowmem mode false alert.
The image only contains external SHARED_DATA_REF and no inlined data
backref.
Before the image, we only have inlined shared data ref, which is not
enough to trigger lowmem mode false alert.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In order to drop dependency on SSL library to compute MD5 in fssum,
we'll use the reference implementation from RFC 6234.
The checksum is not in a cryptographically sensitive context, but we're
going to skip MD5 and SHA-1 anyway.
Signed-off-by: David Sterba <dsterba@suse.com>
In file included from /usr/include/stdio.h:27:0,
from kerncompat.h:22,
from tests/fssum.c:25:
/usr/include/features.h:148:3: warning: #warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-Wcpp]
# warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
^~~~~~~
We've solved that long time ago and config.h now provides the macros.
Signed-off-by: David Sterba <dsterba@suse.com>
The function htonll is not provided by the standard library and we can
replace it by our cpu-to-XX helpers. This switches the endianity of the
checksummed value to LE, but this is not a problem.
Signed-off-by: David Sterba <dsterba@suse.com>
Copy from fstests, originally from
git://git.kernel.org/pub/scm/linux/kernel/git/arne/far-progs.git
Needs libcrypto to link but this check is now missing in configure.
Signed-off-by: Jan Schmidt <list.btrfs@jan-o-sch.net>
Signed-off-by: David Sterba <dsterba@suse.com>
Nearly each use of IMAGE can be replaced by common helpers as there are
no specific requirements on the testing filesystem. There are still a
few left that need to be evaluated and converted eventually.
Signed-off-by: David Sterba <dsterba@suse.com>
clone needs to resolve the paths of the involved subvolumes in the target
fs from their UUIDs. When doing so it might need to strip the prefix
that is mounted as the root of the fs from those paths.
It didn't do so correctly when processing the source of "clone" commands
This is a regression test for
btrfs-progs: receive: handle root subvol path in clone
Signed-off-by: Benedikt Morbach <benedikt.morbach@googlemail.com>
[ copied the fstests version, will be updated later ]
Signed-off-by: David Sterba <dsterba@suse.com>
Drop the dependency on xfs_io as it's not a standard tool, though it
provides convenience. We use a simple write here so dd can manage.
Signed-off-by: David Sterba <dsterba@suse.com>
If the tests are started from non-root user, the fallocate and xfs_io
fail. Use the root helper as a workaround, we'd should fix the perms
instead.
Signed-off-by: David Sterba <dsterba@suse.com>
Current common.local doesn't handle lowmem mode well.
It passes "--mode=lowmem" alone with "--repair", making it unable to
check lowmem mode.
It's caused by the following bugs:
1) Wrong variable in test/common.local
We should check TEST_ARGS_CHECK, not TEST_CHECK, which is not defined
so we never return 1.
2) Wrong parameter passed to _cmd_spec() in test/common
This prevents us from grepping the correct parameters.
Fix it.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lowmem mode exposed several false alerts, all related to file extents
check.
1) Partly written prealloc extent
Cause lowmem mode to report missing csum or prealloc extent should
not have csum
2) Compressed inline extent
Cause lowmem mode to find mismatch on inline len and item len.
While no error message is output but exit silently.
Reported-by: Chris Murphy <chris@colorremedies.com>
Reported-by: Christoph Anton Mitterer <calestyo@scientia.net>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
fsck-tests/013-extent-tree-rebuild uses "--init-extent-tree", which
implies "--repair".
But the test script doesn't specify "--repair" for lowmem mode test to
detect it.
Add it so lowmem mode test can be happy with it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a minimal image which can reproduce the block group used space
false alert for lowmem mode fsck.
Reported-by: Christoph Anton Mitterer <calestyo@scientia.net>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The test case fsck-tests/015-check-bad-memory-access can't be repair by
btrfs check, and it's a fortunate bug makes original mode to forget the
error code from extent tree, making original mode pass it.
So fuzz-tests is more suitable for it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Simple test script for the following patch.
btrfs-progs: qgroup: add sync option to 'qgroup show'
Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Populate fs after convert so we can trigger data chunk allocation.
This can expose too restrict old rollback condition
Reported-by: Chandan Rajendra <chandan@linux.vnet.ibm.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The travis CI does not have losetup with --partscan, skip the check in
that case so we can still run the mkfs testsuite.
Signed-off-by: David Sterba <dsterba@suse.com>
Add convenient support for extending command arguments, now implemented
for 'btrfs check' to cover the low-memory mode. If defined, arguments
are inserted to any 'btrfs check' command in tests. Exceptions could be
defined in common.local.
Signed-off-by: David Sterba <dsterba@suse.com>
The list of rwx permissions is now hardcoded but used to begenerated and
the invalid numbers filtered out. Not necessary anymore.
Signed-off-by: David Sterba <dsterba@suse.com>
In my test environment, following error was occurred because the size
of /lib/modules/`uname -r`/* is larger than 1GB.
# make test-fsck
[TEST] fsck-tests.sh
[TEST/fsck] 013-extent-tree-rebuild
failed: cp -aR /lib/modules/4.9.0-rc5/ /test/btrfs-progs/tests/mnt
test failed for case 013-extent-tree-rebuild
Makefile:272: recipe for target 'test-fsck' failed
make: *** [test-fsck] Error 1
#
In this test case, 'generate_dataset small' is enough for making the
test files, so I will use 'generate_dataset' instead of 'cp'.
For this, move 'generate_dataset()' from 'common.convert' to 'common'.
Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add new keyword to dump the log file after any test fails. Can be useful
for remote analysis of test failures.
Signed-off-by: David Sterba <dsterba@suse.com>
The option --list might not be available on older versions, the
equvalent is --all. Discovered via failed travis build.
Signed-off-by: David Sterba <dsterba@suse.com>
The current user might not be able to peek into the loop files directly,
use the helper. Discovered by running tests in travis.
Signed-off-by: David Sterba <dsterba@suse.com>
Fixing the problems by one does not scale now. Add more images despite
the fuzz tests will fail. They have been for some time already.
Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Separate checksum test from convert_test_post_check and use it to fix
the broken test 005 as reported.
References: https://bugzilla.kernel.org/show_bug.cgi?id=177141
Signed-off-by: David Sterba <dsterba@suse.com>
Sending stream size of clone-src(-c) option is checked.
Fixed by "btrfs-progs: send: fix handling of -c option".
Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add test case to check the basic function of --clear-space-cache.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For image with tree reloc tree, if its height is over level 2, the root
node's backref will point to itself. It's valid for kernel, but quota
verify code can't handle it and cause a infinite call, overflowing the
stack.
Add minimal image to reproduce the bug, as regression test.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ minor cleanups in test.sh ]
Signed-off-by: David Sterba <dsterba@suse.com>
Fixed by commit 7c646c538e btrfs-progs: qgroup: Fix regression leads
to corrupted qgroup status.
Add minimal test image for that fix.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ minor cleanups in test.sh ]
Signed-off-by: David Sterba <dsterba@suse.com>
Add a new test case to check if btrfs-convert copies common inode flags
like append(a), immutable(i).
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ minor fixes ]
Signed-off-by: David Sterba <dsterba@suse.com>
Some fuzzed images cause various tools to crash but the mayfail helper
would not recognize that. We don't mind if the utility failes but it
must not crash.
Signed-off-by: David Sterba <dsterba@suse.com>
Add test case image for unaligned tree block ptr.
It should lead to BUG_ON in free_extent_buffer().
Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ added bko-NNN- prefix to the files ]
Signed-off-by: David Sterba <dsterba@suse.com>
Reported by Lukas and the same image from him.
DATA_RELOC tree's key type is modifed to CHUNK_ITEM, causing btrfsck
interpret it as CHUNK_ITEM and cause 0 num_stripes.
Add the image to fuzz-test.
Reported-by: Lukas Lueg <lukas.lueg@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ added bko-NNN- prefix to the files ]
Signed-off-by: David Sterba <dsterba@suse.com>
Old convert codes uses both 0400 permission and INODE_READONLY flag to
make the converted ext2 image readonly.
While new convert treat the inode just as normal inode, with no special
inode flag and uses 0600 permission.
This makes user able to modify converted image unintentionally and make
rollback fails.
This test case will test the regression.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
If btrfs isn't in the path, this test will fail with:
[TEST/misc] 006-image-on-missing-device
failed: btrfs fi show /dev/loop0
test failed for case 006-image-on-missing-device
Makefile:226: recipe for target 'test-misc' failed
make: *** [test-misc] Error 1
Fix the test script by adding $TOP to the path.
Signed-off-by: Luis Henriques <henrix@camandro.org>
[ updated to full command names ]
Signed-off-by: David Sterba <dsterba@suse.com>
For ext* fs containing a large hole(larger than 128M), btrfs-convert
will only insert one 128M hole extent and skip the remaining.
This leads to discontinuous file extents.
Add test case for it, and since it's a pinpoint regression test case, no
combination of convert options nor checksum verification.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The usage of 'source' is a bashism, and '.' should be used instead. This
is causing fuzz-tests/001-simple-unmounted to fail in systems where
/bin/sh isn't bash:
[TEST/fuzz] 001-simple-unmounted
./test.sh: 5: ./test.sh: source: not found
./test.sh: 7: ./test.sh: setup_root_helper: not found
./test.sh: 8: ./test.sh: check_prereq: not found
./test.sh: 18: ./test.sh: check_all_images: not found
Since most (all?) tests actually use /bin/bash, change this test to use
bash too.
Signed-off-by: Luis Henriques <henrix@camandro.org>
Signed-off-by: David Sterba <dsterba@suse.com>
The extN filesystem type was lost when the separate tests were created
and we've been testing only ext2. The tests pass for ext3 and ext4
though.
Signed-off-by: David Sterba <dsterba@suse.com>
Add a wrapper that sets up environment the same way a test would use it.
Use it for quick prototyping or testing, the commands and output is
logged.
Signed-off-by: David Sterba <dsterba@suse.com>
Split the big function to several helpers so we can use them separately.
Add comments and do minor tweaks.
Signed-off-by: David Sterba <dsterba@suse.com>
We really use root only for mount/umount and access to the ext2_saved
image (that has 0600). Also switch to common variable so we can use
helpers.
Signed-off-by: David Sterba <dsterba@suse.com>
New convert introduced simpler chunk/extent allocation algorithm, at the
cost of complex backup superblock migration codes.
Use specially built ext2 images to test if btrfs-convert can convert and
rollback images without problem.
All these special ext2 image have blocks/holes across 2nd btrfs backup
superblock.
The naming of test image is like the following:
|<------superblock migration range----->|
64M 64M + 64K
|-Data--|-Data--|/Hole//|-Data--|/Hole//|-Data--|--Data--| = drdhdhdrd
These test cases should check all typical layouts and make sure new
convert works.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add support for custom convert test scripts, just like fsck tests.
Instead of generic convert tests, we need more specifically created images
for new convert tests.
This patch provide the needed infrastructure for later convert test
cases.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
New convert framework uses new and simpler chunk layout, while the cost
is the more complex superblock range migration logical, compared to old
convert.
Enhance the convert test script to create file which will takes up 2nd
backup superblock space, to ensure the superblock migration is working
as expected.
Suggested-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Some of the files might not end up in the checksum list because of
permissions. This is reported by md5sum as incorrectly formatted lines.
Signed-off-by: David Sterba <dsterba@suse.com>
In order to confirm that btrfsck supports to check a variety of
refs, add the
following cases:
* keyed_block_ref
* keyed_data_ref
* shared_block_ref
* shared_data_ref
* no_inline_ref (a extent item without inline ref)
* no_skinny_ref
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This adds 4 fuzz testing images, btrfsck either doesn't detect errors
in them or crashes immediately.
Reported-by: Vegard Nossum <vegard.nossum@oracle.com>
Reported-by: Quentin Casasnovas <quentin.casasnovas@oracle.com>
Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Test for "btrfs-progs: subvol sync: fix memory corruption, undersized
array", a lot of deleted subvolumes in the 'subvol sync' will not fit
into the array, should result in a glibc report.
Signed-off-by: David Sterba <dsterba@suse.com>
We use a device mapper device on top of a loop device, the change in
rotational status does not always propagate if change it at the loop
device sysfs node.
Signed-off-by: David Sterba <dsterba@suse.com>
Add regression test for btrfs-find-root gives empty result even the fs
is OK.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ enhanced test ]
Signed-off-by: David Sterba <dsterba@suse.com>
This commit extends the leaf corruption test to try to repair a file
linked from multiple directory. It stresses a case that some links to a
file is broken but others kept valid.
Signed-off-by: Naohiro Aota <naota@elisp.net>
Signed-off-by: David Sterba <dsterba@suse.com>
Catch a buggy condition fixed by "btrfs-progs: fsck: Fix a false alert
where extent record has wrong metadata flag"
Signed-off-by: David Sterba <dsterba@suse.com>
Create filesystem on a partitioned loop device, test for "btrfs-progs:
Fix partitioned loop devices resolving".
Signed-off-by: David Sterba <dsterba@suse.com>
A test for "btrfs-progs: mkfs: increase buffer size in is_ssd". Create a
device with a long name through loop device wrapped to a device mapper
linear device, switch it to the "ssd" mode status.
Signed-off-by: David Sterba <dsterba@suse.com>
With the rootdir option we try to guess the final size of the image and
fill it with zeros, preceded by truncation. After patch
"Btrfs-progs: Do not force mixed block group creation unless '-M' option
is specified"
the misc test 002 will fail, because of the non-mixed mode. I think we
should not touch the image size (no change for block devices) and try to
fit into whatever is provided by user.
Signed-off-by: David Sterba <dsterba@suse.com>
Mixed mode needs equal sectorsize and nodesize. This was fixed by
"Btrfs-progs: Prevent creation of filesystem with 'mixed bgs' and
having differing sectorsize and nodesize"
Signed-off-by: David Sterba <dsterba@suse.com>
No need to log expected failures to the terminal, the results file is
fine; pass the return value of the command.
Signed-off-by: David Sterba <dsterba@suse.com>
Add a way to wrap commands executed by the tests. This means the
common wrappers: run_check, run_check_stdout and run_mayfail , with the
exception of the use root_helper.
The contents of the shell variable INSTRUMENT are prepended to the
command, without quotes. Use with care.
Example: this has been tested with valgrind, the output goes to the
RESULTS file.
$ INSTRUMENT=valgrind make test-misc
Any use of root_helper/SUDO_HELPER will skip the instrumentation.
Signed-off-by: David Sterba <dsterba@suse.com>
We use setup_root_helper in some helpers to make sure that the sudo
helper is set up, and adding that to each test. Make the real test run
only once.
Signed-off-by: David Sterba <dsterba@suse.com>
Testcase for "Btrfs-progs: fix btrfs-convert rollback to check
ROOT_BACKREF", make sure we don't try a rollback if the ext2_subvol is
half-deleted.
Signed-off-by: David Sterba <dsterba@suse.com>
Set the variable TEST_LOG=tty (in the enviroment or as parameter to
make) to print commands executed by 'run_check' helpers to terminal (ie.
/dev/tty). This might be useful to see the test progress beside watching
the results file.
Signed-off-by: David Sterba <dsterba@suse.com>
002-bad-transid outout 'transid verify failed' message in screen
which is just a warning in btrfs-image in normal condition of this
test.
This patch move above warning into $RESULTS, to:
1: Avoid trouble screen output
2: Let user known detail if other error happened in btrfs-image
Before patch:
# ./fsck-tests.sh
[TEST] 001-bad-file-extent-bytenr
[TEST] 002-bad-transid
parent transid verify failed on 29360128 wanted 9 found 755944791
parent transid verify failed on 29360128 wanted 9 found 755944791
Ignoring transid failure
[TEST] 003-shift-offsets
[TEST] 004-no-dir-index
...
After patch:
# ./fsck-tests.sh
[TEST] 001-bad-file-extent-bytenr
[TEST] 002-bad-transid
[TEST] 003-shift-offsets
[TEST] 004-no-dir-index
...
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
To check is btrfs-convert create bad filesystem with
leaf across stripes.
It is happened in progs version <=v4.1.2, and fixed by patch titled:
btrfs: convert: Avoid allocating metadata extent crossing stripe boundary
which was merged in v4.2.
Notice thar this testcase can not report error in old version of
btrfs-progs, because "btrfs check" can't check this type of error
in those version, but we have another testcase in fsck-tests, to
check is "btrfs check" support this check.
So, the above 2 testcase together can check leaf-crossing-stripes
bug in all versions.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
[renamed and other minor changes]
Signed-off-by: David Sterba <dsterba@suse.com>
To test if fsck can check detec "leaf crossing stripes".
This function was introduced from patch titled:
btrfs-progs: fsck: Check if a metadata tree block crossing stripe boundary
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
[renamed and other minor changes]
Signed-off-by: David Sterba <dsterba@suse.com>
Move code for extract image file to a function from check_all_images()
for common use, so caller can use this function to extrace single
image file.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
[minor reformatting and updates]
Signed-off-by: David Sterba <dsterba@suse.com>
To fix following bug:
# ./convert-tests.sh
[TEST] ext2 4k nodesize, btrfs defaults
failed: mount /root/btrfsprogs/tests/test.img /root/btrfsprogs/tests/mnt
# tail convert-tests-results.txt
...
############### mount /root/btrfsprogs/tests/test.img /root/btrfsprogs/tests/mnt
mount: /root/btrfsprogs/tests/test.img is not a block device (maybe try `-o loop'?)
failed: mount /root/btrfsprogs/tests/test.img /root/btrfsprogs/tests/mnt
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
A collection of several images that were produced in a non-standard way
and cause various errors in check or image tools. They do not fit into
the fsck tests as we're not able to repair any of them, but the tools
should not crash or do out-of-bounds access.
Signed-off-by: David Sterba <dsterba@suse.com>
So this test can support both block device and loop device simply.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
mount command in old system can not add "-o loop" option automatically
for loop device, and make following test failed:
# ./fsck-tests.sh
...
[TEST] 013-extent-tree-rebuild
failed: mount /data/btrfsprogs/tests/test.img /data/btrfsprogs/tests/mnt
test failed for case 013-extent-tree-rebuild
Considering that $TEST_DEV can be block or loop device, we need determine
our mount option in a condition for both case.
This patch create a wrapper function for above request, to solve current
problem in 013-extent-tree-rebuild, and support similar request in future.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
mount command in old system can not add "-o loop" option automatically
for a loop device, and make following test 013-extent-tree-rebuild fail.
Considering that $TEST_DEV can be block or loop device, we need to determine
our mount option in a condition for both case.
Introduce a wrapper that will add the loop options if needed.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Some test failed in my nfs dir:
...
[TEST] 006-bad-root-items
tar: test.img: Cannot change ownership to uid 1000, gid 1000: Invalid argument
tar: Exiting with failure status due to previous errors
failed to extract default_case.tar.xz
test failed for case 006-bad-root-items
It is because the image file's owner is:
# tar tvf default_case.tar.xz
-rw-r--r-- fdmanana/fdmanana 2147483648 2014-10-17 17:59 test.img
And make tar failed in chown in nfs.
It is not a big issue because we don't use nfs commonly, but extract
the image file with ownership of current user will be a better choice.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
If a testcase failed, we can't run it(or other tests needs mount) again,
# ./misc-tests.sh 007
[TEST] 007-subvolume-sync
failed: fail
test failed for case 007-subvolume-sync
# ./misc-tests.sh 007
[TEST] 007-subvolume-sync
failed: mount /root/btrfs-progs/tests/test.img /root/btrfs-progs/tests/mnt
test failed for case 007-subvolume-sync
This patch add "umount $TEST_MNT" to clean-tests.sh, to let user
clean mountpoint easily.
After patch:
# ./misc-tests.sh 007
[TEST] 007-subvolume-sync
failed: fail
test failed for case 007-subvolume-sync
#
# clean-tests.sh
#
# ./misc-tests.sh 007
[TEST] 007-subvolume-sync
failed: fail
test failed for case 007-subvolume-sync
Suggested-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
[added sudo helper to umount]
Signed-off-by: David Sterba <dsterba@suse.com>
For example, $TEST_DIR is common used in severial tests, and have
duplicated code for initialize.
These duplicated code not only benifits harddisk vendor, but have
inconsistent details, as:
convert-tests.sh: lack of mkdir
fsck-tests/012-leaf-corruption/test.sh: unnecessary mkdir
fsck-tests/013-extent-tree-rebuild/test.sh: unnecessary init
misc-tests/XXX ...
And severial error message:
_fail "unable to create mount point on $TEST_MNT"
_fail "failed to create mount point"
...
This patch move initizlizaton of $TEST_DIR to common init_env(),
to avoid above problem, and init_env() can be used to add more
things in future.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Current code have following bug for subvolume sync:
1: If there are more than 1 subvolume to sync, the program will
infinitely loop.
2: return !0 in exit
This patch add misc-tests/007-subvolume-sync for above case.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Create a 2 device raid1 filesystem, fill with some data, delete one
device (the device node must not exist) and try to dump a filesystem
image.
Signed-off-by: David Sterba <dsterba@suse.com>
Add test case for inode with no file extents, but still non-zero size.
To test whether fsck will infinite loop.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Current code of btrfs-convert have a bug of thread conflict, which caused
invalid memory accessing between threads, and make program panic.
This patch add a test item for above bug, as:
# ./misc-tests.sh
[TEST] 001-btrfstune-features
[TEST] 002-uuid-rewrite
[TEST] 003-zero-log
[TEST] 004-convert-thread-conflict
failed: btrfs-convert /root/btrfsprogs/tests/test.img
test failed for case 004-convert-thread-conflict
#
# cat misc-tests-results.txt
...
############### btrfs-convert /root/btrfsprogs/tests/test.img
trans 7 running 5
ctree.c:363: btrfs_cow_block: Assertion `1` failed.
btrfs-convert(btrfs_cow_block+0x92)[0x40acaf]
btrfs-convert(btrfs_search_slot+0x1cb)[0x40c50f]
btrfs-convert(btrfs_csum_file_block+0x20f)[0x41d83a]
btrfs-convert[0x43422d]
btrfs-convert[0x4342cd]
btrfs-convert[0x4345ca]
btrfs-convert[0x434767]
btrfs-convert[0x435770]
btrfs-convert[0x439748]
btrfs-convert(main+0x13f8)[0x43b09d]
/lib64/libc.so.6(__libc_start_main+0xfd)[0x335e01ecdd]
btrfs-convert[0x407649]
create btrfs filesystem:
blocksize: 4096
nodesize: 16384
features: extref, skinny-metadata (default)
creating btrfs metadata.
creating ext2fs image file.
failed: btrfs-convert /root/btrfsprogs/tests/test.img
test failed for case 004-convert-thread-conflict
#
Note that this bug is not happened every time, especilly in slow
device as loop(slow cpu with fast block device is likely to trigger).
I set loop count to 20 to make bug happened in 90% tests.
Suggested-by: David Sterba <dsterba@suse.com>
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This code block is used several tests, move it to ./common and add a
helper.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
To avoid following mount error in test:
mount: /root/btrfs/progs/tests/fsck-tests/012-leaf-corruption/test.img
is not a block device (maybe try `-o loop'?)
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
To run a given test set the variable TEST like
$ make test TEST=002-bad-transid
$ make test TEST=002-*
and only tests matching the value will be run. The pattern is glob and
pased to 'find -name'.
The convert tests do not follow the fsck and misc layout and are skipped
if TEST is set.
Signed-off-by: David Sterba <dsterba@suse.com>
Previously in 'filesystem resize get_min_size', now
'inspect-internal min-dev-size'. We'd like to avoid cluttering the
'resize' syntax further.
The test has been updated to exercise the new option.
Signed-off-by: David Sterba <dsterba@suse.com>
Currently there is not way for a user to know what is the minimum size a
device of a btrfs filesystem can be resized to. Sometimes the value of
total allocated space (sum of all allocated chunks/device extents), which
can be parsed from 'btrfs filesystem show' and 'btrfs filesystem usage',
works as the minimum size, but sometimes it does not, namely when device
extents have to relocated to holes (unallocated space) within the new
size of the device (the total allocated space sum).
This change adds the ability to reliably compute such minimum value and
extents 'btrfs filesystem resize' with the following syntax to get such
value:
btrfs filesystem resize [devid:]get_min_size
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a new test case for I_ERR_FILE_WRONG_NBYTES.
The new btrfs-image dump image contains a file in 12K size.
But nbytes in its inode item is a random number.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
The 'Current fsid:' value does not match the real fsid.
Reported-by: Mike Fleetwood <mike.fleetwood@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Simple test to verify that the log_root is reset after the command, but
we yet need to provide a testing image with log_root set to something
sensible or crafted images with borked log_root pointer.
Signed-off-by: David Sterba <dsterba@suse.cz>
A sample test for the misc-test category. Verify that btrfstune sets the
requested fs features. Now implemented extrefs, skinny-metadata and
seeding.
Signed-off-by: David Sterba <dsterba@suse.cz>
Sometimes we need to process the output of the command, but run_check
captures all the output into results file.
Signed-off-by: David Sterba <dsterba@suse.cz>
Currently this test uses the system btrfs-image. If there isn't a
btrfs-image on $PATH, the test fails. The test should be using the
locally compiled btrfs-image, not the system one.
Signed-off-by: WorMzy Tykashi <wormzy.tykashi@gmail.com>
Reviewed-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
We'd like to run each class of tests separately. There were some missing
prerequisities that should be/are verified by the tests, makefile rules
have been synced.
Signed-off-by: David Sterba <dsterba@suse.cz>
If the test image is eg. on NFS it's not writable for root, so chmod
0777 fixes that but then we must not delete the file.
Signed-off-by: David Sterba <dsterba@suse.cz>
Also change the test() to convert_test(), to avoid conflict name with
bash test function.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Add fallback TEST_DEV for test case 013.
Fallback to $TOP/tests/test.img.
Now all test cases of btrfs-progs need no extra setting except sudo.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Update 013-extent-tree-rebuild to use more framework infrastructure,
including:
1) Use run_check other than open-coded redirect
2) Add root privillege
3) Add dependency on 'btrfs-debug-tree' command
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Since we have already had TEST_MNT fallback setup to $TOP/tests/mnt,
just remove duplicated setting in 012-leaf-corruption/test.sh
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Use upper case variant name for the following variants:
1) top -> TOP
2) script_dir -> SCRIPT_DIR
And change the following variant name:
1) RESULT -> RESULTS
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
convert-tests now test both 4096 and 16384 nodesizes.
Signed-off-by: Sebastian Thorarensen <sebth@naju.se>
Signed-off-by: David Sterba <dsterba@suse.cz>
Although fsck-test/012 uses sudo, it uses 'sudo -n', which won't prompt
user to input password and will return 1 if no valid credential is
found.
And this makes test result quite annoying since it fails to mount and
still continue, which will always fail.
This patch will check 'sudo -v -n' and 'sudo -n true' to determine
whether sudo works fine in different version/settings, since in some
setting/version, 'sudo -v -n' will fail even the user is set NOPASSWD.
Also, remove the 'have_root_helper' variant, since there is a
possibility that sudo credential will timeout during the test and
'have_root_helper' won't help to detect such problem.
New '_sudo' command will do credential check if needed to avoid such
problem.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Previously we used to just set FULL_BACKREF if we couldn't lookup an extent info
for an extent. Now we just bail out if we can't lookup the extent info, which
is less than good since fsck is supposed to fix these very problems. So instead
figure out the flag we are supposed to use and pass that along instead. This
patch also provides a test image to test this functionality. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Sudo may not be configured to run without user prompt, in that case the
tests would be stuck.
Reported-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Most of the checks run fine without root, but some of them may need to
do a mount test or access the data. Add the support to selectively run
commands under root, hardcoded to sudo for now.
Signed-off-by: David Sterba <dsterba@suse.cz>
Move leaf-corruption type no extent data case to its dir, and add
verification script in test.sh
The verification script is based on manual btrfs-debug-tree check.
The image can also be reused to other leaf-corruption type, like
corrupted leaf contains regular file extent data case.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Now 006-bad-root-items has two cases, one for default case, and one for
skinny metadata case.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Move these obvious btrfs-image to its corresponding dir to use the new
infrastructure.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Change the old btrfsck test infrastructure (btrfs-image dump or xz raw
dump) to the new test infrastructure.
1) Test case layout
The new infrastructure is dir based, each dir is one test type, and can
contain multiple images/scripts for different corner cases.
So layout will be the following:
btrfs-progs
|-tests
|-fsck-tests
|-001-SOME-CORRUPT-TYPE
|-IMAGE-FOR-CASE1
|-IMAGE-FOR-CASE2
2) Test case image types
Only 2 types for test case images.
a) btrfs-image dump
This one is the simplest case, one only needs to add the image to
corresponding dir.
b) custom script
This one is for all the resting cases which can't fit btrfs-image,
like csum error or script can generate the image (this reduces the
size obviously and good for review)
The old binary dump also belongs to this type, so need to add script
to extract them.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This adds two new test images
1) 008-bad-offset-snapshots. This has a corrupt item with multiple snapshots
pointing to it, to make sure the bad block repair stuff doesn't loop and
actually repairs stuff. It also requires the dir index repair stuff to pass our
built in tests which is why it's not tied to the same commit.
2) 009-bad-dir-index-name.img. This has a corrupt name in a dir index to make
sure our dir index repair stuff is working properly.
Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
These test the recreating of missing dir item/dir index pairs, fixing the no
rootdir inode item and no inode item for normal files. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
This change adds code to detect and fix the issue introduced in the kernel
release 3.17, where creation of read-only snapshots lead to a corrupted
filesystem if they were created at a moment when the source subvolume/snapshot
had orphan items. The issue was that the on-disk root items became incorrect,
referring to the pre orphan cleanup root node instead of the post orphan
cleanup root node.
A test filesystem can be generated with the test case recently submitted for
xfstests/fstests, which is essencially the following (bash script):
workout()
{
ops=$1
procs=$2
num_snapshots=$3
_scratch_mkfs >> $seqres.full 2>&1
_scratch_mount
snapshot_cmd="$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT"
snapshot_cmd="$snapshot_cmd $SCRATCH_MNT/snap_\`date +'%H_%M_%S_%N'\`"
run_check $FSSTRESS_PROG -p $procs \
-x "$snapshot_cmd" -X $num_snapshots -d $SCRATCH_MNT -n $ops
}
ops=10000
procs=4
snapshots=500
workout $ops $procs $snapshots
Example of btrfsck's (btrfs check) behaviour against such filesystem:
$ btrfsck /dev/loop0
root item for root 311, current bytenr 44630016, current gen 60, current level 1, new bytenr 44957696, new gen 61, new level 1
root item for root 1480, current bytenr 1003569152, current gen 1271, current level 1, new bytenr 1004175360, new gen 1272, new level 1
root item for root 1509, current bytenr 1037434880, current gen 1300, current level 1, new bytenr 1038467072, new gen 1301, new level 1
root item for root 1562, current bytenr 33636352, current gen 1354, current level 1, new bytenr 34455552, new gen 1355, new level 1
root item for root 3094, current bytenr 1011712000, current gen 2935, current level 1, new bytenr 1008484352, new gen 2936, new level 1
root item for root 3716, current bytenr 80805888, current gen 3578, current level 1, new bytenr 73515008, new gen 3579, new level 1
root item for root 4085, current bytenr 714031104, current gen 3958, current level 1, new bytenr 716816384, new gen 3959, new level 1
Found 7 roots with an outdated root item.
Please run a filesystem check with the option --repair to fix them.
$ echo $?
1
$ btrfsck --repair /dev/loop0
enabling repair mode
fixing root item for root 311, current bytenr 44630016, current gen 60, current level 1, new bytenr 44957696, new gen 61, new level 1
fixing root item for root 1480, current bytenr 1003569152, current gen 1271, current level 1, new bytenr 1004175360, new gen 1272, new level 1
fixing root item for root 1509, current bytenr 1037434880, current gen 1300, current level 1, new bytenr 1038467072, new gen 1301, new level 1
fixing root item for root 1562, current bytenr 33636352, current gen 1354, current level 1, new bytenr 34455552, new gen 1355, new level 1
fixing root item for root 3094, current bytenr 1011712000, current gen 2935, current level 1, new bytenr 1008484352, new gen 2936, new level 1
fixing root item for root 3716, current bytenr 80805888, current gen 3578, current level 1, new bytenr 73515008, new gen 3579, new level 1
fixing root item for root 4085, current bytenr 714031104, current gen 3958, current level 1, new bytenr 716816384, new gen 3959, new level 1
Fixed 7 roots.
Checking filesystem on /dev/loop0
UUID: 2186e9b9-c977-4a35-9c7b-69c6609d4620
checking extents
checking free space cache
cache and super generation don't match, space cache will be invalidated
checking fs roots
checking csums
checking root refs
found 618537000 bytes used err is 0
total csum bytes: 130824
total tree bytes: 601620480
total fs tree bytes: 580288512
total extent tree bytes: 18464768
btree space waste bytes: 136939144
file data blocks allocated: 34150318080
referenced 27815415808
Btrfs v3.17-rc3-2-gbbe1dd8
$ echo $?
0
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
We can deal with corrupt items by deleting them in a few cases. Fsck can easily
recover from a missing extent item or a dir index item. So if we notice a item
is completely bogus and it is of a key that we know we can repair then just
delete it and carry on. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
A user had a corrupted fs where the items had been shifted improperly. This
patch adds the ability to fix this sort of problem within fsck. We will simply
shift the item over to the proper offset and update the offsets to make sure
they are correct. I tested this with a hand crafted fs that was broken in the
same way as the user, and I've included the file as a new test. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If we have an inode backref entry then we know enough to add back a missing dir
index. When messing with the inode backrefs we need to do all of that first
before we process the inode recs themselves as we may clear errors on the inode
recs as we fix the directory indexes. This adds the framework for fixing
backref errors and fixes missing dir index issues. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
It is highly obnoxious to have to go put in a testdev when all you really want
is to run the quick image tests. Make this part optional so if we don't have a
testdev specified we just don't run that particular test. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
We need test to verify extent tree rebuilding work, this test
create a strange filesystem with some snapshots, destroy
extent root node, and run fsck with "--init-extent-tree".
Since this tests need btrfs internal tool(btrfs-corrupt-block),so
i add this test into btrfs-progs.
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
On my system, this brings the FS conversion test suite's runtime from over
ten seconds down to under two.
Thanks to Julien Muchembled for the suggestion.
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
These use the system's mke2fs, and don't require loop devices
or root privileges.
They don't pick up anything with the default flags right now,
but they do pick up some sanitizer issues when the tools are
compiled with any of -fsanitize={address,memory,thread}.
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This is a verification test for the transid recow functionality of btrfsck.
I've also adjusted the test script to spit out which image it's testing so I can
be sure the image was getting tested. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
We need to start adding some sanity tests to btrfs-progs to make sure we aren't
breaking things with our patches. The most important of these tools is btrfsck.
This patch gets things started by adding a basic btrfsck test that makes sure we
can fix a corruption problem we know we can fix. Thanks,
Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Qu Wenruo