Use TEST_TOP as base for tests to reference any files, this will be used
for git and external testsuite.
INTERNAL_BIN is needed for referencing binaries that could reside in
different paths in git vs external testsuite.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ add quotes around sourced files, update changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
Move the testsuite to tests/ and make the tarball generation more
deterministic. As there could be many random temporary files left in the
test directories, we can't just copy them. Use 'git ls-tree' to
filter just what we want, this needs a slight extension of the file list
specification.
Signed-off-by: David Sterba <dsterba@suse.com>
Export the testsuite files to a separate tar. Since fsck tests depend
on btrfs-corrupt-block, and misc tests depends on both
btrfs-corrupt-block and fssum, so set it as prerequisites for package
commad.
Because, althougth fssum can be generated by source that are all in
tests directory, and has no rely on the btrfs's structure. But
btrfs-corrupt-block deeply relys on btrfs's structure. For consistency,
at the present stage, generete the two when create test tar.
Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ applied without changes, the generated tarball will be different from
the one after the follow up commits ]
Signed-off-by: David Sterba <dsterba@suse.com>
There are 2 fsck tests with the same number 027:
tree-reloc-tree
bad-extent-inline-ref-type
And we also have a hole in 015, so just rename tree-reloc-tree to 015,
to get rid of the duplicated test number and fill in the hole.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The run_check_umount_test_dev umounts the TEST_DEV and also optionally
uses the arguments but this would not work as expected if the TEST_DEV
is not a vald path for umount (eg. a restored image).
Update the helper so it tries to umount all paths, or fallback to
TEST_DEV to keep the current behaviour.
Signed-off-by: David Sterba <dsterba@suse.com>
Node sizes larger than 16k will fail due to enospc in the mount test.
This is likely caused by the kernel. Keep the condition only local to
travis so any other testing environment could see the failure
eventually.
Signed-off-by: David Sterba <dsterba@suse.com>
As btrfs is specific to Linux, %m can be used instead of strerror(errno)
in format strings. This has some size reduction benefits for embedded
systems.
glibc, musl, and uclibc-ng all support %m as a modifier to printf.
A quick glance at the BIONIC libc source indicates that it has
support for %m as well. BSDs and Windows do not but I do believe
them to be beyond the scope of btrfs-progs.
Compiled sizes on Ubuntu 16.04:
Before:
3916512 btrfs
233688 libbtrfs.so.0.1
4899 bcp
2367672 btrfs-convert
2208488 btrfs-corrupt-block
13302 btrfs-debugfs
2152160 btrfs-debug-tree
2136024 btrfs-find-root
2287592 btrfs-image
2144600 btrfs-map-logical
2130760 btrfs-select-super
2152608 btrfstune
2131760 btrfs-zero-log
2277752 mkfs.btrfs
9166 show-blocks
After:
3908744 btrfs
233256 libbtrfs.so.0.1
4899 bcp
2366560 btrfs-convert
2207432 btrfs-corrupt-block
13302 btrfs-debugfs
2151104 btrfs-debug-tree
2134968 btrfs-find-root
2281864 btrfs-image
2143536 btrfs-map-logical
2129704 btrfs-select-super
2151552 btrfstune
2130696 btrfs-zero-log
2276272 mkfs.btrfs
9166 show-blocks
Total savings: 23928 (24 kilo)bytes
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We use the prepare_test_dev helper to make sure the image has at least
this size. The "at least" part is not desired by some tests as the
device might be larger than the test expects.
Signed-off-by: David Sterba <dsterba@suse.com>
Though the newly added mkfs profiles should not be affected, let's add
the remaining valid single device profiles for better coverage.
Signed-off-by: David Sterba <dsterba@suse.com>
Add test case to check if the first device extent is occupying reserved
0~1M range.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Transform the test to the common helpers and don't manage the loop
devices here. The test category changes from check to misc.
Signed-off-by: David Sterba <dsterba@suse.com>
This functionality regressed some time ago and it was never caught. Seems no
one complained of that, but to be sure add a regression test to prevent future
regressions.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
To test regression 460e93f25754 ("btrfs-progs: mkfs: check the status of
file at mkfs").
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ update test to create a out of /tmp ]
Signed-off-by: David Sterba <dsterba@suse.com>
New test case to test if the minimal device size given by "mkfs.btrfs"
failure case is valid.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ renamed script ]
Signed-off-by: David Sterba <dsterba@suse.com>
So prepare_test_dev() can be called several times in one test case, to
test different device sizes.
Signed-off-by: Qu Wenruo <wqu@suse.com>
[ switch to [ ] ]
Signed-off-by: David Sterba <dsterba@suse.com>
With extended tests in the following patch a file based filesystem image
also needs -f, otherwise it will fail.
Signed-off-by: David Sterba <dsterba@suse.com>
Reloc tree is a special tree with very short life span. It acts as a
special snapshot for any tree, with related nodes/leaves or EXTENT_DATA
modified to point to new position.
Considering the short life span and its special purpose, it should be
quite reasonable to keep them as both corner case for fsck and
educational dump for anyone interested in relocation.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For snapshot shared tree blocks with source subvolume, the keyed backref
counter only counts the exclusive owned references.
In the following case, 258 is a snapshot of 257, which inherits all the
reference to this data extent.
------
item 4 key (12582912 EXTENT_ITEM 524288) itemoff 3741 itemsize 140
refs 179 gen 9 flags DATA
extent data backref root 257 objectid 258 offset 0 count 49
extent data backref root 257 objectid 257 offset 0 count 1
extent data backref root 256 objectid 258 offset 0 count 128
extent data backref root 256 objectid 257 offset 0 count 1
------
However lowmem mode used to iterate the whole inode to find all
references, and doesn't care if a reference is already counted by the
shared tree block.
Add the test case to check it.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image is dumped by modifying kernel to sleep long enough before
merging relocation trees, so we can just copy the whole image to other
place before kernel begins to merge reloc trees.
And the base image is created by the following script to bump metadata
size:
------
dev=~/test.img
mnt=/mnt/btrfs
umount $mnt &> /dev/null
fallocate -l 128M $dev
mkfs.btrfs -f -n 4k -m single -d single $dev
mount $dev $mnt -o nospace_cache,max_inline=2048
btrfs subvolume create $mnt/src
for i in $(seq -w 0 128); do
xfs_io -f -c "pwrite 0 2k" $mnt/src/file_$i > /dev/null
done
for i in $(seq -w 0 64); do
btrfs subvolume snapshot $mnt/src/ $mnt/snapshot_$i
touch $mnt/snapshot_$i/new
done
sync
------
The image triggers several corner cases that the old lowmem mode didn't
consider.
Like metadata backref with FULL_BACKREF flag and only SHARED_BLOCK_REF
backrefs for metadata. And several tree reloc trees with shared
leaves/nodes to confuse old lowmem mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new test image, which has an extent item with no inlined
extent data ref, but all keyed extent data ref.
Only in this case we can trigger fase data extent backref lost bug in
lowmem mode.
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a image which can reproduce the extent item referencer count
mismatch false alert for lowmem mode.
Reported-by: Marc MERLIN <marc@merlins.org>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add a image that the inlined extent coexist with the regular extent.
Reported-by: Marc MERLIN <marc@merlins.org>
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The image has 2 problems mixed:
1) Too small super total_bytes
This super total_bytes is manually modified to create such problem.
2) Unaligned dev item total_bytes
This is created by v4.12 kernel, with 128M + 2K device added, and
original device removed.
Then we can create such image with unaligned dev item total_bytes.
Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add new test to check functionality of subvol get/set-default.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
[ fix style issues, add missing SUDO_HELPER ]
Signed-off-by: David Sterba <dsterba@suse.com>
This case is for avoiding crash in lowmem check mode.
Field type of extent_inline_ref in an extent is corrupted.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since lowmem mode can repair certain corruptions (mostly in fs tree),
insert a beacon into each fsck test cases to allow some of them be
tested in lowmem mode.
With this patch, fsck option override will check the beacon file
".lowmem_repairable" in the same directory of the test image, and if the
beacon exists, then it will also run lowmem mode repair to repair the
image.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Lowmem mode only repairs few cases which has a beacon file
".lowmem_repairable" in the case' directory.
However, defining TEST_ENABLE_OVERRIDE=true in command line does work
in above strategy.
Because _skip_spec() in tests/common.local isn't interpreted by shell
in that case.
Solve it by making _skip_spec() always be defined in common.local.
Reported-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
[ keep the _skip_spec check ]
Signed-off-by: David Sterba <dsterba@suse.com>
kernel 4.14 introduces new function for checking if all chunks is ok for
mount with -o degraded option.
commit 21634a19f646 ("btrfs: Introduce a function to check if all
chunks a OK for degraded rw mount")
As a result, raid0 profile cannot be mounted with -o degraded on 4.14.
This causes failure of the misc-test 011 "delete missing device".
Fix this by using raid1 profile for both data and metadata.
This also should work for kernel before 4.13.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
BCP 78 applies to RFC 6234, but sha224-256.c is Simplified BSD.
This causes the following lintian error when building on Debian and
Debian derivatives:
E: btrfs-progs source: license-problem-non-free-RFC-BCP78
tests/sha224-256.c
Please consult the following email from debian-legal@lists.debian.org
for more information:
https://lists.debian.org/debian-legal/2017/08/msg00004.html
Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
[ I've copied too much from the RFC that's not related to the code,
covered by the explicit copyright notice in the file ]
Signed-off-by: David Sterba <dsterba@suse.com>
The test cli/007-check-force reports something like:
$ type -p '--string that starts with dashes'
bash: type: --: invalid option
Add the option/argument separator.
Signed-off-by: David Sterba <dsterba@suse.com>
Adds tests for the new features based on a prebuilt btrfs image with a
zstd compressed file.
Split from the previous patch.
Signed-off-by: Nick Terrell <terrelln@fb.com>
[ with some cleanups ]
Signed-off-by: David Sterba <dsterba@suse.com>
The sanity check in run_mustfail does not work as expected and allowed
the command to continue without the missing parameter
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=196893
Signed-off-by: David Sterba <dsterba@suse.com>
This new test checks inspect-internal rootid
- handle path to subvolume/directory/file as an argument
- get different id for each subvolume
- get the expected id for each file/directory (i.e. the same as
containing subvolume)
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The post-rollback helper still assumes just extN, we need an extra
argument that'll get passed to fsck. Change all callsites at once so the
tests do not fail temporarily.
Signed-off-by: David Sterba <dsterba@suse.com>
First patch causes test-convert fails. This is because
generate_dataset() creates a name containing trailing spaces for
"slow_symlink" type, and cause getfacl error in convert_test_perm().
(This is not noticed since original run_check_stdout() throws away the
error.)
Fix this by use space for delimiter for cut.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
run_check_stdout() uses "... | tee ... || _fail". However, since tee
won't fail, _fail() is not called even if first command fails.
Fix this by checking PIPESTATUS in the end.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add test case which checks if -r|--rootdir mkfs option can handle
symlink/char/block/fifo files.
Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The convert tests generate lots of log material, travis CI has limit 4MB
so we don't see anything useful when a late test fails.
Signed-off-by: David Sterba <dsterba@suse.com>
Test failure in convert tests with log dump does not happen because
_fail is called before that and exits. Other test types are ok.
Signed-off-by: David Sterba <dsterba@suse.com>
Many of the test cases for convert apply regardless of what the source
file system is and using ext4 is sufficient. I've included several
test cases that are reiserfs-specific.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
[ patch split from the previous one, minor cleanups in common.convert ]
Signed-off-by: David Sterba <dsterba@suse.com>
type -p returns an empty string for nonexistent commands, but the -f
test on an empty string does not behave the same on all shells. To be
safe, use the quoted value.
Signed-off-by: David Sterba <dsterba@suse.com>
This function has an extra argument and can get forgotten, add a sanity
check so the bad usage can be caught during development.
Signed-off-by: David Sterba <dsterba@suse.com>
In this test case, all name in dir_item, dir_index, inode_ref
are corrupted to another one.
btrfs check should report errors about the corrupted dir_item but
btrfs can't repair the case now.
Signed-off-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The dd in convert-tests/008-readonly-image is expected to fail, so
there being a typo in the file name has gone unnoticed.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Without it, mount (at least from util-linux 2.20.1) tries (and fails) to
mount some filesystems as NTFS.
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We'll enforce the btrfs type for mount everwhere so we must provide a
way to mount converted filesystems. Add a new helper that will try to
mount the given type.
Signed-off-by: David Sterba <dsterba@suse.com>
We'll use mount -t $fstype later on, extend the API so we don't have to
parse the type from other parameters.
Signed-off-by: David Sterba <dsterba@suse.com>
Doing a straight 'make test' would fail because some misc and fsck
tests require particular tools to already be built. Add dependencies
at the Makefile and shell-script level.
Signed-off-by: Adam Buchbinder <abuchbinder@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add test case which we have NO_HOLES incompat flag while still have
hole file extent.
This can be created by enabling NO_HOLES feature on an existing
filesystem, which lowmem mode would cause false alert for it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ minor adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
Test that we are able to create an image from a multiple devices fs, that
we are able to restore that image into a single device and finally that we
are able to mount it.
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: Liu Bo <bo.li.liu@oracle.com>
[ added shell quotation and chmod a+w so testsuite on NFS works ]
Signed-off-by: David Sterba <dsterba@suse.com>
Patches "btrfs-progs: tests: correctly receive clones to mounted subvol"
(8eaf63bc9a) and followup are missing last
unmount which leads to failure of misc/020.
Signed-off-by: David Sterba <dsterba@suse.com>
tests/fssum.c:599:13: warning: In the GNU C Library, "major" is defined
by <sys/sysmacros.h>. For historical compatibility, it is
currently defined by <sys/types.h> as well, but we plan to
remove this soon. To use "major", include <sys/sysmacros.h>
directly. If you did not intend to use a system-defined macro
"major", you should undefine it after including <sys/types.h>.
sum_add_u64(&cs, major(st.st_rdev));
Signed-off-by: David Sterba <dsterba@suse.com>
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
[ enhance tests to take extra options and use for -e ]
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new image, which contains external SHARED_DATA_REF items to
trigger a lowmem mode false alert.
The image only contains external SHARED_DATA_REF and no inlined data
backref.
Before the image, we only have inlined shared data ref, which is not
enough to trigger lowmem mode false alert.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In order to drop dependency on SSL library to compute MD5 in fssum,
we'll use the reference implementation from RFC 6234.
The checksum is not in a cryptographically sensitive context, but we're
going to skip MD5 and SHA-1 anyway.
Signed-off-by: David Sterba <dsterba@suse.com>
In file included from /usr/include/stdio.h:27:0,
from kerncompat.h:22,
from tests/fssum.c:25:
/usr/include/features.h:148:3: warning: #warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-Wcpp]
# warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
^~~~~~~
We've solved that long time ago and config.h now provides the macros.
Signed-off-by: David Sterba <dsterba@suse.com>
Gu Jinxiang