Commit Graph

3769 Commits (8eccfa86c0c7f1645556a119d8b217d7173af10d)
 

Author SHA1 Message Date
David Sterba 8eccfa86c0 btrfs-progs: fix debugging macro checks
Sparse warns that the DEBUG_*_ERROR macros are not defined when checked.
Use the ifdef check rather then relying on the implicit behaviour.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:17 +02:00
Nicholas D Steeves 5b2fbc6f4e btrfs-progs: tests: Remove misleading BCP 78 boilerplate from SHA implementation
BCP 78 applies to RFC 6234, but sha224-256.c is Simplified BSD.

This causes the following lintian error when building on Debian and
Debian derivatives:

    E: btrfs-progs source: license-problem-non-free-RFC-BCP78
       tests/sha224-256.c

Please consult the following email from debian-legal@lists.debian.org
for more information:

https://lists.debian.org/debian-legal/2017/08/msg00004.html
Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
[ I've copied too much from the RFC that's not related to the code,
  covered by the explicit copyright notice in the file ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:15 +02:00
Nicholas D Steeves ac71751828 btrfs-progs: tests: Add required IETF Trust copyright to SHA implementation
Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:12 +02:00
David Sterba bc2c60ad31 btrfs-progs: docs: start ioctl documentation manual page
Start documenting the ioctl interface to btrfs. The overall structure
should be settled, the formatting of the ioctl description may change in
the future, newly added ioctl descriptions should follow the examples of
BTRFS_IOC_SUBVOL_CREATE.

The document is not finished yet and will not be installed until most of
ioctls' details are filled in.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:09 +02:00
David Sterba 509af95c92 btrfs-progs: tests: make sure _is_file_or_command does not get confused
The test cli/007-check-force reports something like:

$ type -p '--string that starts with dashes'
bash: type: --: invalid option

Add the option/argument separator.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:18:05 +02:00
Qu Wenruo a7e555d706 btrfs-progs: Fix one-byte overlap bug in free_block_group_cache
free_block_group_cache() calls clear_extent_bits() with wrong end, which
is one byte larger than the correct range.

This will cause the next adjacent cache state to be split.  And due to
the split, private pointer (which points to block group cache) will be
reset to NULL.

This is very hard to detect as this function only gets called in
cleanup_temp_chunks() which is just before mkfs finishes.  This bug only
gets exposed when reworking --rootdir option.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:52 +02:00
Qu Wenruo 02b58051e6 btrfs-progs: Refactor find_next_chunk to get rid of parameter root and objectid
Function find_next_chunk() is used to find next chunk start position,
which should only do search on chunk tree and objectid is set to
BTRFS_FIRST_CHUNK_TREE_OBJECTID.

So refactor the parameter list to get rid of @root, which should be
obtained from fs_info->chunk_root, and @objectid, which is set to
BTRFS_FIRST_CHUNK_TREE_OBJECTID.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:48 +02:00
Naohiro Aota 3efd322db2 btrfs-progs: build: omit unnecessary -MD flag
According to gcc(1), "-MD is equivalent to -M -MF file, except that -E is not
implied." Since the rule in the Makefile is just generating dependency file
and not building object file, it is no use to have "-MD" here. Also, it's
overridden and conflicting with the following "-MM" flag. I guess we can drop
it.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:45 +02:00
Naohiro Aota fd8c78a47c btrfs-progs: build: generate all dependency files
We're missing several dependency files like:

$ diff -u <(find -name '*.o'|cut -d. -f2|sort) <(find -name '*.o.d'|cut -d. -f2|sort)
@@ -3,7 +3,6 @@
 /btrfs-corrupt-block
 /btrfs-debug-tree
 /btrfs-find-root
-/btrfs-list
 /btrfs-map-logical
 /btrfs-select-super
 /btrfstune
@@ -29,11 +28,6 @@
 /cmds-scrub
 /cmds-send
 /cmds-subvolume
-/convert/common
-/convert/main
-/convert/source-ext2
-/convert/source-fs
-/convert/source-reiserfs
 /ctree
 /dir-item
 /disk-io
<snip>

This is due to moving things out of objects and cmds_objects variables. Such
missing dependency files cause mis-building of some source files (try touch
utils.h; make mkfs/main.o).

This patch introduce a new variable "all_objects" to keep all the objects and
use the variable to generate proper dependency file building rules.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:42 +02:00
yingyil f7267bc85c btrfs-progs: mkfs: refactor create_data_reloc_tree
Add an objectid parameter to make the function a general one for
inserting root items and rename it to create_tree. The change cascades
down to the callchain.

Signed-off-by: yingyil <yingyil@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:39 +02:00
David Sterba bcaadc2e49 btrfs-progs: build: whitespace adjustment of [LD] fssum
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:35 +02:00
David Sterba 01bcf68cf5 btrfs-progs: tests: remove temporary images in mkfs/005 and mkfs/006
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:31 +02:00
David Sterba 03fb34b6b4 btrfs-progs: tests: remove temporary loopdev files
Remove files for testing loop devices when using the helper.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:28 +02:00
David Sterba 29faad6339 btrfs-progs: convert: use correct string printing for errcode_t
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:21 +02:00
Misono, Tomohiro 629688a2d2 btrfs-progs: cleanup whitespaces of btrfs-completion
Clean up spaces and use only tabs. No functional change.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:19 +02:00
Nick Terrell 0766330f85 btrfs-progs: tests: add testing image for zstd for btrfs-restore
Adds tests for the new features based on a prebuilt btrfs image with a
zstd compressed file.

Split from the previous patch.

Signed-off-by: Nick Terrell <terrelln@fb.com>
[ with some cleanups ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:15 +02:00
Nick Terrell 717a8b1e5d btrfs-progs: Add zstd support
Adds zstd support to the btrfs program. An optional dependency on libzstd
>= 1.0.0 is added. Autoconf accepts `--enable-zstd' or `--disable-zstd' and
defaults to detecting if libzstd is present using `pkg-config'.

The patch is also available in my fork of btrfs-progs [1], which passes
Travis-CI with the new tests. The prebuilt binary is available there.

I haven't updated Android.mk.

[1] https://github.com/terrelln/btrfs-progs/tree/devel

Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:13 +02:00
Misono, Tomohiro 86529862e1 btrfs-progs: update btrfs-completion
This patch updates btrfs-completion:
 - add "filesystem du" and "rescure zero-log"
 - restrict _btrfs_mnts to show btrfs type only
 - add more completion in last case statements

(This file contains both spaces/tabs and may need cleanup.)

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:11 +02:00
David Sterba 5768fc03a4 btrfs-progs: build: sync Makefile to Android.mk
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:09 +02:00
David Sterba 67b7b4bc40 btrfs-progs: tests: fix run_mustfail in cli-tests/007-check-force
The sanity check in run_mustfail does not work as expected and allowed
the command to continue without the missing parameter

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=196893
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:06 +02:00
David Sterba fb48edcffb btrfs-progs: README: update links, enhance contribution section
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:04 +02:00
Piotr Pawlow 0e88d6d1d3 btrfs-progs: image: use CRC32C reversing instead of brute force to find collisions
Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:01 +02:00
Piotr Pawlow 334d83053a btrfs-progs: image: add a function to check if generated filename suffix is valid
Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:00 +02:00
Piotr Pawlow c859336f47 btrfs-progs: image: add a function to calculate CRC32C collisions
The function uses the reverse CRC32C table to quickly calculate a
4-byte suffix, that when added to the original data will make it
match desired checksum.

Author: Piotr Pawlow <pp@siedziba.pl>
[ minor adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:16:59 +02:00
Piotr Pawlow a172244163 btrfs-progs: image: add reverse CRC32C table
The table will be used to speed up calculations of CRC32C collisions.

Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:16:58 +02:00
Piotr Pawlow bc98731f4c btrfs-progs: image: move core find_collision code to a separate function
Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:16:56 +02:00
Piotr Pawlow 658e7b897b btrfs-progs: image: fix non-printable characters in generated file names
Function find_collision sometimes generated file names with non-printable
DEL characters (code 127), for example file name "|5gp!" would be changed
to "U'2<DEL>y" when using "crc-collisions" sanitize mode.

Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:16:54 +02:00
Uli Heller c5b6e3fad8 btrfs-progs: Removed missing header file, fixes compilation error
(From pull request #61.)

Author: Uli Heller <uli.heller@daemons-point.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:16:52 +02:00
David Sterba cb1be701ce
Btrfs progs v4.13
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:20:05 +02:00
David Sterba 3d341f5baa btrfs-progs: update CHANGES for v4.13
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Misono, Tomohiro a351dd8478 btrfs-progs: test: add new test for inspect-internal rootid
This new test checks inspect-internal rootid
 - handle path to subvolume/directory/file as an argument
 - get different id for each subvolume
 - get the expected id for each file/directory (i.e. the same as
	 containing subvolume)

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba 6fb88e2859 btrfs-progs: tests: check for kernel support for reiserfs
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba 7299e0d294 btrfs-progs: tests: enhance post-rollback fsck tests
The post-rollback helper still assumes just extN, we need an extra
argument that'll get passed to fsck. Change all callsites at once so the
tests do not fail temporarily.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Zhang Yu e96921bcaa Btrfs-progs: print-tree: check num_stripes in print_chunk
[TEST/fuzz] case: 004-simple-dump-tree

Since the wrong key(DATA_RELOC_TREE CHUNK_ITEM 0) in root tree,
error calling print_chunk(), resulting in num_stripes == 0.

ERROR:
     [TEST/fuzz]   004-simple-dump-tree
ctree.h:317: btrfs_chunk_item_size: BUG_ON `num_stripes == 0`
        triggered, value 1

failed (ignored, ret=134): /myproject/btrfs-progs/btrfs
inspect-internal dump-tree
/myproject/btrfs-progs/tests/fuzz-tests/images/
bko-155201-wrong-chunk-item-in-root-tree.raw.restored

test failed for case 004-simple-dump-tree
Makefile:288: recipe for target 'test-fuzz' failed
make: *** [test-fuzz] Error 1

So, check on num_stripes in print_chunk

Signed-off-by: Zhang Yu <zhangyu-fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Misono, Tomohiro cb39164f9d btrfs-progs: test: fix name generation not to contain trailing spaces
First patch causes test-convert fails.  This is because
generate_dataset() creates a name containing trailing spaces for
"slow_symlink" type, and cause getfacl error in convert_test_perm().
(This is not noticed since original run_check_stdout() throws away the
error.)

Fix this by use space for delimiter for cut.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Misono, Tomohiro 02e9bb9f23 btrfs-progs: test: fix run_check_stdout() call _fail()
run_check_stdout() uses "... | tee ... || _fail".  However, since tee
won't fail, _fail() is not called even if first command fails.

Fix this by checking PIPESTATUS in the end.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Qu Wenruo dbe96ecd3f btrfs-progs: tests: Add test case for mkfs --rootdir parameter
Add test case which checks if -r|--rootdir mkfs option can handle
symlink/char/block/fifo files.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Qu Wenruo 081e4e9bb8 btrfs-progs: mkfs: Fix wrong file type for dir items and indexes when specifying root directory
[Bug]
If using mkfs.btrfs with "-r" parameter and specified directory has
fifo/socket/char/block special file, then created filesystem can't pass
fsck:

------
checking fs roots
	unresolved ref dir 241158 index 3 namelen 9 name S.dirmngr filetype 0 errors 80, filetype mismatch
ERROR: errors found in fs roots
------

[Reason]
Btrfs dir items/indexes records inode type, while "-r" only handles
directories, regular files and symlink, it makes such special files type
to be regular file and caused the problem.

[Fix]
Add missing types for add_directory_items(), so that result of
"mkfs.btrfs -r" can pass mkfs.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Gu Jinxiang 944e14da14 btrfs-progs: mkfs: Move the tree root creation to own function
make_btrfs is too long to understand, make creatation of root tree
in a function.

Some of the tree roots are now created in a loop, where the code is just
copypasted. We now make use of the reference_root_table to translate
block index to root objectid.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
[ updated changelog ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba df4a04484a btrfs-progs: tests: missing device and slack space report
Verify that a missing device will not result in reporting a negative
value interpreted as 16EiB.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Patrik Lundquist 8c6b05d726 btrfs-progs: device usage: don't calculate slack on missing device
Print      Device slack:              0.00B
instead of Device slack:           16.00EiB

Signed-off-by: Patrik Lundquist <patrik.lundquist@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Misono, Tomohiro 88ef0b8397 btrfs-progs: inspect rootid: Allow a file to be specified
Since cmd_inspect_rootid() calls btrfs_open_dir(), it rejects a file to
be specified. But as the document says, a file should be supported.

This patch introduces btrfs_open_file_or_dir(), which is a counterpart
of btrfs_open_dir(), to safely check and open btrfs file or directory.
The original btrfs_open_dir() content is moved to btrfs_open() and shared
by both function.

Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba f47587d83d btrfs-progs: tests: convert misc/011-delete-missing-device to loopdevs
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba 83fe48c54b btrfs-progs: tests: convert misc/006-image-on-missing-device to loopdevs
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba 528a5bf6ad btrfs-progs: tests: move loopdev helpers out of the testcase to common
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba 3a6895b823 btrfs-progs: tests: cleanup loop device helpers
Make the loop device helpers a bit more generic before moving them to
the common helpers.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba 36db1080c3 btrfs-progs: print-tree: factor out extent_csum dump
Factor out code to own helper and tweak the format so it matches the
rest.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Josef Bacik 5fbc00cc73 btrfs-progs: print the csum length in debug-tree
While looking at a log of a corrupted fs I needed to verify we were
missing csums for a given range.  Make this easier by printing out the
range of bytes a csum item covers.

Signed-off-by: Josef Bacik <jbacik@fb.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
David Sterba 872837ebbf btrfs-progs: tests: add testcase for 'fi du' and empty subvol
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Goffredo Baroncelli b5665d66e8 btrfs-progs: fi du: don't call lookup_path_rootid for BTRFS_EMPTY_SUBVOL_DIR_OBJECTID
When ino is BTRFS_EMPTY_SUBVOL_DIR_OBJECTID, the item is not referred to
any file-tree. So lookup_path_rootid() doesn't return any meaningful
value.

As was reported, this can be triggered by

$ btrfs sub create test1
$ btrfs sub create test1/test2
$ btrfs sub snap test1 test1.snap
$ btrfs fi du -s test1
  Total   Exclusive  Set shared  Filename
  0.00B       0.00B       0.00B  test1
$ btrfs fi du -s test1.snap
  Total   Exclusive  Set shared  Filename
ERROR: cannot check space of 'test1.snap': Inappropriate ioctl for device

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00