In v4.8-rc1, gcc 5.3.1 gives following warning. Fixed it.
[CC] btrfs-image.o
btrfs-image.c: In function 'flush_pending':
btrfs-image.c:708:17: warning: 'start' may be used uninitialized in this function [-Wmaybe-uninitialized]
header->bytenr = cpu_to_le64(start);
^
btrfs-image.c:927:6: note: 'start' was declared here
u64 start;
^
Signed-off-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
If an extent is found to have length 0, we'd loop endlessly in
copy_from_extent_tree. Reproduced by fuzzed image
bko-166361-blocksize-zero.raw within test 002-simple-image .
Signed-off-by: David Sterba <dsterba@suse.com>
Replace old and not so informal srand()/rand() calls to new random
number API.
Including btrfs-corrupt-block(main user), btrfs-image and btrfs-crc.
Some tests like dir-test/random-test/quick-test is not modified.
As random-test itself can't even pass build.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Previously btrfs-image restore would set the chunk items to have 1 stripe,
even if the chunk is dup. If you use btrfsck on the restored file system,
some dev_extent will not find any relative chunk stripe, and the
bytes-used of dev_item will not equal to the dev_extents's total_bytes.
This patch store a additional physical just for the dup case when build
the in-memory chunk-tree.
Currently btrfsck on the restored file system, only single and dup is no
problem. raid* support should be added in the future.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Nodesize is used in kernel, the values are always equal. We have to keep
leafsize in headers, similarly the tree setting functions still take and
set leafsize, but it's effectively a no-op.
Signed-off-by: David Sterba <dsterba@suse.com>
Add new btrfsck option, '--chunk-root', to specify chunk root bytenr.
And allow open_ctree_fs_info() function accept chunk_root_bytenr to
override the bytenr in superblock. This will be mainly used when chunk
tree corruption.
Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In update_super() 'chunk->stripe.devid' and 'super->dev_item.devid' both
are little endian. So we should not use endian helper
btrfs_set_stack_stripe_devid().
Signed-off-by: Byongho Lee <bhlee.kernel@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Comparing unsigned type for <= 0 does not make much sense, we should
really check the signed value returned by sysconf.
Resolves-coverity-id: 1324536
Signed-off-by: David Sterba <dsterba@suse.com>
This patch is generated from a coccinelle semantic patch:
identifier t;
expression e;
statement s;
@@
-t = malloc(e);
+t = calloc(1, e);
(
if (!t) s
|
if (t == NULL) s
|
)
-memset(t, 0, e);
Signed-off-by: Silvio Fricke <silvio.fricke@gmail.com>
[squashed patches into one]
Signed-off-by: David Sterba <dsterba@suse.com>
Devices opened by check_mounted() in main() should closed before
the exit of btrfs-image.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In case of creating an image without compression, we don't need to
create unused threads.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Bug:
# btrfs-image -t0 -c9 /dev/sda6 /tmp/btrfs_image.img
(hang)
# btrfs-image -r -t0 /tmp/btrfs_image.img /dev/sda6
(hang)
Reason:
The program need to create at least 1 thread for
compression/decompression. If the user specifies -t0, it overwrites the
default value of 1, then the program really creates 0 thread, and
ends up looping
Fix:
We can add a check, to make the program not to allow -t0 argument,
but there is another problem:
For example, on a node with 4 cpus:
btrfs-image -c9 -t1: 4 threads (1 means use NR_CPUS)
-c9 -t2: 2 threads
-c9 -t3: 3 threads
...
(-t1 has more threads than -t2 and -t3)
So we change to use value of 0 as "use NR_CPUS threads", then:
btrfs-image [no -t arg]: use NR_CPUS threads
-t0: use NR_CPUS threads
-t val: use val threads.
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In update_disk_super_on_device,the buffer size is small, allocate it
onstack and get rid of one error path.
Signed-off-by: David Sterba <dsterba@suse.com>
Function read_data_extent() in btrfs-image.c is using manual-and-read
codes.
Replace it with existing read_extent_data() to reduce duplication.
Also, now we can use other mirror to try our best to do the dump, just
like read_tree_block().
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Offline btrfs tools, like btrfs-image, will infinitely loop when there
is missing device.
The reason is, for missing device, it's fd will be set to -1, but before
we reading, we only check the fd validation by checking if it's 0.
So in that case, -1 will pass the validation check, and cause pread to
return 0, and loop to read.
Just change the validation check from "== 0" to "<= 0" to avoid such
problem.
Reported-by: Timothy Normand Miller <theosib@gmail.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When we go to fixup the dev items after a restore we scan all existing devices.
If you happen to be a btrfs developer you could possibly open up some random
device that you didn't just restore onto, which gives you weird errors and makes
you super cranky and waste a day trying to figure out what is failing. This
will make it so that we use the fd we've already opened for opening our ctree.
Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
When we restore a multi disk image onto a single disk we need to update the dev
items used and total bytes so that fsck doesn't freak out and that we get normal
results from stuff like btrfs fi show. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
The METADUMP super flag makes us skip doing the chunk tree reading which isn't
helpful for the new restore since we have a valid chunk tree. But we still want
to have a way for the kernel to know that this is a metadump restore so it
doesn't do things like verify data checksums. We also want to skip some of the
device extent checks in fsck since those will obviously not match. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
For some reason we only allow btrfs-image restore to have one thread, which is
incredibly slow with large images. So allow us to do work with more than just
one thread. This made my restore go from 16 minutes to 3 minutes. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
If you create a metadump from a striped volume you will have chunks that refer
to different logical offsets with the same physical offset on different devices.
So when we do the restore we just truncate the number of stripes in each chunk
item and carry on, which causes problems because we then have chunks that point
to the same physical offset for different logical offsets. To handle this
problem we keep track of logical extents that overlap on physical extents.
Then we go back and remap these extents into different physical extents on the
disk we are restoring onto. This makes us actually able to restore a multi disk
image onto a single disk and have everything work out properly. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
When btrfs-image makes a metadump it'll map all the blocks from their logical
address to their physical. This works out fine with the exception of the super
block, which is the physical offset. Normally this just works, but if the user
has balanced their fs it'll either crash btrfs-image or it'll copy some
completely arbitrary data. This forces btrfs-image to read the super directly
from the disk. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Allow read_tree_block() and read_node_slot() to return error pointer.
This should help caller to get more specified error number.
For existing callers, change (!eb) judgmentt to
(!extent_buffer_uptodate(eb)) to keep the compatibility, and for caller
missing the check, use PTR_ERR(eb) if possible.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
- use standard PACKAGE_{NAME,VERSION,STRING,URL,...} autoconf macros
rather than homemade BTRFS_BUILD_VERSION
- don't #include version.h, now the file is necessary for library API only
Note that "btrfs version" returns "btrfs-progs <version>" instead of
the original confusing "btrfs <version>".
Signed-off-by: Karel Zak <kzak@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
glibc 2.10+ (5+ years old) enables all the desired features:
_XOPEN_SOURCE 700, __XOPEN2K8, POSIX_C_SOURCE, DEFAULT_SOURCE; with a
single _GNU_SOURCE define in the makefile alone. For portability to
other libc implementations (e.g. dietlibc) _XOPEN_SOURCE=700 is also
defined.
This also resolves Debian bug report filed by Michael Tautschnig -
"Inconsistent use of _XOPEN_SOURCE results in conflicting
declarations". Whilst I was not able to reproduce the results, the
reported fact is that _XOPEN_SOURCE set to 500 in one set of files
(e.g. cmds-filesystem.c) generates/defines different struct stat from
other files (cmds-replace.c).
This patch thus cleans up all feature defines, and sets them at a
consistent level.
Bug-Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=747969
Signed-off-by: Dimitri John Ledkov <dimitri.j.ledkov@intel.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Add explain on 'source' and 'target', which is somewhat confusing for
users who want to restore dumped image.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[reworded texts]
Signed-off-by: David Sterba <dsterba@suse.cz>
When btrfs-image run on a mounted filesystem,
the undergoing fs operations may change what you have imaged a while ago.
In this case, give a warning to remind the user that he may not
get a consistent image he wants.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
[tweaked the message]
Signed-off-by: David Sterba <dsterba@suse.cz>
The btrfs-image requires at least 2 args to run,
one for the source dev/file, the other for the target dev/file.
This patch depends on patch:
btrfs-progs: move the check_argc_* functions into utils.c
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Don't bother free the buffer if the malloc failed.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Steps to reproduce:
# mkfs.btrfs -f <dev1>
# btrfs-image <dev1> <image_file>
# btrfs-image -r -o <image_file> <dev2>
# btrfs check <dev2>
btrfs check output:
: read block failed check_tree_block
: Couldn't read tree root
: Couldn't open file system
The btrfs-image should not mess with the chunk tree under the old_restore way.
The new restore way was introduced by:
commit d6f7e3da0d
Btrfs-progs: make btrfs-image restore with a valid chunk tree V2
...
And the following commit enhanced the new restore on the valid chunk tree
building stuff:
commit ef2a8889ef
Btrfs-progs: make image restore with the original device offsets
...
But the second commit should not effect the old_restore way since the
old_restore way doesn't try to build a valid chunk tree at all.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
If the malloc above fails, the btrfs-image will exit directly
without any error messages.
Now just return the ENOMEM errno and let the caller prompt the
error message.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Handle the malloc failure for dump_worker in the same way as
the restore worker.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
When btrfs-image failed to create an image, the invalid output file
had better be deleted to prevent being used mistakenly in the future.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
For btrfs-image,
dump may not come with option '-o'
-r may not come with option '-c', '-s', '-w', dev_cnt != 1
-m may not come with dev_cnt < 2
All of the above should be regarded as invalid combinations,
and the usage will show up.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The btrfs-image support multiple devices with -m specified.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
The value of variable leaf in while loop don't have to be set
for every round. Just move it outside.
Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Tsutomu Itoh