The old check here tried to ensure that empty streams are not considered valid.
The old check however, will always fail when only one run through the while(1)
loop is needed and honor_end_cmd is set. So this:
btrfs send /some/subvol | btrfs receive -e /some/
will consistently fail because -e causes honor_cmd_to be set and
btrfs_read_and_process_send_stream() to correctly return 1. So the command will
be successful but btrfs receive will error out because the send - receive
concluded in one run through the while(1) loop.
If we want to exclude empty streams we need a way to tell the difference between
btrfs_read_and_process_send_stream() returning 1 because read_buf() did not
detect any data and read_and_process_cmd() returning 1 because honor_end_cmd was
set. Without introducing too many changes the best way to me seems to have
btrfs_read_and_process_send_stream() return -ENODATA in the first case. The rest
stays the same. We can then check for -ENODATA in do_receive() and report a
proper error in this case. This should also be backwards compatible to previous
versions of btrfs receive. They will fail on empty streams because a negative
value is returned. The only thing that they will lack is a nice error message.
Signed-off-by: Christian Brauner <christian.brauner@ubuntu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Preparatory work to support more global options. The current parser
abuses the subcommand table to understand help and version when
specified as options (--). These are now special case when processing
the global options.
Signed-off-by: David Sterba <dsterba@suse.com>
The easiest way to reproduce the error is to try to build
btrfs-progs with
$ make LDFLAGS=-Wl,--no-undefined
btrfs-list.o: In function `lookup_ino_path':
btrfs-list.c:(.text+0x7d2): undefined reference to `__error'
Noticed by Denis Descheneaux when snapper tool
stopped working after upgrade to btrfs-progs-4.10.
As soname didn't change in 4.9 -> 4.10 release
I assume it's just an object file omission
in library depends and not the API/ABI change
of the library error printing.
Cc: linux-btrfs@vger.kernel.org
Cc: Mike Gilbert <floppym@gentoo.org>
Reported-by: Denis Descheneaux
Bug: https://bugs.gentoo.org/show_bug.cgi?id=613890
Signed-off-by: Sergei Trofimovich <slyfox@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new image, which contains external SHARED_DATA_REF items to
trigger a lowmem mode false alert.
The image only contains external SHARED_DATA_REF and no inlined data
backref.
Before the image, we only have inlined shared data ref, which is not
enough to trigger lowmem mode false alert.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In check_extent_data_item(), after checking extent item of one data
extent, we search inlined data backref, then EXTENT_DATA_REF_KEY.
But we didn't search SHARED_DATA_REF, so if the backref is
SHARED_DATA_REF, then we will raise a false alert about backref lost.
Fix by also checking SHARED_DATA_REF_KEY in check_extent_data_item().
Reported-by: Chris Murphy <chris@colorremedies.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In check_convert_image(), for normal HOLE case, if the file extents are
smaller than image size, we set ret to -EINVAL and print error message.
But forget to return.
This patch adds the missing return to fix it.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The implementation of message helpers use very generic names so we
should at least use a prefix, as they're going to be usied from within
the library. The build fix will follow.
Reported-by: Mike Gilbert <floppym@gentoo.org>
Signed-off-by: David Sterba <dsterba@suse.com>
Explicitly include all headers exported to library, as a light build
check that the files are really present.
Signed-off-by: David Sterba <dsterba@suse.com>
Since btrfs_reserved_ranges array is just used to store btrfs reserved
ranges, no one will nor should modify them at run time, make them static
and const will be better.
This also eliminates the use of immediate number 3.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ definition stays in source-fs.c ]
Signed-off-by: David Sterba <dsterba@suse.com>
In latest linux api headers, __bitwise is already defined in
/usr/include/linux/types.h.
So kerncompat.h will re-define __bitwise, and cause gcc warning.
Fix it by checking if __bitwise is already define.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In btrfs-dump-tree, we output any existing log tree, however we don't
output the log root tree, which records all root items for log trees.
This makes it confusing for any one who want to know where the log tree
comes from.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
When using -t option to output trees not in root tree (chunk/root/log
root), then we output the tree twice.
Fix it
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In order to drop dependency on SSL library to compute MD5 in fssum,
we'll use the reference implementation from RFC 6234.
The checksum is not in a cryptographically sensitive context, but we're
going to skip MD5 and SHA-1 anyway.
Signed-off-by: David Sterba <dsterba@suse.com>
In file included from /usr/include/stdio.h:27:0,
from kerncompat.h:22,
from tests/fssum.c:25:
/usr/include/features.h:148:3: warning: #warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE" [-Wcpp]
# warning "_BSD_SOURCE and _SVID_SOURCE are deprecated, use _DEFAULT_SOURCE"
^~~~~~~
We've solved that long time ago and config.h now provides the macros.
Signed-off-by: David Sterba <dsterba@suse.com>
The function htonll is not provided by the standard library and we can
replace it by our cpu-to-XX helpers. This switches the endianity of the
checksummed value to LE, but this is not a problem.
Signed-off-by: David Sterba <dsterba@suse.com>
Copy from fstests, originally from
git://git.kernel.org/pub/scm/linux/kernel/git/arne/far-progs.git
Needs libcrypto to link but this check is now missing in configure.
Signed-off-by: Jan Schmidt <list.btrfs@jan-o-sch.net>
Signed-off-by: David Sterba <dsterba@suse.com>
Nearly each use of IMAGE can be replaced by common helpers as there are
no specific requirements on the testing filesystem. There are still a
few left that need to be evaluated and converted eventually.
Signed-off-by: David Sterba <dsterba@suse.com>
We need to build outside of the topdir so we can use the "btrfs/" prefix
for includes and not accidentally include other files.
Make magic is simple:
- build dependencies inside TOPDIR
- build inside temporary directory, link back to TOPDIR
- library-test.o not built anymore obviously
Signed-off-by: David Sterba <dsterba@suse.com>
The library-test is supposed to compile and link as an external
program, so we should update the way the headers are included.
Signed-off-by: David Sterba <dsterba@suse.com>
The patch "btrfs-progs: Introduce kernel sizes to cleanup large
intermediate number" (a2203246ae) was taken from kernel but not
properly ported so the build breaks because the header linux/sizes.h is
not exported.
The build tests of library do not cover the case when the macro
BTRFS_FLAT_INCLUDES is not defined (ie. an external build).
Reported-by: Holger Hoffstätte <holger@applied-asynchrony.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Build under musl libc fails because of missing PATH_MAX and XATTR_NAME_MAX
macro declarations. Add the required headers.
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Rework rollback to a more easy to understand way.
New convert behavior makes us to have a more flex chunk layout, which
only data chunk containing old fs data will be at the same physical
location, while new chunks (data/meta/sys) can be mapped anywhere else.
This behavior makes old rollback behavior can't handle it.
As old behavior assumes all data/meta is mapped in a large chunk, which is
mapped 1:1 on disk.
So rework rollback to handle new convert behavior, enhance the check by
only checking all file extents of convert image, only to check if these
file extents and therir chunks are mapped 1:1.
This new rollback check behavior can handle both new and old convert
behavior, as the new behavior is a superset of old behavior.
Further more, introduce a simple rollback mechanisim:
1) Read reserved data (offset = file offset) from convert image
2) Write reserved data into disk (offset = physical offset)
Since old fs image is a valid fs, and we only need to rollback
superblocks (btrfs reserved ranges), then we just read out data in
reserved range, and write it back.
Due to the fact that all other file extents of converted image is mapped
1:1 on disk, we put the missing piece back, then the fs is as good as
old one.
Then what we do in btrfs is just another dream.
With this new rollback mechanisim, we can open btrfs read-only, so we
won't cause any damage to current btrfs, until the final piece (0~1M,
containing 1st super block) is put back.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ port to v4.10 ]
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a function, check_convert_image() to check if that image is
rollback-able.
This means all file extents except one of the btrfs reserved ranges, must
be mapped 1:1 on disk.
1:1 mapped file extents must match the following conditions:
1) Their file_offset(key.offset) matches its disk_bytenr
2) The corresponding chunk must be mapped 1:1 on disk
That's to say, it's a SINGLE chunk, and chunk logical matches with
stripe physical.
Above 2 conditions ensured that file extent lies the exactly the same
position as in the old filesystem.
For data in reserved ranges of btrfs, they are relocated to new places,
and in that case, we use btrfs_read_file() to read out the content for
later rollback use.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Introduce a new function, read_reserved_ranges(), to allow later
rollback to use these data to do rollback.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Introduce a new function, btrfs_read_file(), to read out data of a file
inode.
This function will iterate through EXTENT_DATA items and handle
inline/prealloc/hole file extents.
Compression is not supported yet.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Since we have reserved ranges array now, we can use them to skip all
these open codes.
And add some comment and asciidoc art for related part.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ port to v4.10 ]
Signed-off-by: David Sterba <dsterba@suse.com>
Introduce a new strucutre, simple_range, to present one contingous
range.
Also, use such structure to define btrfs_reserved_ranges(), which
convert and rollback will use.
Suggested-by: David Sterba <dsterba@suse.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ split hunks to new file structure ]
Signed-off-by: David Sterba <dsterba@suse.com>
Convert is now a little complex due to that fact we need to separate
metadata and data chunks for different profiles.
Add a comment with ascii art explaining the whole design and point
out the really complex part, so any newcomers interested in convert can
get a quick overview of it before digging into the hard to read code.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ wording and formatting adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
clone needs to resolve the paths of the involved subvolumes in the target
fs from their UUIDs. When doing so it might need to strip the prefix
that is mounted as the root of the fs from those paths.
It didn't do so correctly when processing the source of "clone" commands
This is a regression test for
btrfs-progs: receive: handle root subvol path in clone
Signed-off-by: Benedikt Morbach <benedikt.morbach@googlemail.com>
[ copied the fstests version, will be updated later ]
Signed-off-by: David Sterba <dsterba@suse.com>
testcase:
# ro subvol /src/parent
# rw subvol /src/foo
clone /src/parent/file /src/foo/file
subvol snapshot -r /src/foo /src/foo.snap
# generates a "clone parent/file -> foo.snap/file" send command
send -p /src/parent /src/foo.snap
# target fs:
# dest/
# |--- parent/...
# mounted with -o subvol=dest, such that "parent" is at <target>/parent
receive <target>
result:
ERROR: cannot open dest/parent/file: No such file or directory
expected:
"dest/" get's stripped from the clone source path to get the actual
path in the target fs, if reachable from the mount point/chroot.
This is exactly what process_snapshot does, which gets called on
_every_ incremental receive and I'm quite certain is correct in
doing so
Signed-off-by: Benedikt Morbach <benedikt.morbach@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Two fixes:
1)
Check that the parent subvol actually is reachable via our root path.
The previous code wouldn't catch
parent subvol: foo/bar
root path: bar (i.e. mounted with -o subvol=bar)
where the parent isn't reachable from the root path.
(but the original "strstr(parent, root_path) == NULL" check still doesn't hold)
Also check for the slash after "root_path", i.e. throw an error on
parent subvol: foobar
root path: foo
2)
If the parent subvol is the one that is mounted we obviously can't
receive into it, as it has to be read-only by definition.
We'd get a rather cryptic:
At subvol /tmp/test/dest.snap
At snapshot dest.snap
ERROR: creating snapshot / -> dest.snap failed: Invalid cross-device link
(not sure what it says if "/" isn't even a btrfs)
But with this we get
At subvol /tmp/test/dest.snap
At snapshot dest.snap
ERROR: creating snapshot . -> dest.snap failed: Read-only file system
which is both more helpful and more correct.
Signed-off-by: Benedikt Morbach <benedikt.morbach@googlemail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Christian Brauner