Eventually, we'll have several lists and trees, as well as some statistics.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We have the infrastructure to cache extent buffers but we don't actually
do the caching. As soon as the last reference is dropped, the buffer
is dropped. This patch keeps the extent buffers around until the max
cache size is reached (defaults to 25% of memory) and then it drops
the last 10% of the LRU to free up cache space for reallocation. The
cache size is configurable (for use by e.g. lowmem) when the cache is
initialized.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
[ update codingstyle, switch total_memory to bytes ]
Signed-off-by: David Sterba <dsterba@suse.com>
We now have two data structures that can be used to iterate the same data
set, and there may be quite a few of them in memory. Eliminating the
list_head member will reduce memory consumption while iterating over
the extent backrefs.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
For the pathlogical case, like xfstests generic/297 that creates a
large file consisting of one, repeating reflinked extent, fsck can
take hours. The root cause is that calling find_data_backref while
iterating the extent records is an O(n^2) algorithm. For my
example test run, n was 2*2^20 and fsck was at 8 hours and counting.
This patch supplements the list with an rbtree and drops the runtime
of that testcase to about 20 seconds.
A previous version of this patch introduced a regression that would
have corrupted file systems during repair. It was traced to the
compare algorithm honoring ->bytes regardless of whether the
reference had been found and a failure to reinsert nodes after
the target reference was found.
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Mkfs uses make_path that is duplicate of path_cat* functions, so we can
switch to them and add the error handling.
Signed-off-by: David Sterba <dsterba@suse.com>
A convert parameter is added as a flag to indicate if btrfs_mksubvol()
is used for btrfs-convert. The change cascades down to the callchain.
Signed-off-by: Yingyi Luo <yingyil@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
link_subvol() is moved to inode.c and renamed as btrfs_mksubvol().
The change cascades down to the callchain.
Signed-off-by: Yingyi Luo <yingyil@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Fix 'subvolume delete --commit-after' to work properly:
- SYNC ioctl will be issued even when last delete fails
- SYNC ioctl will be issued on each file system only once in the end
To achieve this, get_fsid() and add_seen_fsid() are called after each
delete to keep only one fd for each fs.
In the end, seen_fsid_hash will be traversed and SYNC is issued on each
fs.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Change seen_fsid to hold fd and DIR in order to keep access to each fs.
This will be used for 'subvol delete --commit-after'.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Move is_seen_fsid()/add_seen_fsid()/free_seen_fsid() to common functions.
This will be used for 'subvol delete --commit-after'.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Make get_fsid() to a common functions.
This will be used for 'subvol delete --commit-after'.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Current code is reversed in --commit-after and --commit-each operation,
i.e. --commit-after means --commit-each actually. This patch fixes this
and also introduces enum type for more readable code.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Sparse does not seem to support gnu90 that we'd like to keep for the
regular build. Use gnu89 for C=1 build.
Signed-off-by: David Sterba <dsterba@suse.com>
Sparse complains that BTRFSCONVERT_EXT2 and _RESIERFS are not defined.
The per-target options are missing from the default CFLAGS and only
added in the specific rule.
Signed-off-by: David Sterba <dsterba@suse.com>
Sparse reports:
free-space-cache.c:921:13: warning: incorrect type in assignment (different base types)
free-space-cache.c:921:13: expected unsigned long long [unsigned] [usertype] ino
free-space-cache.c:921:13: got restricted __le64 [addressable] [usertype] objectid
we need to use the helper to read objectid, as read_eb_member just
copies the data without disk->cpu transformation. This could lead to
bogus results on bigendian machines.
Signed-off-by: David Sterba <dsterba@suse.com>
Sparse warns
print-tree.c:1261:49: warning: incorrect type in argument 4 (different base types)
print-tree.c:1261:49: expected unsigned long long [unsigned] [usertype] start
print-tree.c:1261:49: got restricted __le64 [addressable] [usertype] offset
which means we have to use the helper instead of directly accessing the offset
member. This could print bogus number on bigendian arch.
Signed-off-by: David Sterba <dsterba@suse.com>
Sparse warns that the DEBUG_*_ERROR macros are not defined when checked.
Use the ifdef check rather then relying on the implicit behaviour.
Signed-off-by: David Sterba <dsterba@suse.com>
BCP 78 applies to RFC 6234, but sha224-256.c is Simplified BSD.
This causes the following lintian error when building on Debian and
Debian derivatives:
E: btrfs-progs source: license-problem-non-free-RFC-BCP78
tests/sha224-256.c
Please consult the following email from debian-legal@lists.debian.org
for more information:
https://lists.debian.org/debian-legal/2017/08/msg00004.html
Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
[ I've copied too much from the RFC that's not related to the code,
covered by the explicit copyright notice in the file ]
Signed-off-by: David Sterba <dsterba@suse.com>
Start documenting the ioctl interface to btrfs. The overall structure
should be settled, the formatting of the ioctl description may change in
the future, newly added ioctl descriptions should follow the examples of
BTRFS_IOC_SUBVOL_CREATE.
The document is not finished yet and will not be installed until most of
ioctls' details are filled in.
Signed-off-by: David Sterba <dsterba@suse.com>
The test cli/007-check-force reports something like:
$ type -p '--string that starts with dashes'
bash: type: --: invalid option
Add the option/argument separator.
Signed-off-by: David Sterba <dsterba@suse.com>
free_block_group_cache() calls clear_extent_bits() with wrong end, which
is one byte larger than the correct range.
This will cause the next adjacent cache state to be split. And due to
the split, private pointer (which points to block group cache) will be
reset to NULL.
This is very hard to detect as this function only gets called in
cleanup_temp_chunks() which is just before mkfs finishes. This bug only
gets exposed when reworking --rootdir option.
Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Function find_next_chunk() is used to find next chunk start position,
which should only do search on chunk tree and objectid is set to
BTRFS_FIRST_CHUNK_TREE_OBJECTID.
So refactor the parameter list to get rid of @root, which should be
obtained from fs_info->chunk_root, and @objectid, which is set to
BTRFS_FIRST_CHUNK_TREE_OBJECTID.
Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
According to gcc(1), "-MD is equivalent to -M -MF file, except that -E is not
implied." Since the rule in the Makefile is just generating dependency file
and not building object file, it is no use to have "-MD" here. Also, it's
overridden and conflicting with the following "-MM" flag. I guess we can drop
it.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
We're missing several dependency files like:
$ diff -u <(find -name '*.o'|cut -d. -f2|sort) <(find -name '*.o.d'|cut -d. -f2|sort)
@@ -3,7 +3,6 @@
/btrfs-corrupt-block
/btrfs-debug-tree
/btrfs-find-root
-/btrfs-list
/btrfs-map-logical
/btrfs-select-super
/btrfstune
@@ -29,11 +28,6 @@
/cmds-scrub
/cmds-send
/cmds-subvolume
-/convert/common
-/convert/main
-/convert/source-ext2
-/convert/source-fs
-/convert/source-reiserfs
/ctree
/dir-item
/disk-io
<snip>
This is due to moving things out of objects and cmds_objects variables. Such
missing dependency files cause mis-building of some source files (try touch
utils.h; make mkfs/main.o).
This patch introduce a new variable "all_objects" to keep all the objects and
use the variable to generate proper dependency file building rules.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Add an objectid parameter to make the function a general one for
inserting root items and rename it to create_tree. The change cascades
down to the callchain.
Signed-off-by: yingyil <yingyil@google.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Clean up spaces and use only tabs. No functional change.
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Adds tests for the new features based on a prebuilt btrfs image with a
zstd compressed file.
Split from the previous patch.
Signed-off-by: Nick Terrell <terrelln@fb.com>
[ with some cleanups ]
Signed-off-by: David Sterba <dsterba@suse.com>
Adds zstd support to the btrfs program. An optional dependency on libzstd
>= 1.0.0 is added. Autoconf accepts `--enable-zstd' or `--disable-zstd' and
defaults to detecting if libzstd is present using `pkg-config'.
The patch is also available in my fork of btrfs-progs [1], which passes
Travis-CI with the new tests. The prebuilt binary is available there.
I haven't updated Android.mk.
[1] https://github.com/terrelln/btrfs-progs/tree/devel
Signed-off-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This patch updates btrfs-completion:
- add "filesystem du" and "rescure zero-log"
- restrict _btrfs_mnts to show btrfs type only
- add more completion in last case statements
(This file contains both spaces/tabs and may need cleanup.)
Signed-off-by: Tomohiro Misono <misono.tomohiro@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The sanity check in run_mustfail does not work as expected and allowed
the command to continue without the missing parameter
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=196893
Signed-off-by: David Sterba <dsterba@suse.com>
The function uses the reverse CRC32C table to quickly calculate a
4-byte suffix, that when added to the original data will make it
match desired checksum.
Author: Piotr Pawlow <pp@siedziba.pl>
[ minor adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
The table will be used to speed up calculations of CRC32C collisions.
Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
Function find_collision sometimes generated file names with non-printable
DEL characters (code 127), for example file name "|5gp!" would be changed
to "U'2<DEL>y" when using "crc-collisions" sanitize mode.
Author: Piotr Pawlow <pp@siedziba.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
Jeff Mahoney