Commit Graph

352 Commits (master)

Author SHA1 Message Date
David Sterba 890f4a61d7 btrfs-progs: cleanup, kill trivial btrfs_key_type helper
Signed-off-by: David Sterba <dsterba@suse.com>
2016-10-03 11:33:15 +02:00
David Sterba 555743075b btrfs-progs: cleanup, kill trivial btrfs_set_key_type helper
Signed-off-by: David Sterba <dsterba@suse.com>
2016-10-03 11:33:15 +02:00
Omar Sandoval 02a9b6cbef Revert "btrfs-progs: fix compat_ro mask for free space tree"
btrfs-progs can't mount space_cache=v2 filesystems read-write, which is
why the compat bit wasn't added to the supported mask in the first
place. Remove it.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-09-21 11:50:09 +02:00
David Sterba d57187703e btrfs-progs: add rest of stack operations for btrfs_dir_item
Signed-off-by: David Sterba <dsterba@suse.com>
2016-08-25 19:02:18 +02:00
David Sterba 2ff04db5f4 btrfs-progs: two staged filesystem creation
The filesystem existence on a device is manifested by the signature,
during the mkfs process we write it first and then create other
structures. Such filesystem is not valid and should not be registered
during device scan nor listed among devices from blkid.

This patch will introduce two staged creation. In the first phase, the
signature is wrong, but recognized as a partially created filesystem (by
open or scan helpers). Once we successfully create and write everything,
we fixup the signature. At this point automated scanning should find
a valid filesystem on all devices.

We can also rely on the partially created filesystem to do better error
handling during creation. We can just bail out and do not need to clean
up.

The partial signature is '!BHRfS_M', can be shown by

  btrfs inspect-internal dump-super -F image

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-08-24 14:37:31 +02:00
David Sterba 3c350dec06 btrfs-progs: introduce signature for a partially set up filesystem
Currently the superblock is created first, with a valid signaure, but
the rest of the filesystem is missing. When the creation process is
interrupted, the filesystem still might be considered as valid.

To prevent that, create the filesytem with an invalid signature that
would be still recognized during the mkfs process, and finalize at the
end.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-08-24 14:36:58 +02:00
Qu Wenruo bdadea75e4 btrfs-progs: convert: Fix a regression that ext2_save/image is not readonly
The new convert treats the convert image as a normal file, without any
special flags and permissions.

This is different from original code:
1) Permission changed from 0400 to 0600
2) Inode lacks READONLY flag

This makes we can read-write mount the ext2 image and cause rollback
failure.

Follow old code behavior, use 0400 permission and add back READONLY
flag to fix it.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-08-19 13:52:43 +02:00
Lu Fengqi b0d360b541 btrfs-progs: check: introduce function to check data backref in extent tree
Introduce a new function check_data_extent_item() to check if the
corresponding data backref exists in extent tree.

Signed-off-by: Lu Fengqi <lufq.fnst@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Reviewed-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-08-17 18:51:19 +02:00
Mark Fasheh a804254361 btrfs-progs: check: write corrected qgroup info to disk
Now that we can verify all qgroups, we can write the corrected qgroups out
to disk when '--repair' is specified. The qgroup status item is also updated
to clear any out-of-date state. The repair_ functions were modeled after the
inode repair code in cmds-check.c.

I also renamed the 'scan' member of qgroup_status_item to 'rescan' in order
to keep consistency with the kernel.

Testing this was easy, I just reproduced qgroup inconsistencies via the
usual routes and had btrfsck fix them.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-07-13 18:44:26 +02:00
Qu Wenruo cfdcb4fa61 btrfs-progs: Cleanup old btrfs-convert
Cleanup all the old btrfs-convert facilities, including:
1) btrfs_convert_operations->alloc/free/test_extents*
   No need to do non-standard extent allocation.
   After init_btrfs() everything can be done by normal routine.

   Now only 4 functions are needed in btrfs_convert_operations.
   1) open_fs
   2) read_used_space
   3) copy_inodes
   4) close_fs

2) fs_info->extent_ops
   Same as above.

3) Old init_btrfs(), create_image(), create_file_image_range()
   Replaced with newer and cleaner one.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-06-07 18:15:19 +02:00
Qu Wenruo 561f773433 btrfs-progs: convert: Strictly avoid meta or system chunk allocation
Before this patch, btrfs-convert only rely on large enough initial
system/metadata chunk size to ensure no newer system/meta chunk will be
created.

But that's not safe enough. So add two new members in fs_info,
avoid_sys/meta_chunk_alloc flags to prevent any newer system or meta
chunks to be created before init_btrfs_v2().

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-06-07 18:15:19 +02:00
Qu Wenruo a21cc1ca3f btrfs-progs: extent-tree: Enhance btrfs_record_file_extent
Btrfs_record_file_extent() has some small problems like:
1) Can't handle overlapping extents
2) May create extent larger than BTRFS_MAX_EXTENT_SIZE

So enhance it using previously added facilites.
This is used for later btrfs-convert, as for new convert, we create
saved image first, then copy inode.
Which will also cause extent overlapping.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-06-07 18:15:19 +02:00
Qu Wenruo f5e77e4c52 btrfs-progs: extent-tree: Introduce function to find the first overlapping extent
Introduce a new function, btrfs_search_overlap_extent() to find the first
overlapping extent.

It's useful for later btrfs-convert rework.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-06-07 18:15:19 +02:00
Nicholas D Steeves bd2cc320af btrfs-progs: typo review of strings and comments
Signed-off-by: Nicholas D Steeves <nsteeves@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-06-01 14:56:56 +02:00
David Sterba 64a4800875 btrfs-progs: add getters for ioctl search_header
The search header is usually accessed in an unaligned way, we could
trigger errors (SIGBUS) on architectures that do not support that.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-05-11 15:53:16 +02:00
David Sterba 3be6e3e7c9 btrfs-progs: deprecate and stop using btrfs_level_size
Size of a b-tree node is always nodesize, regardless of the level.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-05-02 14:40:23 +02:00
David Sterba 2a796d84af btrfs-progs: replace leafsize with nodesize
Nodesize is used in kernel, the values are always equal. We have to keep
leafsize in headers, similarly the tree setting functions still take and
set leafsize, but it's effectively a no-op.

Signed-off-by: David Sterba <dsterba@suse.com>
2016-05-02 14:40:18 +02:00
Qu Wenruo 43318324d2 btrfs-progs: Allow open_ctree to return fs_info even chunk tree is corrupted
Current open_ctree_fs_info() won't return anything if chunk tree root is
corrupted.
This makes some function, like btrfs-find-root, unable to find any older
chunk tree root, even it is possible to use system_chunk_array in super
block.

And at least two users in mail list has reported such heavily chunk
corruption.
Although we have 'btrfs rescue chunk-recovery' but it's too time
consuming and sometimes not able to cope with a specific filesystem
corruption.

This patch adds a new open ctree flag,
OPEN_CTREE_IGNORE_CHUNK_TREE_ERROR, allowing fs_info to be returned from
open_ctree_fs_info() even there is no valid tree root in it.

Also adds a new close_ctree() variant, close_ctree_fs_info() to handle
possible fs_info without any root.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ adjusted error messages ]
Signed-off-by: David Sterba <dsterba@suse.com>
2016-02-26 17:27:58 +01:00
David Sterba d30b0be836 btrfs-progs: fix compat_ro mask for free space tree
The BTRFS_FEATURE_COMPAT_RO_FREE_SPACE_TREE bit is supposed to be in the
COMPAT_RO_SUPP bitmask.

Reported-by: Holger Hoffstätte <holger.hoffstaette@googlemail.com>
Reported-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-01-25 15:35:30 +01:00
Omar Sandoval e4797df6a9 btrfs-progs: check the free space tree in btrfsck
This reuses the existing code for checking the free space cache, we just
need to load the free space tree. While we do that, we check a couple of
invariants on the free space tree itself. This requires pulling in some
code from the kernel to exclude the super stripes.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-01-12 15:02:54 +01:00
Omar Sandoval d8c537e71f btrfs-progs: add basic awareness of the free space tree
To start, let's tell btrfs-progs to read the free space root and how to
print the on-disk format of the free space tree. However, we're not
adding the FREE_SPACE_TREE read-only compat bit to the set of supported
bits because progs doesn't know how to keep the free space tree
consistent.

Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2016-01-12 15:02:54 +01:00
Zhao Lei f248b10786 btrfs-progs: Avoid uninitialized data in output of btrfs-convert
The sequence, transid and reserved fields of inode were writen to disk
with uninitizlized value, this patch fixes it.

Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2015-09-09 17:43:05 +02:00
Zhao Lei 537fb08026 btrfs-progs: fix typo in set/get of btrfs_inode_item::sequence
s/generation/sequence/
for BTRFS_SETGET_STACK_FUNCS(stack_inode_sequence, ...)

Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2015-09-09 17:42:22 +02:00
Qu Wenruo c11eeecd78 btrfs-progs: convert: Avoid allocating metadata extent crossing stripe boundary
As convert implement its own alloc extent, avoid such metadata problem
too.

Reported-by: Chris Murphy <lists@colorremedies.com>
Reported-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: Zhao Lei <zhaolei@cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2015-08-31 19:25:10 +02:00
Filipe Manana a57606e815 Btrfs-progs: add feature to get mininum size for resizing a fs/device
Currently there is not way for a user to know what is the minimum size a
device of a btrfs filesystem can be resized to. Sometimes the value of
total allocated space (sum of all allocated chunks/device extents), which
can be parsed from 'btrfs filesystem show' and 'btrfs filesystem usage',
works as the minimum size, but sometimes it does not, namely when device
extents have to relocated to holes (unallocated space) within the new
size of the device (the total allocated space sum).

This change adds the ability to reliably compute such minimum value and
extents 'btrfs filesystem resize' with the following syntax to get such
value:

   btrfs filesystem resize [devid:]get_min_size

Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2015-08-31 19:25:03 +02:00
Qu Wenruo 80fc1e0fc6 btrfs-progs: extent-tree: Introduce btrfs_free_block_group function
This function will be used to free a empty chunk.

This provides the basis for later temp chunk cleanup.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2015-07-10 14:39:39 +02:00
Qu Wenruo 20ce7d3ee6 btrfs-progs: Remove non-exist csum size.
Current btrfs only support CRC32 as checksum algorithm.
But in btrfs_csum_sizes array, we have an extra 0 at tail, causing
csum_type 1 can still be considered as supported csum type.

Fix it by removing the tailing 0.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-05-14 15:41:08 +02:00
Qu Wenruo 0c8b978881 btrfs-progs: Introduce change_header_uuid function
This function is used to change fsid and chunk_tree_uuid of a node/leaf.
The function does it without transaction protection.

This is the basis of offline uuid change.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-05-14 15:41:07 +02:00
Qu Wenruo 2259f03439 btrfs-progs: Add open_ctree check for uuid changing
Now open_ctree will exit if it found the superblock is marked
CHANGING_FSID, except given IGNORE_FSID open ctree flags.

Kernel will do the same thing later.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[removed the chunk tree flag, reworded the error message]
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-05-14 15:41:07 +02:00
Qu Wenruo 7cc792872a btrfs-progs: Add CHANGING_FSID super flag
Add the super flag to inform kernel not to mount a filesystem wich fsid
change is in progress.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[removed the chunk tree flag]
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-05-14 15:41:07 +02:00
Josef Bacik 2681e00f00 btrfs-progs: check for matching free space in cache
We have this check in the kernel but not in userspace, which makes fsck
fail when we wouldn't have a problem in the kernel.  This was meant to
catch this case because it really isn't good, unfortunately it will
require a design change to fix in the kernel so in the meantime add this
check so we can be sure our tests only catch real problems.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-04-24 15:42:04 +02:00
Qu Wenruo 0f67b5aa01 btrfs-progs: Add stack get/set functions for btrfs_qgroup_status_item
This provides the basis for later qgroup related changes.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-03-24 00:28:34 +01:00
Qu Wenruo 9922c42701 btrfs-progs: Update qgroup status flags and replace qgroup level/subvid calculation with inline function
Ctree.h of btrfs-progs contains wrong flags for btrfs_qgroup_status.
Update it with the one in kernel.

Also, introduce the inline function btrfs_qgroup_(level/subvid) to get
the level/subvolid of qgroup, to replace the old open-coded bit
operations.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-03-24 00:07:59 +01:00
Qu Wenruo 0c05f623b1 btrfs-progs: Add support to suppress tree block csum error output
Add new open ctree flag OPEN_CTREE_SUPPRESS_CHECK_BLOCK_ERRORS to
suppress tree block csum error output.

Provides the basis for new btrfs-find-root and other enhancement on
btrfs offline tools output.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[renamed vars and funcs, added comments]
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-02-11 17:14:05 +01:00
Josef Bacik 08a4597223 Btrfs-progs: remove global transaction from fsck
We hold a transaction open for the entirety of fixing extent refs.  This works
out ok most of the time but we can be tight on space and run out of space when
fixing things.  To get around this just push down the transaction starting dance
into the functions that actually fix things.  This keeps us from ending up with
ENOSPC because we pinned everything and allows the code to be a bit simpler.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
2015-02-09 14:53:15 -05:00
Josef Bacik 797a937e5d Btrfs-progs: Introduce metadump_v2
The METADUMP super flag makes us skip doing the chunk tree reading which isn't
helpful for the new restore since we have a valid chunk tree.  But we still want
to have a way for the kernel to know that this is a metadump restore so it
doesn't do things like verify data checksums.  We also want to skip some of the
device extent checks in fsck since those will obviously not match.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
2015-02-09 14:53:15 -05:00
Qu Wenruo aae2c8421b btrfs-progs: Add btrfs_get_extent() and btrfs_punch_hole().
Add btrfs_get_extent() and btrfs_punch_hole() for btrfs-progs.

Btrfs_get_extent() will act much like kernel one, return the first
extent that covers the given range.
The difference will be that progs btrfs_get_extent() can't handle
no-holes feature, which means caller should handle it carefully.

Btrfs_punch_hole() will punch a hole in given range of given inode,
however it differs from kernel one since it won't zero any page or drop
any extents if there is any extent in the hole range.

These functions are mainly used for later I_ERR_FILE_EXTENT_DISCOUNT
repair function.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-02-02 14:16:38 +01:00
Qu Wenruo 0ddf63c09f btrfs-progs: Record orphan data extent ref to corresponding root.
Before this patch, when a extent's data ref points to a invalid key in
fs tree, this happens if a leaf/node of fs tree is corrupted, btrfsck
can't do any repair and just exit.

In fact, such problem can be handled in fs tree repair routines, rebuild
the inode item(if missing) and add back the extent data (with some
assumption).

So this patch records such data extent refs for later fs tree recovery
routine.

TODO:
    Restore orphan data extent refs into btrfs_root is not the best
    method. It's best to directly restore it into inode_record, however
    current extent tree and fs tree can't cooperate together, so use
    btrfs_root as a temporary storage until inode_cache is built.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2015-02-02 14:16:38 +01:00
David Sterba 07ce7005fc btrfs-progs: unify header file inclusion protections
There are missing ifdefs or defines with very generic names.

Signed-off-by: David Sterba <dsterba@suse.cz>
2015-01-21 17:49:26 +01:00
Qu Wenruo 55fea7186c btrfs-progs: Add inode item rebuild function.
Add a basic inode item rebuild function for I_ERR_NO_INODE_ITEM.
The main use case is to repair btrfs which fs root has corrupted leaf,
but it is already working for case if the corrupteed fs root leaf/node
contains no inode extent_data.

The repair needs 3 elements for inode rebuild:
1. inode number
   This is quite easy, existing inode_record codes will detect it quite
   well.

2. inode type
   This is the trick part. The only reliable method is to recovery it from
   parent's dir_index/item.
   The remaining method will search for regular file extent for FILE
   type or child's backref for DIR(todo).
   Fallback will be FILE.

Inode name(inode_ref) will be recoverd by nlink repair function.

This is just a fundamental implement, some advanced recovery can be
improved later with btrfs-progs infrastructure change.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:52:28 +01:00
Qu Wenruo 43c36f3cfd btrfs-progs: Add btrfs_mkdir() function for the incoming 'lost+found' fsck mechanism.
With the previous btrfs inode operations patches, now we can use
btrfs_mkdir() to create the 'lost+found' dir to do some data salvage in
btrfsck.

This patch along with previous ones will make data salvage easier.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:14:26 +01:00
Qu Wenruo 0cc75eddd0 btrfs-progs: Add btrfs_unlink() and btrfs_add_link() functions.
Add btrfs_unlink() and btrfs_add_link() functions in inode.c,
for the incoming btrfs_mkdir() and later inode operations functions.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 13:13:06 +01:00
Qu Wenruo c0af17d07f btrfs-progs: Import lookup/del_inode_ref() function.
Import lookup/del_inode_ref() function in inode-item.c, as base functions
for the incoming btrfs_add_link() and btrfs_unlink() functions.

Also modify btrfs_insert_inode_ref() and split_leaf() making them able
to deal with EXTENT_IREF incompat flag.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 11:53:48 +01:00
Qu Wenruo 2606756577 btrfs-progs: Import btrfs_insert/del/lookup_extref() functions.
Import btrfs_insert/del/lookup_extref() functions form kernel for the
incoming btrfs_add_link() and btrfs_unlink() functions.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-10 11:53:25 +01:00
Goffredo Baroncelli e12e91e859 btrfs-progs: Enhance the command btrfs filesystem df
Enhance the command "btrfs filesystem df" to show space usage information
for a mount point(s). It shows also an estimation of the space available,
on the basis of the current one used.

Signed-off-by: Goffredo Baroncelli <kreijack@inwind.it>
[code moved under #if 0 instead of deletion]
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-12-04 16:48:09 +01:00
Josef Bacik b25a40651d Btrfs-progs: delete bogus dir indexes
We may run across dir indexes that are corrupt in such a way that it makes them
useless, such as having a bad location key or a bad name.  In this case we can
just delete dir indexes that don't show up properly and then re-create what we
need.  When we delete dir indexes however we need to restart scanning the fs
tree as we could have greated bogus inode recs if the location key was bad, so
set it up so that if we had to delete an dir index we go ahead and free up our
inode recs and return -EAGAIN to check_fs_roots so it knows to restart the loop.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:45:03 +02:00
Josef Bacik d90d8d2323 Btrfs-progs: pull back backref.c and fix it up
This patch pulls back backref.c, adds a couple of helpers everywhere that it
needs, and cleans up backref.c to fit in btrfs-progs.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
[removed free_some_buffers after "do not reclaim extent buffer"]
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-10-14 10:39:38 +02:00
David Sterba da5db5596c btrfs-progs: read global reserve size from space infos
Kernels >= 3.15 export the global block reserve as a space info presented
by 'btrfs fi df' but would display 'unknown' instead of some meaningful
string.

Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-28 02:03:33 +02:00
Gui Hecheng 5f1313a45d btrfs-progs: remove unused flags for btrfs_path
The three flags of @btrfs_path:
	btrfs_path {
		unsigned int keep_locks:1;
		unsigned int skip_locking:1;
		unsigned int leave_spinning:1;
	}
have little meaning, because the userspace @btrfs_search_slot()
is free of locking and no other routines will decide their behavior
on these. So just remove them.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-28 02:03:18 +02:00
Gui Hecheng ad70b55b66 btrfs-progs: replace BTRFS_NUM_MIRRORS with BTRFS_MAX_MIRRORS
The chunk-recover.c/BTRFS_NUM_MIRRORS in the userspace means
the same thing as ctree.h/BTRFS_MAX_MIRRORS in the kernelspace,
so to stay consistent with the kernelspace, just make this movement
in the userspace:
	chunk-recover.c/BTRFS_NUM_MIRRORS
		===>
	ctree.h/BTRFS_MAX_MIRRORS

This provides convenience for future use.

Signed-off-by: Gui Hecheng <guihc.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-22 14:43:11 +02:00
Mark Fasheh 96ec888aad btrfs-progs: add quota group verify code
This patch adds functionality (in qgroup-verify.c) to compute bytecounts in
subvolume quota groups. The original groups are read in and stored in memory
so that after we compute our own bytecounts, we can compare them with those
on disk. A print function is provided to do this comparison and show the
results on the console.

A 'qgroup check' pass is added to btrfsck. If any subvolume quota groups
differ from what we compute, the differences for them are printed.  We also
provide an option '--qgroup-report' which will run only the quota check code
and print a report on all quota groups.  Other than making it possible to
verify that our qgroup changes work correctly, this mode can also be used in
xfstests for automated checking after qgroup tests.

This patch does not address the following:
- compressed counts are identical to non compressed, because kernel doesn't
  make the distinction yet.  Adding the code to verify compressed counts
  shouldn't be hard at all though once kernel can do this.
- It is only concerned with subvolume quota groups (like most of
  btrfs-progs).

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-08-22 14:39:33 +02:00
Filipe David Borba Manana c17d0a73c1 Btrfs-progs: update btrfs_file_extent_inline_len to match kernel version
The following kernel commit changed the definition of the inline function
btrfs_file_extent_inline_len():

    commit 514ac8ad8793a097c0c9d89202c642479d6dfa34
    Author: Chris Mason <clm@fb.com>
    Date:   Fri Jan 3 21:07:00 2014 -0800

    Btrfs: don't use ram_bytes for uncompressed inline items

    If we truncate an uncompressed inline item, ram_bytes isn't updated to reflect
    the new size.  The fixe uses the size directly from the item header when
    reading uncompressed inlines, and also fixes truncate to update the
    size as it goes.

Not having this new definition implies that the restore tool might misbehave when
restoring files with an inline extent that got truncated on a kernel older than
release 3.14.

Signed-off-by: Filipe David Borba Manana <fdmanana@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-04-11 19:47:29 +02:00
Kusanagi Kouichi 779387ba2a btrfs-progs: Constify getters
Signed-off-by: Kusanagi Kouichi <slash@ac.auone-net.jp>
Signed-off-by: David Sterba <dsterba@suse.cz>
2014-04-11 19:40:04 +02:00
Wang Shilong 41d6c3c265 Btrfs-progs: use bitfield instead of integer for some variants in fs_info
Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-03-21 06:23:29 -07:00
Josef Bacik 147114581d Btrfs-progs: add support for the no holes incompat flag
This adds the flag to ctree.h, adds the feature option to mkfs to turn it on and
fixes fsck so it doesn't complain about missing hole extents in files when this
flag is set.

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:23 -08:00
Josef Bacik 70749a77fe Btrfs-progs: deal with invalid key orderings and bad orphan items V2
A user had a fs where the objectid of an orphan item was not the actual orphan
item objectid.  This screwed up fsck because the block has keys in the wrong
order, also the fs scanning stuff will freak out because we have an inode with
nlink 0 and no orphan item.  So this patch is pretty big but is all related.

1) Deal with bad key ordering.  We can easily fix this up, so fix the checking
stuff to tell us exactly what it found when it said there was a problem.  Then
if it's bad key ordering we can reorder the keys and restart the scan.

2) Deal with bad keys.  If we find an orphan item with the wrong objectid it's
likely to screw with stuff, so keep track of these sort of things with a
bad_item list and just run through and delete any objects that don't make sense.
So far we just do this for orphan items but we could extend this as new stuff
pops up.

3) Deal with missing orphan items.  This is easy, if we have a file with i_nlink
set to 0 and no orphan item we can just add an orphan item.

4) Add the infrastructure to corrupt actual key values.  Needed this to create a
test image to verify I was fixing things properly.

This patch fixes the corrupt image I'm adding and passes the other make test
tests.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:15 -08:00
Wang Shilong 52ddfa74fe Btrfs-progs: chunk-recover: add new flag to prepare recovering for ordered data chunk
When reading block groups we will searching it's corresponding chunk, however, at this
time, some chunks has not been built(data chunks raid0/raid10/raid56), don't bug_on here,
we will try to rebuild these chunks later.

Signed-off-by: Wang Shilong <wangsl.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:07 -08:00
Ross Kirk 7ff512ce38 btrfs-progs: Make btrfs_header_chunk_tree_uuid() return unsigned long
Internally, btrfs_header_chunk_tree_uuid() calculates an unsigned
long, but casts it to a pointer, while all callers cast it to unsigned
long again.

From btrfs commit b308bc2f05a86e728bd035e21a4974acd05f4d1e

Signed-off-by: Ross Kirk <ross.kirk@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
2014-01-31 08:22:04 -08:00
Chris Mason d1570a0618 mkfs: fix mkfs -r to properly allocate space
mkfs -r wasn't creating chunks properly, making it very difficult to
allocate space for anything except tiny filesystems.

This changes it around to use more of the generic infrastructure, and
to do actual logical->physical block number translation.

It also allocates space to the files in smaller extents (max 1MB), which
keeps the allocator from trying to allocate an extent bigger than a
single chunk.

It doesn't quite support multi-device mkfs -r yet, but is much closer.

Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-10-16 10:42:53 -04:00
Wang Shilong 1e174d2621 Btrfs-progs: restructure show_qgroups
The current show_qgroups() just shows a little information, and it is hard to
add some functions which the users need in the future, so i restructure it, make
it easy to add new functions.

In order to improve the scalability of show_qgroups(), i add some important
structures:

	struct qgroup_lookup {
		struct rb_root root;
	}
 	/*
	*store qgroup's information
	*/
	struct btrfs_qgroup {
		struct rb_node  rb_node;
		u64 qgroupid;

		u64 generation;
		u64 rfer;
		u64 rfer_cmpr;
		u64 excl_cmpr;

		u64 flags;
		u64 max_rfer;
		u64 max_excl;
		u64 rsv_rfer;
		u64 rsv_excl;

		struct list_head qgroups;
		struct list_head members;
	}
	/*
	*glue structure to represent the relations
	*between qgroups
	*/
	struct btrfs_qgroup_list {
		struct list_head next_qgroups;
		struct list_head next_member;
		struct btrfs_qgroup *qgroup;
		struct btrfs_qgroup *member;
	}
The above 3 structures are used to manage all the information
of qgroups.

	struct {
		char *name;
		char *column_name;
		int need_print;
	} btrfs_qgroup_columns[]

We define a arrary to manage all the columns that can be
outputed, and use a member variant(->need_print) to control
the output of the relative column. Some columns are outputed
by default. But we can change it according to the requirement
of the users.

For example:
	if outputing max referenced size of qgroup is needed,the function
'btrfs_qgroup_setup_column()' will be called, and the parameter 'BTRFS_QGROUP_MAX_RFER'
(extend in the future) will be passsed to the function. After the function is done,
when showing qgroups, max referenced size of qgroup will be output.

Signed-off-by: Wang Shilong <wangsl-fnst@cn.fujitsu.com>
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-10-16 08:23:12 -04:00
Josef Bacik c64485544b Btrfs-progs: keep track of transid failures and fix them if possible
A user was reporting an issue with bad transid errors on his blocks.  The thing
is that btrfs-progs will ignore transid failures for things like restore and
fsck so we can do a best effort to fix a users file system.  So fsck can put
together a coherent view of the file system with stale blocks.  So if everything
else is ok in the mind of fsck then we can recow these blocks to fix the
generation and the user can get their file system back.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-10-16 08:23:12 -04:00
Anand Jain 23c47a132f btrfs-progs: move out print in cmd_df to another function
This is a prepatory work for the btrfs fi show command
fixes. So that we have a function get_df to get the fs sizes

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-10-16 08:23:11 -04:00
Ross Kirk 33ce9a82b8 btrfs-progs: Make btrfs_header_fsid() return unsigned long
Internally, btrfs_header_fsid() calculates an unsigned long, but casts
it to a pointer, while all callers cast it to unsigned long again.

Committed to btrfs as fba6aa75654394fccf2530041e9451414c28084f

Fix line length issues and match changes to kernelspace

Signed-off-by: Ross Kirk <ross.kirk@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-10-16 08:23:11 -04:00
Ross Kirk db6feaadfe btrfs-progs: remove unused parameter from btrfs_header_fsid
Remove unused parameter, 'eb'. Unused since introduction in
7777e63b42

Signed-off-by: Ross Kirk <ross.kirk@gmail.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-10-16 08:23:10 -04:00
Ross Kirk af86684e9b btrfs-progs: drop unused parameter from btrfs_item_nr
Remove unused eb parameter from btrfs_item_nr, unused since introduced
in 7777e63b42

Signed-off-by: Ross Kirk <ross.kirk@gmail.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-10-16 08:23:10 -04:00
Hidetoshi Seto 03e9539615 btrfs-progs: calculate available blocks on device properly
I found that mkfs.btrfs aborts when assigned multi volumes contain
a small volume:

  # parted /dev/sdf p
  Model: LSI MegaRAID SAS RMB (scsi)
  Disk /dev/sdf: 72.8GB
  Sector size (logical/physical): 512B/512B
  Partition Table: msdos

  Number  Start   End     Size    Type     File system  Flags
   1      32.3kB  72.4GB  72.4GB  primary
   2      72.4GB  72.8GB  461MB   primary

  # ./mkfs.btrfs -f /dev/sdf1 /dev/sdf2
  :
  SMALL VOLUME: forcing mixed metadata/data groups
  adding device /dev/sdf2 id 2
  mkfs.btrfs: volumes.c:852: btrfs_alloc_chunk: Assertion `!(ret)' failed.
  Aborted (core dumped)

This failure of btrfs_alloc_chunk was caused by following steps:
 1) since there is only small space in the small device, mkfs was
    going to allocate a chunk from free space as much as available.
    So mkfs called btrfs_alloc_chunk with
        size = device->total_bytes - device->used_bytes.
 2) (According to the comment in source code, to avoid overwriting
    superblock,) btrfs_alloc_chunk starts taking chunks at an offset
    of 1MB. It means that the layout of a disk will be like:
     [[1MB at beginning for sb][allocated chunks]* ... free space ... ]
    and you can see that the available free space for allocation is:
        avail = device->total_bytes - device->used_bytes - 1MB.
 3) Therefore there is only free space 1MB less than requested. damn.

>From further investigations I also found that this issue is easily
reproduced by using -A, --alloc-start option:

  # truncate --size=1G testfile
  # ./mkfs.btrfs -A900M -f testfile
   :
  mkfs.btrfs: volumes.c:852: btrfs_alloc_chunk: Assertion `!(ret)' failed.
  Aborted (core dumped)

In this case there is only 100MB for allocation but btrfs_alloc_chunk
was going to allocate more than the 100MB.

The root cause of both of above troubles is a same simple bug:
btrfs_chunk_alloc does not calculate available bytes properly even
though it researches how many devices have enough room to have a
chunk to be allocated.

So this patch introduces new function btrfs_device_avail_bytes()
which returns available bytes for allocation in specified device.

Signed-off-by: Hidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-10-16 08:20:42 -04:00
Zach Brown 323bf8c587 btrfs-progs: add ULL to u64 constant
This silences a sparse warning:

	warning: constant 0x4D5F53665248425F is so big it is long

from

  commit 52162700bb
  Author: Zach Brown <zab@redhat.com>
  Date:   Thu Jan 17 11:54:47 2013 -0800

      btrfs-progs: treat super.magic as an le64

High fives, past me!

Signed-off-by: Zach Brown <zab@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-09-03 19:41:01 +02:00
Eric Sandeen 989ca65a11 btrfs-progs: mark static & remove unused from shared kernel code
In files copied from the kernel, mark many functions as static,
and remove any resulting dead code.

Some functions are left unmarked if they aren't static in the
kernel tree.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-09-03 19:40:53 +02:00
Eric Sandeen 560f071313 btrfs-progs: make set/get functions for the super compat_ro flags use compat_ro
Port of commit 12534832 to userspace:

 commit 12534832cb7b0abc7369298246e8b7af03b863ca
 Author: Josef Bacik <josef@redhat.com>
 Date:   Thu Dec 17 21:32:27 2009 +0000

    Btrfs: make set/get functions for the super compat_ro flags use compat_ro

    Our set/get functions for compat_ro_flags actually look at compat_flags.  This
    will mess any attempt to use compat flags up.  The fix is obvious.  Thanks,

    Signed-off-by: Josef Bacik <josef@redhat.com>
    Signed-off-by: Chris Mason <chris.mason@oracle.com>

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-09-03 19:40:48 +02:00
Eric Sandeen c82ded6dbb btrfs-progs: fix definition of struct btrfs_extent_inline_ref
Port kernel commit 1bec1aed to userspace.

use __le64 instead of u64 in on-disk structure definition.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-09-03 19:40:48 +02:00
Eric Sandeen 4e8c4d4ba7 btrfs-progs: drop unused parameter from btrfs_release_path
Port of commit b3b4aa7 to userspace.

parameter tree root it's not used since commit
5f39d397dfbe140a14edecd4e73c34ce23c4f9ee ("Btrfs: Create extent_buffer
interface for large blocksizes")

This gets userspace a tad closer to kernelspace by removing
this unused parameter that was all over the codebase...

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-09-03 19:40:46 +02:00
Qu Wenruo 6c10240e4e btrfs-progs: Cleanup for using BTRFS_SETGET_STACK instead of raw convert
Some codes still use the cpu_to_lexx instead of the
BTRFS_SETGET_STACK_FUNCS declared in ctree.h.

Also added some BTRFS_SETGET_STACK_FUNCS for btrfs_header and
btrfs_super.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-08-09 14:32:32 +02:00
Stefan Behrens 1ff26009b3 Btrfs-progs: add uuid_tree_gen field to btrfs-show-super
Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-08-09 14:32:31 +02:00
Stefan Behrens ad280c1b3a Btrfs-progs: add UUID tree lookup methods
This commit adds UUID tree lookup methods that make use of the search
ioctl. The code is based on the kernel code.

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-08-09 14:32:31 +02:00
Stefan Behrens 64075c5be9 Btrfs-progs: Support UUID tree and UUID items in btrfs-debug-tree
Support printing these things.

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-08-09 14:32:31 +02:00
Eric Sandeen 37003d3627 Btrfs-progs: remove some dead/unbuilt code
Remove some commented-out & #if 0'd code:

 * close_blocks()
 * btrfs_drop_snapshot()
 * btrfs_realloc_node()
 * btrfs_find_dead_roots()

There are still some #if 0'd functions in there, but I'm hedging
on those for now, they have been copied to cmds-check.c and I want
to see if they can be brough back into ctree.c eventually.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-08-09 14:32:28 +02:00
Eric Sandeen 14b53c0736 Btrfs-progs: move btrfs_fsck_reinit_root to cmds-check.c
cmds-check.c contains the only caller of btrfs_fsck_reinit_root;
moving it to the caller's source file gets ctree.c a little
closer to kernelspace, although it does require exporting
add_root_to_dirty_list(), which is not done in kernelspace.

Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-08-09 14:32:28 +02:00
Chris Mason 0bae08fdab Merge branch 'liubo-image-restore'
Signed-off-by: Chris Mason <chris.mason@fusionio.com>

Conflicts:
	disk-io.c
	volumes.h
2013-07-03 14:24:43 -04:00
Liu Bo 095e21af45 Btrfs-progs: enhance btrfs-image to restore image onto multiple disks
This adds a 'btrfs-image -m' option, which let us restore an image that
is built from a btrfs of multiple disks onto several disks altogether.

This aims to address the following case,
$ mkfs.btrfs -m raid0 sda sdb
$ btrfs-image sda image.file
$ btrfs-image -r image.file sdc
---------
so we can only restore metadata onto sdc, and another thing is we can
only mount sdc with degraded mode as we don't provide informations of
another disk.  And, it's built as RAID0 and we have only one disk,
so after mount sdc we'll get into readonly mode.

This is just annoying for people(like me) who're trying to restore image
but turn to find they cannot make it work.

So this'll make your life easier, just tap
$ btrfs-image -m image.file sdc sdd
---------
then you get everything about metadata done, the same offset with that of
the originals(of course, you need offer enough disk size, at least the disk
size of the original disks).

Besides, this also works with raid5 and raid6 metadata image.

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-07-03 14:16:10 -04:00
Miao Xie ff04981b3f Btrfs-progs: use rb-tree instead of extent cache tree for fs/file roots
Because the fs/file roots are not extents, so it is better to use rb-tree
to manage them. Fix it.

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-07-03 14:06:54 -04:00
Josef Bacik 439ce45e48 Btrfs-progs: add --init-extent-tree to btrfsck
In some cases the extent tree can just be so gone there is no point in trying to
figure out how to put it back together.  So add a --init-extent-tree mode which
will zero out the extent tree and then re-add extents for all of the blocks we
find.  This will also undo any balance that was going on at the time of the
crash, this is needed because the reloc tree seems to confuse fsck at the
moment.  With this patch I can put back together a users file system that was
completely gone.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-06-19 13:52:34 -04:00
Josef Bacik 11be10f71e Btrfs-progs: make fsck fix certain file extent inconsistencies
The tree log bug I introduced could create inconsistent file extent entries in
the file system tree and in some worst cases even create multiple extent entries
for the same entry.  To fix this we need to do a few things

1) Keep track of extent items that overlap and then pick the one that covers the
largest area and delete the rest of the items.

2) Keep track of file extent items that land in extent items but don't match
disk_bytenr/disk_num_bytes exactly.  Once we find these we need to figure out
who is the right ref and then fix all of the other refs to agree.

Each of these cases require a complete rescan of all of the extents, so
unfortunately if you hit this particular problem the fsck is going to take quite
a while since it will likely rescan all the trees 2 or 3 times.  With this patch
the broken file system a user sent me is fixed and a broken file system that was
created by my reproducer is also fixed.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-06-19 13:52:16 -04:00
Josef Bacik d93cad2677 Btrfs-progs: add a free space cache checker to fsck V2
In trying to track down a weird tree log problem I wanted to make sure that the
free space cache was actually valid, which we currently have no way of doing.
So this patch adds a bunch of support for the free space cache code and then a
checker to fsck.  Basically we go through and if we can actually load the free
space cache then we will walk the extent tree and verify that the free space
cache exactly matches what is in the extent tree.  Hopefully this will always be
correct, the only time it wouldn't is if the extent tree is corrupt or we have
some sort of awful bug in the free space cache.  Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
2013-04-23 18:56:21 +02:00
Josef Bacik 7b20da8d52 Btrfs-progs: add skinny metadata support to progs V3
This fixes up the progs to properly deal with skinny metadata.  This adds the -x
option to mkfs and btrfstune for enabling the skinny metadata option.  This also
makes changes to fsck so it can properly deal with the skinny metadata entries.
Thanks,

Signed-off-by: Josef Bacik <jbacik@fusionio.com>
2013-04-23 18:56:20 +02:00
Arne Jansen 6d37fbfc1f Btrfs-progs: tool to visualize fragmentation
This tool draws per-chunk pngs representing the allocation map. A black
or colored dot means the block is allocated.
The output is written to a subdirectory, together with an index.html to be
viewed in a browser.
There are options to control whether color should be used and which block
group types should be printed.
To build, you need to have libpng and libgd installed. It is not part of
the 'all' target, so please build it explicitely with make btrfs-fragments.

A (rather untypical) example can be seen at

http://sensille.com/fragments

Please regard this as a first scratch version and feel free to improve it :)

Signed-off-by: Arne Jansen <sensille@gmx.net>
2013-03-19 18:24:51 +01:00
David Sterba dd21bc16ad btrfs-progs: separate super_copy out of fs_info
Allocate fs_info::super_copy dynamically of full BTRFS_SUPER_INFO_SIZE
and use it directly for saving superblock to disk.

This fixes incorrect superblock checksum after mkfs.

Signed-off-by: David Sterba <dsterba@suse.cz>
2013-03-10 16:12:21 +01:00
Mark Fasheh e5cb128a95 btrfs-progs: libify some parts of btrfs-progs
External software wanting to use the functionality provided by the btrfs
send ioctl has a hard time doing so without replicating tons of work. Of
particular interest are functions like btrfs_read_and_process_send_stream()
and subvol_uuid_search(). As that functionality requires a bit more than
just send-stream.c and send-utils.c we have to pull in some other parts of
the progs package.

This patch adds code to the Makefile and headers to create a library,
libbtrfs which the btrfs command now links to.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
Signed-off-by: David Sterba <dsterba@suse.cz>
2013-02-27 15:24:25 +01:00
Chris Mason 82ac34581e Merge branch 'cov-fixes-v1-integration-20130201' of http://git.zabbo.net/cgit/btrfs-progs into merged 2013-02-06 12:51:58 -05:00
Chris Mason 7b1c567c84 Merge branch 'for-chris' of git://repo.or.cz/btrfs-progs-unstable/devel into raid56
Conflicts:
	ctree.h

Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-02-06 12:42:24 -05:00
Zach Brown 52162700bb btrfs-progs: treat super.magic as an le64
The super block magic is a le64 whose value looks like an unterminated
string in memory.  The lack of null termination leads to clumsy use of
string functions and causes static analysis tools to warn that the
string will be unterminated.

So let's just treat it as the le64 that it is.  Endian wrappers are used
on the constant so that they're compiled into run-time constants.

Signed-off-by: Zach Brown <zab@redhat.com>
2013-02-05 16:09:32 -08:00
David Woodhouse 4d48b96b28 Add basic RAID[56] support
David Woodhouse originally contributed this code, and Chris Mason
changed it around to reflect the current design goals for raid56.

The original code expected all metadata and data writes to be full
stripes.  This meant metadata block size == stripe size, and had a few
other restrictions.

This version allows metadata blocks smaller than the stripe size.  It
implements both raid5 and raid6, although it does not have code to
rebuild from parity if one of the drives is missing or incorrect.

Signed-off-by: David Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2013-02-01 14:22:07 -05:00
Mark Fasheh 905b078928 btrfs-progs: Basic support for extended inode refs
This patch syncs the extended inode ref definitions from kernels ctree.h and
adds support in btrfs-debug-tree for visualizing the state of extended refs
on disk.

Signed-off-by: Mark Fasheh <mfasheh@suse.de>
2013-01-31 22:29:13 +01:00
Stefan Behrens 7e08a9116d Btrfs-progs: add support for device replace procedure
This is the user mode part of the device replace patch series.

The command group "btrfs replace" is added with three commands:
- btrfs replace start srcdev|srcdevid targetdev [-Bfr] mount_point
- btrfs replace status mount_point [-1]
- btrfs replace cancel mount_point

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
2013-01-31 13:47:26 +01:00
Stefan Behrens 5b8826ddfd Btrfs-progs: add command to get/reset device stats via ioctl
"btrfs device stats" is used to retrieve and print the device stats.
"btrfs device stats -z" is used to atomically retrieve, reset and
print the stats.

Signed-off-by: Stefan Behrens <sbehrens@giantdisaster.de>
2013-01-31 13:47:17 +01:00
Ben Peddell 7b668965f0 btrfs-progs: fix unaligned accesses v2
gcc optimizes out the memcpy calls at -O2 and -Os.

Replacing memcpy with memmove does't work - gcc treats memmove
the same way it treats memcpy.

This patch brings in {get|put}_unaligned_le{16|32|64} (using the
packed struct method), and uses them in the failing get/set calls.

On architectures where unaligned accesses are cheap, these unaligned
macros should be optimized out by the compiler.

Signed-off-by: Ben Peddell <klightspeed@killerwolves.net>
2013-01-28 18:06:43 +01:00
Arne Jansen 0b0a954e43 Btrfs-progs: fix unaligned accesses
There are some unaligned accesses in progs that cause malfunction or
crashes on ARM.
This patch fixes the ones we stumbled upon.

Signed-off-by: Arne Jansen <sensille@gmx.net>
2013-01-17 18:27:53 +01:00
Robin Dong 32be2a1164 btrfs-progs: limit the max value of leafsize and nodesize
Using mkfs.btrfs like:

        mkfs.btrfs -l 131072 /dev/sda

will return no error, but after mount it, the dmesg will report:

	BTRFS: couldn't mount because metadata blocksize (131072) was too large

The leafsize and nodesize are equal at present, so we just use one function
"check_leaf_or_node_size" to limit leaf and node size below BTRFS_MAX_METADATA_BLOCKSIZE.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Reviewed-by: David Sterba <dave@jikos.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
2012-10-04 16:26:34 -04:00
Miao Xie 3defb82384 Btrfs-progs: enhance btrfs subvol list only to show read-only snapshots
We want 'btrfs subvolume list' only to list readonly subvolumes, this patch set
introduces a new option 'r' to implement it.

You can use the command like that:

        btrfs subvolume list -r <path>

Original-Signed-off-by: Zhou Bo <zhoub-fnst@cn.fujitsu.com>
Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
2012-10-04 16:26:33 -04:00
Miao Xie 7cb1cf7542 Btrfs-progs: fix wrong way to check if the root item contains otime and uuid
Now we check if the root item contains otime and uuid or not by comparing
->generation_v2 and ->generation of the btrfs_root_item structure, it is
wrong because it is possbile that ->generation may equal to the first
variant of the next item. We fix this problem by check the size of btrfs_root_item,
if it is larger than the original one, the new btrfs_root_item contains otime
and uuid. we needn't worry the case that the new filesystem is mounted on the
old kernel. because the otime and uuid are not changed on the old kernel, we can
get the correct result even on the kernel.

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
2012-10-04 16:26:32 -04:00
Arne Jansen 89fe5b5f66 Btrfs progs: quota groups support
Signed-off-by: Jan Schmidt <list.btrfs@jan-o-sch.net>
Signed-off-by: Arne Jansen <sensille@gmx.net>
2012-09-04 11:15:49 +02:00
Alexander Block 84b9586a89 Btrfs-progs: update btrfs-progs for subvol uuid+times support
Update ctree.h and ioctl.h for the new uuid+times for subvolumes.

Signed-off-by: Alexander Block <ablock84@googlemail.com>
2012-07-26 14:51:17 -04:00
Chris Mason 2588b4f8be Merge branch 'for-chris' of git://github.com/idryomov/btrfs-progs into 0.20 2012-07-05 11:35:42 -04:00
Wang Sheng-Hui f9ffe57844 btrfs-progs: cleanup: remove the redundant BTRFS_CSUM_TYPE_CRC32 macro definition in ctree.h
Signed-off-by: Wang Sheng-Hui <shhuiw@gmail.com>
2012-06-05 13:35:58 +01:00
Chris Mason 1957076ab4 Add incompat flag for big metadata blocks
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-03-28 12:05:27 -04:00
Ilya Dryomov 842a2fb01b Btrfs-progs: make print-tree.c aware of free space cache
This adds proper formatting for free space and inode cache items in
btrfs-debug-tree output.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-03-13 18:36:56 +02:00
Ilya Dryomov a0bdde58f8 Btrfs-progs: bring 'subvol get-default' back in
Commit bab2c565 accidentally broke 'subvol get-default' command by
removing almost all of the underlying code.  Bring it back with some
fixes and improvements.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-03-12 19:17:39 +02:00
Chris Mason af00db292e Btrfsck: add the ability to prune corrupt extent allocation tree blocks
When we discover bad blocks in the extent allocation tree, repair can
now discard them and recreate the references from the rest of the trees.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-02-21 14:37:21 -05:00
Chris Mason 62b7993154 btrfsck: remove extents from the fsck reference tracker as they are freed
During btrfsck --repair, we make an index of extents that have incorrect
reference counts.  Once we've collect the whole index, we go through
and modify the extent allocation tree to reflect the correct results.

Changing the extent allocation tree may free blocks, and so it may
end up removing a block that had a missing reference structure.  The
fsck code may then circle back around and add the reference back.

The result is an extent that isn't actually used, but is recorded in the
extent allocation tree.

This commit adds a hook called as extents are freed.  The hook searches
the index of incorrect references and updates it to reflect the freeing
of the extent.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-02-10 13:28:50 -05:00
Chris Mason 47b104b7f1 btrfsck: add --init-csum-tree to replace the csum root with an empty one
This will effectively delete all of your crcs, but at least you'll
be able to mount the FS with nodatasum.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-02-09 10:38:05 -05:00
Chris Mason 5fd54bc94b btrfsck: fix block group accounting during repair
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-02-08 21:29:13 -05:00
Chris Mason f5c4c4f3b7 btrfsck: add code to rebuild extent records
This also includes a new --repair btrfsck option.  For now it can
only fix errors in the extent allocation tree.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
2012-02-07 10:58:32 -05:00
Ilya Dryomov 4f3a15d09a Btrfs-progs: add restriper headers
Add restriper headers and update print-tree.c

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2012-02-03 21:02:29 +02:00
Chris Mason 13eced9a0c Btrfs: add the tree history log to btrfs-progs
This also adds btrfs-debug-tree -R to print the history

Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-11-05 14:58:41 -04:00
Josef Bacik be826706b5 btrfs-progs: add a recovery utility to pull files from damanged filesystems
Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-10-27 12:49:54 -04:00
Jan Schmidt eecfba1d34 btrfs-progs: scrub ioctls
- scrub structs added
- ioctls for scrub
- BTRFS_FSID_SIZE moved

Signed-off-by: Jan Schmidt <list.btrfs@jan-o-sch.net>
Signed-off-by: Hugo Mills <hugo@carfax.org.uk>
2011-10-25 09:18:59 -04:00
Li Zefan a418b24318 Update for lzo support
[Btrfs-Progs][V2] Update for lzo support

- Add incompat flag, otherwise btrfs-progs will report error
  when operating on btrfs filesystems mounted with lzo option.

- Update man page.

- Allow to turn on lzo compression for defrag operation:

  # btrfs filesystem defragment -c[zlib, lzo] <file>

  Note: "-c zlib" will fail, because that's how getopt() works
  for optional arguments.

Signed-off-by: Li Zefan <lizf@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-10-25 09:18:31 -04:00
Josef Bacik b8802ae3fa Btrfs-progs: add support for mixed data+metadata block groups
So alot of crazy people (I'm looking at you Meego) want to use btrfs on phones
and such with small devices.  Unfortunately the way we split out metadata/data
chunks it makes space usage inefficient for volumes that are smaller than
1gigabyte.  So add a -M option for mixing metadata+data, and default to this
mixed mode if the filesystem is less than or equal to 1 gigabyte.  I've tested
this with xfstests on a 100mb filesystem and everything is a-ok.

Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-10-25 09:18:31 -04:00
Josef Bacik e2a6859d93 Btrfs-progs: update super fields for space cache
This patch updates the super field to add the cache_generation member.  It also
makes us set it to -1 on mkfs so any new filesystem will get the space cache
stuff turned on.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2011-10-25 09:18:31 -04:00
Josef Bacik 944eea95c0 fix incompat flags
I forgot to add BTRFS_FEATURE_INCOMPAT_DEFAULT_SUBVOL to the incompat flags in
btrfs-progs.  This adds it so that our tools don't freak out when touching a fs
with the default subvolume changed.  Thanks,

Signed-off-by: Josef Bacik <josef@redhat.com>
2010-09-23 20:26:49 -04:00
Chris Mason 8f55b769c7 Add btrfs subvol find-new command
btrfs-subvol find-new <subvol> <id> will search through a given subvol
and print out all the files with extents newer than a given id.

Care must be taken to make sure any pending delalloc is on disk before
running this because that won't show up in the output.
2010-03-18 12:32:32 -04:00
Chris Mason 4ff9e2af17 Add btrfs-list for listing subvolumes
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2010-02-28 15:35:38 -05:00
Chris Mason 95d3f20b51 Mixed back reference (FORWARD ROLLING FORMAT CHANGE)
This commit introduces a new kind of back reference for btrfs metadata.
Once a filesystem has been mounted with this commit, IT WILL NO LONGER
BE MOUNTABLE BY OLDER KERNELS.

The new back ref provides information about pointer's key, level and in which
tree the pointer lives. This information allow us to find the pointer by
searching the tree. The shortcoming of the new back ref is that it only works
for pointers in tree blocks referenced by their owner trees.

This is mostly a problem for snapshots, where resolving one of these fuzzy back
references would be O(number_of_snapshots) and quite slow.  The solution used
here is to use the fuzzy back references in the common case where a given tree
block is only referenced by one root, and use the full back references when
multiple roots have a reference
2009-06-08 13:30:36 -04:00
Chris Mason cc04d99e90 Add scan of the btrfs log tree to btrfs-debug-tree 2009-04-15 14:30:14 -04:00
Yan Zheng 9a6930e9be Add semantic checks to btrfsck for files and directories
This patch makes btrfsck check more things, including
directory items, file extents, checksumming, inode link
counts etc.

The code for these checks is similar to the code verifies
extent back references. The main difference is that
shared tree blocks are treated specially. The partial
checking results(unresolved references and/or errors)
of shared sub-trees are cached. This avoids scanning
the shared blocks several times. Thank you,

Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
2009-01-07 14:57:12 -05:00
Yan Zheng 0d53b212d8 Btrfs: update converter for the new disk format
This patch updates the ext3 to btrfs converter for the new
disk format. This mainly involves changing the convert's
data relocation and free space management code. This patch
also ports some functions from kernel module to btrfs-progs.
Thank you,

Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
2008-12-17 16:10:07 -05:00
Chris Mason b238b4b072 Btrfs: Add inode sequence number for NFS and reserved space in a few structs
This adds a sequence number to the btrfs inode that is increased on
every update.  NFS will be able to use that to detect when an inode has
changed, without relying on inaccurate time fields.

While we're here, this also:

Puts reserved space into the super block and inode

Adds a log root transid to the super so we can pick the newest super
based on the fsync log as well as the main transaction ID.  For now
the log root transid is always zero, but that'll get fixed.

Adds a starting offset to the dev_item.  This will let us do better
alignment calculations if we know the start of a partition on the disk.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 17:01:14 -05:00
Chris Mason d79f499eae Btrfs: move data checksumming into a dedicated tree
Btrfs stores checksums for each data block.  Until now, they have
been stored in the subvolume trees, indexed by the inode that is
referencing the data block.  This means that when we read the inode,
we've probably read in at least some checksums as well.

But, this has a few problems:

* The checksums are indexed by logical offset in the file.  When
compression is on, this means we have to do the expensive checksumming
on the uncompressed data.  It would be faster if we could checksum
the compressed data instead.

* If we implement encryption, we'll be checksumming the plain text and
storing that on disk.  This is significantly less secure.

* For either compression or encryption, we have to get the plain text
back before we can verify the checksum as correct.  This makes the raid
layer balancing and extent moving much more expensive.

* It makes the front end caching code more complex, as we have touch
the subvolume and inodes as we cache extents.

* There is potentitally one copy of the checksum in each subvolume
referencing an extent.

The solution used here is to store the extent checksums in a dedicated
tree.  This allows us to index the checksums by phyiscal extent
start and length.  It means:

* The checksum is against the data stored on disk, after any compression
or encryption is done.

* The checksum is stored in a central location, and can be verified without
following back references, or reading inodes.

This makes compression significantly faster by reducing the amount of
data that needs to be checksummed.  It will also allow much faster
raid management code in general.

The checksums are indexed by a key with a fixed objectid (a magic value
in ctree.h) and offset set to the starting byte of the extent.  This
allows us to copy the checksum items into the fsync log tree directly (or
any other tree), without having to invent a second format for them.

Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-08 17:00:31 -05:00
Chris Mason d45ee76e4f Rev the disk format for the compat code and the csum selection
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-12-02 10:20:36 -05:00
Josef Bacik 1148e55804 btrfs-progs: support for different csum algorithims
This is the btrfs-progs version of the patch to add the ability to have
different csum algorithims.  Note I didn't change the image maker since it
seemed a bit more complicated than just changing some stuff around so I will let
Yan take care of that.

Everything else was converted and for now a mkfs just
sets the type to be BTRFS_CSUM_TYPE_CRC32.

Signed-off-by: Josef Bacik <jbacik@redhat.com>
2008-12-02 09:58:23 -05:00
Josef Bacik 76b8244a7a btrfs-progs: add support for compat flags
This patch updates btrfs-progs with the disk format changes for the
compatability flags.

Signed-off-by: Josef Bacik <jbacik@redhat.com>
2008-12-02 07:18:36 -05:00
Yan Zheng aa62e84c84 Btrfs image tool
This patch adds btrfs image tool. The image tool is
a debugging tool that creates/restores btrfs metadump
image.

Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
2008-11-20 09:52:48 -05:00
Chris Mason 49bc666d5f Update the super magic string to match the seed and root format changes 2008-11-18 14:13:44 -05:00
Chris Mason 2d9bc57b9a Add disk format requirements for subvol backward and forward refs 2008-11-18 10:34:08 -05:00
Yan Zheng 4d1d3a59d6 update btrfs-progs for seed device support
This patch does the following:

1) Update device management code to match the kernel code.

2) Allocator fixes.

3) Add a program called btrfstune to set/clear the SEEDING
   super block flags.
2008-11-18 10:40:06 -05:00
Yan Zheng 95470dfaf1 Add fallocate support v2
This patch updates btrfs-progs for fallocate support.
 
Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
2008-10-31 12:48:02 -04:00
Chris Mason 712c1dea5e Rev the disk format for compression and root pointer generation fields
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-29 14:09:40 -04:00
Yan Zheng 38702ea7c6 Add root tree pointer transaction ids
This patch adds transaction IDs to root tree pointers.
Transaction IDs in tree pointers are compared with the
generation numbers in block headers when reading root
blocks of trees. This can detect some types of IO errors.

Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
2008-10-29 14:07:47 -04:00
Chris Mason c830821ddf Add disk format elements for compression
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-29 14:37:41 -04:00
Chris Mason b431f25ec7 Rev the disk format for the new back reference format
Signed-off-by: Chris Mason <chris.mason@oracle.com>
2008-10-09 13:40:23 -04:00
Yan Zheng 9559e0b09e Count space allocated to file in bytes
This patch updates btrfs-progs for counting space
allocated to file in bytes.

Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
2008-10-09 11:55:26 -04:00
Yan Zheng 5986faaf47 Remove offset field from struct btrfs_extent_ref
The offset field in struct btrfs_extent_ref records the position
inside file that file extent is referenced by. In the new back
reference system, tree leaves holding reference to file extent
are recorded explicitly. We can quickly scan these tree leaves, so the
offset field is not required.

This patch also makes the back reference system check the objectid
when extents are being deleted

Signed-off-by: Yan Zheng <zheng.yan@oracle.com>
2008-10-09 11:55:30 -04:00
Zheng Yan 428b7fa630 Full back reference support
This patch makes the back reference system to explicit record the
location of parent node for all types of extents. The location of
parent node is placed into the offset field of backref key. Every
time a tree block is balanced, the back references for the affected
lower level extents are updated.
2008-09-23 12:29:10 -04:00
Chris Mason cea88ec1d7 Disk format changes required for write ahead tree log 2008-09-05 16:15:58 -04:00
Chris Mason 2f2f9ef77a Rev the disk format 2008-08-21 15:49:34 -04:00
Josef Bacik 0045e0dd70 btrfs-progs: add orphan support to print-tree
This adds orphan support to print-tree so when debug_tree hits an orphan item it
will print out "orphan item" under it so you know what it is
2008-07-30 09:15:02 -04:00
Chris Mason e74b89d675 Rev the disk format 2008-07-24 13:52:04 -04:00
Josef Bacik 059c20b384 btrfs-progs new dir index support 2008-07-24 12:13:32 -04:00
Chris Mason a62332eeb2 Add a readonly flag open_ctree to force RO opens 2008-05-05 09:45:26 -04:00
Chris Mason 083faf794f Add mkfs.btrfs -A offset to control allocation start on devices
This is a utility option for the resizer, it makes sure to allocate
at offset bytes in the disk or higher.  It ensures the resizer will have
something to move when testing it.
2008-04-25 16:55:21 -04:00
Yan Zheng 4415143185 Speed improvement and bug fixes for ext3 converter
This patch improves converter's allocator and fixes a bug in data relocation
function. The new allocator caches free blocks as Btrfs's default allocator.
In testing here, the user CPU time reduced to half of the original when
checksum and small file packing was disabled. This patch also enlarges the
size of block groups created by the converter.
2008-04-24 14:57:50 -04:00