Commit Graph

179 Commits (master)

Author SHA1 Message Date
Qu Wenruo 77b2af0a9a btrfs-progs: do proper error handling in btrfs_chunk_readonly()
[BUG]
For a fuzzed image, `btrfs check` both modes trigger BUG_ON():

  Opening filesystem to check...
  volumes.c:1795: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value 1
  btrfs(+0x2f712)[0x557beff3b712]
  btrfs(+0x32059)[0x557beff3e059]
  btrfs(btrfs_read_block_groups+0x282)[0x557beff30972]
  btrfs(btrfs_setup_all_roots+0x3f3)[0x557beff2ab23]
  btrfs(+0x1ef53)[0x557beff2af53]
  btrfs(open_ctree_fs_info+0x90)[0x557beff2b1a0]
  btrfs(+0x6d3f9)[0x557beff793f9]
  btrfs(main+0x94)[0x557beff200c4]
  /usr/lib/libc.so.6(__libc_start_main+0xf3)[0x7f623ac97ee3]
  btrfs(_start+0x2e)[0x557beff2035e]

[CAUSE]
The fuzzed image has a bad extent tree:

        item 0 key (288230376165343232 BLOCK_GROUP_ITEM 8388608) itemoff 16259 itemsize 24
                block group used 0 chunk_objectid 256 flags DATA

There is no corresponding chunk for the block group.

In then we hit the BUG_ON(), which expects chunk mapping for
btrfs_chunk_readonly().

[FIX]
Remove that BUG_ON() with proper error handling, and make
btrfs_read_block_groups() handle the -ENOENT error from
read_one_block_group() to continue.

So one corrupted block group item won't screw up the remaining block
group items.

Issue: #209
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:10 +01:00
Qu Wenruo 6466dbfdab btrfs-progs: Add extra chunk item size check
[BUG]
For one fuzzed image, `btrfs check` both modes will trigger a BUG_ON():
  Opening filesystem to check...
  Checking filesystem on issue_208.raw
  UUID: 99e50868-0bda-4d89-b0e4-7e8560312ef9
  [1/7] checking root items
  [2/7] checking extents
  ctree.h:320: btrfs_chunk_item_size: BUG_ON `num_stripes == 0` triggered, value 1
  btrfs(+0x2f712)[0x55829afa6712]
  btrfs(+0x322e5)[0x55829afa92e5]
  btrfs(+0x6892a)[0x55829afdf92a]
  btrfs(+0x69099)[0x55829afe0099]
  btrfs(+0x69c68)[0x55829afe0c68]
  btrfs(+0x6dc27)[0x55829afe4c27]
  btrfs(main+0x94)[0x55829af8b0c4]
  /usr/lib/libc.so.6(__libc_start_main+0xf3)[0x7f3edc715ee3]
  btrfs(_start+0x2e)[0x55829af8b35e]

[CAUSE]
The fuzzed image has an invalid chunk item in chunk tree:
  item 1 key (FIRST_CHUNK_TREE CHUNK_ITEM 13631488) itemoff 16105 itemsize 80
  invalid num_stripes: 0

Which triggers that BUG_ON().

[FIX]
Here we enhance the verification of btrfs_check_chunk_valid(), to check
the num_stripes and item size.

Issue: #208
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:10 +01:00
Qu Wenruo 6a06115861 btrfs-progs: disk-io: Verify the bytenr passed in is mapped for read_tree_block()
[BUG]
For a fuzzed image, `btrfs check` will segfault at open_ctree() stage:

  $ btrfs check --mode=lowmem issue_207.raw
  Opening filesystem to check...
  extent_io.c:665: free_extent_buffer_internal: BUG_ON `eb->refs < 0` triggered, value 1
  btrfs(+0x6bf67)[0x56431d278f67]
  btrfs(+0x6c16e)[0x56431d27916e]
  btrfs(alloc_extent_buffer+0x45)[0x56431d279db5]
  btrfs(read_tree_block+0x59)[0x56431d2848f9]
  btrfs(btrfs_setup_all_roots+0x29c)[0x56431d28535c]
  btrfs(+0x78903)[0x56431d285903]
  btrfs(open_ctree_fs_info+0x90)[0x56431d285b60]
  btrfs(+0x45a01)[0x56431d252a01]
  btrfs(main+0x94)[0x56431d2220c4]
  /usr/lib/libc.so.6(__libc_start_main+0xf3)[0x7f6e28519153]
  btrfs(_start+0x2e)[0x56431d22235e]

[CAUSE]
The fuzzed image has a strange log root bytenr:

  log_root                61440
  log_root_transid        0

In fact, the log_root seems to be fuzzed, as its transid is 0, which is
invalid.

Note that range [61440, 77824) covers the physical offset of the primary
super block.

The bug is caused by the following sequence:

1. cache for tree block [64K, 68K) is created by open_ctree()
   __open_ctree_fd()
   |- btrfs_setup_chunk_tree_and_device_map()
      |- btrfs_read_sys_array()
         |- sb = btrfs_find_create_tree_block()
         |- free_extent_buffer(sb)

   This created an extent buffer [64K, 68K) in fs_info->extent_cache, then
   reduce the refcount of that eb back to 0, but not freed yet.

2. Try to read that corrupted log root
   __open_ctree_fd()
   |- btrfs_setup_chunk_tree_and_device_map()
   |- btrfs_setup_all_roots()
      |- find_and_setup_log_root()
         |- read_tree_block()
            |- btrfs_find_create_tree_block()
               |- alloc_extent_buffer()

   The final alloc_extent_buffer() will try to free that cached eb
   [64K, 68K), since it doesn't match with current search.
   And since that cached eb is already released (refcount == 0), the
   extra free_extent_buffer() will cause above BUG_ON().

[FIX]
Here we fix it through a more comprehensive method, instead of simply
verifying log_root_transid, here we just don't pollute eb cache when
reading sys chunk array.

So that we won't have an eb cache [64K, 68K), and will error out at
logical mapping phase.

Issue: #207
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2020-01-09 14:27:09 +01:00
David Sterba 1f5094bb5c btrfs-progs: add support for raid1c3 and raid1c4
Add support for 3- and 4- copy variants of RAID1. This adds resiliency
against 2 or resp. 3 devices lost or damaged.

$ ./mkfs.btrfs -m raid1c4 -d raid1c3 /dev/sd[abcd]

Label:              (null)
UUID:               f1f988ab-6750-4bc2-957b-98a4ebe98631
Node size:          16384
Sector size:        4096
Filesystem size:    8.00GiB
Block group profiles:
  Data:             RAID1C3         273.06MiB
  Metadata:         RAID1C4         204.75MiB
  System:           RAID1C4           8.00MiB
SSD detected:       no
Incompat features:  extref, skinny-metadata, raid1c34
Number of devices:  4
Devices:
   ID        SIZE  PATH
    1     2.00GiB  /dev/sda
    2     2.00GiB  /dev/sdb
    3     2.00GiB  /dev/sdc
    4     2.00GiB  /dev/sdd

Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-22 19:09:50 +01:00
Nikolay Borisov beef042d50 btrfs-progs: Remove convert param from btrfs_alloc_data_chunk
Convert is always set to true so there's no point in having it as a
function parameter or using it as a predicate inside
btrfs_alloc_data_chunk.  Remove it and all relevant code which would
have never been executed.  No semantics changes.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-22 19:07:21 +01:00
Nikolay Borisov f28a5a1673 btrfs-progs: Remove type argument from btrfs_alloc_data_chunk
It's always set to BTRFS_BLOCK_GROUP_DATA so sink it into the function.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-22 19:07:21 +01:00
Nikolay Borisov 6718ab4d33 btrfs-progs: Initialize sub_stripes to 1 in btrfs_alloc_data_chunk
sub_stripe variables is by default initialized to 0 and it's overriden
only in case we have RAID10 mode. This leads to the following (minor)
artifacts on a freshly created filesystem:

item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 30408704) itemoff 15863 itemsize 112
		length 1073741824 owner 2 stripe_len 65536 type METADATA|RAID1
		io_align 65536 io_width 65536 sector_size 4096
		num_stripes 2 sub_stripes 0
			stripe 0 devid 2 offset 9437184
			dev_uuid a020fc2f-b526-4800-9278-156f2f431fe9
			stripe 1 devid 1 offset 30408704
			dev_uuid 0f78aa72-4626-4057-a8f2-285f46b2c664

After balance resulting chunk item is:

item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 3251634176) itemoff 15863 itemsize 112
		length 268435456 owner 2 stripe_len 65536 type METADATA|RAID1
		io_align 65536 io_width 65536 sector_size 4096
		num_stripes 2 sub_stripes 1
			stripe 0 devid 2 offset 3230662656
			dev_uuid a020fc2f-b526-4800-9278-156f2f431fe9
			stripe 1 devid 1 offset 3251634176
			dev_uuid 0f78aa72-4626-4057-a8f2-285f46b2c664

Kernel code usually initializes it to 1, since it takes the value from
the raid description table which has it set to 1 for all but RAID10 types.
In userspace it has to be statically initialized to 1 since we don't
have btrfs_bg_flags_to_raid_index. Eventually the kernel/userspace needs
to be merged but for now it wouldn't bring much value if this function
is copied.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-11-22 19:07:21 +01:00
Anand Jain c501c9e3b8 btrfs-progs: mkfs: match devid order to the stripe index
At the time mkfs.btrfs the device id and stripe index gets reversed as
shown in [1]. This patch helps to keep them in order at the time of
mkfs.btrfs. And makes it easier to debug.

Before:
Stripe 0 is on devid 2; Stipe 1 is on devid 1;

./mkfs.btrfs -fq -draid1 -mraid1 /dev/sdb /dev/sdc && btrfs in dump-tree -d /dev/sdb | grep -A 10000 "chunk tree" | grep -B 10000 "device tree" | grep -A 13  "FIRST_CHUNK_TREE CHUNK_ITEM"
	item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 22020096) itemoff 15975 itemsize 112
		length 8388608 owner 2 stripe_len 65536 type SYSTEM|RAID1
		io_align 65536 io_width 65536 sector_size 4096
		num_stripes 2 sub_stripes 0
			stripe 0 devid 2 offset 1048576
			dev_uuid d9fe51c4-6e79-446d-87ee-5be3184798cd
			stripe 1 devid 1 offset 22020096
			dev_uuid 16f626ca-1a54-469b-ac7e-25623af884ab
	item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 30408704) itemoff 15863 itemsize 112
		length 268435456 owner 2 stripe_len 65536 type METADATA|RAID1
		io_align 65536 io_width 65536 sector_size 4096
		num_stripes 2 sub_stripes 0
			stripe 0 devid 2 offset 9437184
			dev_uuid d9fe51c4-6e79-446d-87ee-5be3184798cd
			stripe 1 devid 1 offset 30408704
			dev_uuid 16f626ca-1a54-469b-ac7e-25623af884ab
	item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 298844160) itemoff 15751 itemsize 112
		length 314572800 owner 2 stripe_len 65536 type DATA|RAID1
		io_align 65536 io_width 65536 sector_size 4096
		num_stripes 2 sub_stripes 0
			stripe 0 devid 2 offset 277872640
			dev_uuid d9fe51c4-6e79-446d-87ee-5be3184798cd
			stripe 1 devid 1 offset 298844160
			dev_uuid 16f626ca-1a54-469b-ac7e-25623af884ab

After:
Stripe 0 is on devid 1; Stripe 1 is on devid 2

./mkfs.btrfs -fq -draid1 -mraid1 /dev/sdb /dev/sdc && btrfs in dump-tree -d /dev/sdb | grep -A 10000 "chunk tree" | grep -B 10000 "device tree" | grep -A 13  "FIRST_CHUNK_TREE CHUNK_ITEM"
/dev/sdb: 8 bytes were erased at offset 0x00010040 (btrfs): 5f 42 48 52 66 53 5f 4d
/dev/sdc: 8 bytes were erased at offset 0x00010040 (btrfs): 5f 42 48 52 66 53 5f 4d
	item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 22020096) itemoff 15975 itemsize 112
		length 8388608 owner 2 stripe_len 65536 type SYSTEM|RAID1
		io_align 65536 io_width 65536 sector_size 4096
		num_stripes 2 sub_stripes 0
			stripe 0 devid 1 offset 22020096
			dev_uuid 6abc88fa-f42e-4f0c-9bc3-2225735e51d1
			stripe 1 devid 2 offset 1048576
			dev_uuid 73746d27-13a6-4d58-ac6b-48c90c31d94d
	item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 30408704) itemoff 15863 itemsize 112
		length 268435456 owner 2 stripe_len 65536 type METADATA|RAID1
		io_align 65536 io_width 65536 sector_size 4096
		num_stripes 2 sub_stripes 0
			stripe 0 devid 1 offset 30408704
			dev_uuid 6abc88fa-f42e-4f0c-9bc3-2225735e51d1
			stripe 1 devid 2 offset 9437184
			dev_uuid 73746d27-13a6-4d58-ac6b-48c90c31d94d
	item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 298844160) itemoff 15751 itemsize 112
		length 314572800 owner 2 stripe_len 65536 type DATA|RAID1
		io_align 65536 io_width 65536 sector_size 4096
		num_stripes 2 sub_stripes 0
			stripe 0 devid 1 offset 298844160
			dev_uuid 6abc88fa-f42e-4f0c-9bc3-2225735e51d1
			stripe 1 devid 2 offset 277872640
			dev_uuid 73746d27-13a6-4d58-ac6b-48c90c31d94d

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-10-14 17:28:05 +02:00
David Sterba c07960c8be btrfs-progs: move utils.[ch] to common/
Update include paths and remove some duplicates.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 20:49:04 +02:00
David Sterba eb37e8212d btrfs-progs: sync btrfs_raid_attr from kernel
The table has been updated, copy the changes so that we can utilize it
for cleanups.

Note, ncopies for raid5 and rai6 was wrong and is now correct.

Signed-off-by: David Sterba <dsterba@suse.com>
2019-07-03 13:31:16 +02:00
Qu Wenruo 3ef4bd5cb8 btrfs-progs: Unify metadata chunk size with kernel
Mkfs tends to create pretty large metadata chunk compared to kernel:
  Node size:          16384
  Sector size:        4096
  Filesystem size:    10.00GiB
  Block group profiles:
    Data:             single            8.00MiB
    Metadata:         DUP               1.00GiB
    System:           DUP               8.00MiB

While kernel only tends to create 256MiB metadata chunk:
		/* for larger filesystems, use larger metadata chunks */
		if (fs_devices->total_rw_bytes > 50ULL * SZ_1G)
			max_stripe_size = SZ_1G;
		else
			max_stripe_size = SZ_256M;

This won't cause problems in real world, but it's still better to make
the behavior unified.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-03-01 17:26:02 +01:00
Qu Wenruo 0dd9031159 btrfs-progs: Port kernel fs_devices::total_rw_bytes
Unlike kernel, btrfs-progs doesn't (yet) support devices grow/shrink,
the port only needs to handle open_ctree() time initialization (at
read_one_dev()), and btrfs_add_device() used for mkfs.

This provide the basis for incoming unification of chunk allocator
behavior.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-03-01 17:26:02 +01:00
Nikolay Borisov f7717d8cdb btrfs-progs: Remove fsid/metdata_uuid fields from fs_info
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-01-15 16:20:09 +01:00
Nikolay Borisov c4aadd9af2 btrfs-progs: Add support for metadata_uuid field
Add support for a new metadata_uuid field. This is just a preparatory
commit which switches all users of the fsid field for metdata comparison
purposes to utilize the new field. This more or less mirrors the
kernel patch, additionally:

 * Update 'btrfs inspect-internal dump-super' to account for the new
 field. This involes introducing the 'metadata_uuid' line to the
 output and updating the logic for comparing the fs uuid to the
 dev_item uuid.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-12-06 12:51:36 +01:00
Qu Wenruo e6c1fa297a btrfs-progs: volumes: Refactor btrfs_alloc_dev_extent() into two functions
We have btrfs_alloc_dev_extent() accepting @convert flag to toggle
special handling for convert.

However the @convert flag only determines whether we call
find_free_dev_extent(), and we may later need to insert dev extents
without searching dev tree.

So refactor btrfs_alloc_dev_extent() into 2 functions,

- btrfs_alloc_dev_extent(), which will try to find free dev extent, and
- btrfs_insert_dev_extent(), which will just inserts a dev extent

For implementation, btrfs_alloc_dev_extent() will call
btrfs_insert_dev_extent() anyway, so there's no duplicated code.

This removes the need of @convert parameter, and make
btrfs_insert_dev_extent() public for later usage.

Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-12-05 15:47:23 +01:00
David Sterba e578b59bf6 btrfs-progs: convert strerror to implicit %m
Similar to the changes where strerror(errno) was converted, continue
with the remaining cases where the argument was stored in another
variable.

The savings in object size are about 4500 bytes:

 $ size btrfs.old btrfs.new
   text    data     bss     dec     hex filename
 805055   24248   19748  849051   cf49b btrfs.old
 804527   24248   19748  848523   cf28b btrfs.new

Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-31 18:24:14 +01:00
Qu Wenruo 40073c3ca9 btrfs-progs: exit gracefully when device extent allocation fails
Another BUG_ON() during fuzz/003:

  ====== RUN MAYFAIL btrfs check --repair tests/fuzz-tests/images/bko-199833-reloc-recovery-crash.raw.restored
  [1/7] checking root items
  Fixed 0 roots.
  [2/7] checking extents
  ctree.c:1650: leaf_space_used: Warning: assertion `data_len < 0` failed, value 1
  bad key ordering 18 19
  bad block 29409280
  ERROR: errors found in extent allocation tree or chunk allocation
  WARNING: minor unaligned/mismatch device size detected
  WARNING: recommended to use 'btrfs rescue fix-device-size' to fix it
  [3/7] checking free space cache
  [4/7] checking fs roots
  ctree.c:1650: leaf_space_used: Warning: assertion `data_len < 0` failed, value 1
  bad key ordering 18 19
  root 18446744073709551608 missing its root dir, recreating
  Unable to find block group for 0
  Unable to find block group for 0
  Unable to find block group for 0
  volumes.c:564: btrfs_alloc_dev_extent: BUG_ON `ret` triggered, value -28
  failed (ignored, ret=134): btrfs check --repair tests/fuzz-tests/images/bko-199833-reloc-recovery-crash.raw.restored
  mayfail: returned code 134 (SIGABRT), not ignored
  test failed for case 003-multi-check-unmounted

However the culprit function btrfs_alloc_dev_extent() has proper error
handling label err:, just using that label would solve the problem easily.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-10-23 14:48:40 +02:00
james harvey 7c5a355b74 btrfs-progs: btrfs_close_devices: only fsync if device->writeable is set
Prevent unnecessary error from failing fsync(), if opened read only.

Performed 'grep "writeable = " *.h *.c' to make sure there were no odd
situations where fsync() might still be desired here.  They're all straight-
forward.  The only situation where writeable will be 0 is if btrfs_open_devices
is given flags without O_RDWR.  There is no situation where a writeable volume
temporarily becomes unwriteable, or anything like that.  Given that it's being
opened O_RDWR, there's no reason to attempt fsync().

utils.c

   int btrfs_add_to_fsid() {
      ...
      device->writeable = 1;

volumes.c

   int btrfs_close_devices() {
      ...
      while (!list_empty(&fs_devices->devices)) {
         ...
         // just after the fsync() being patched
267:     device->writeable = 0;
   ...
   int btrfs_open_devices() {
      ...
      list_for_each_entry(device, &fs_devices->devices, dev_list) {
         ...
         if (flags & O_RDWR)
332:        device->writeable = 1

kernel btrfs_close_devices() does not have a corresponding fsync() that I see.

Signed-off-by: James Harvey <jamespharvey20@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:03:33 +02:00
Qu Wenruo b6582421ed btrfs-progs: Exit gracefully when overlapping chunks are detected
BUG_ON() can be triggered if some image contains overlappin chunks.

volumes.c:1930: read_one_chunk: BUG_ON `ret` triggered, value -17
btrfs(+0x2cf12)[0x5601efa17f12]
btrfs(+0x2fd8b)[0x5601efa1ad8b]
btrfs(btrfs_read_chunk_tree+0x2bf)[0x5601efa1b30f]
btrfs(btrfs_setup_chunk_tree_and_device_map+0xe8)[0x5601efa07718]
btrfs(+0x1c944)[0x5601efa07944]
btrfs(open_ctree_fs_info+0x90)[0x5601efa07b90]
btrfs(cmd_check+0x4d7)[0x5601efa4f8c7]
btrfs(main+0x88)[0x5601ef9fd768]
/usr/lib/libc.so.6(__libc_start_main+0xeb)[0x7f3c7787306b]
btrfs(_start+0x2a)[0x5601ef9fd88a]

Extent cache code can already detect it without problems, we only need
to remove the BUG_ON() and exit gracefully.

Reported-by: Xu Wen <wen.xu@gatech.edu>
Link: https://bugzilla.kernel.org/show_bug.cgi?id=200409
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:02:59 +02:00
Qu Wenruo 4721d774b6 btrfs-progs: Don't BUG_ON() if we failed to load one device or one chunk
Don't panic for btrfs_read_chunk_tree() if one device or chunk is
corrupted.
Caller can already handle it pretty well.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=199839
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-08-06 15:01:23 +02:00
Nikolay Borisov 9e2bf8c8ab btrfs-progs: Use symbolic names for read ahead behavior
Presently btrfs-progs haven't pulled the enum defining the symbolic
names of read ahead constants. This commit adds the enum and
simultaneously converts all usages to respective symbolic name.
No functional change, just making the code human readable.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-06-07 16:37:38 +02:00
Nikolay Borisov 873fba7101 btrfs-progs: Remove devid parameter from btrfs_rmap_block
This parameter was introduced with the original implementation of the
function but has never really been used, so just remove it.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-06-07 16:37:32 +02:00
Gu Jinxiang a5e29bbc1b btrfs-progs: Let function find_device to be consistent with kernel
Make find_device to be consistent with kernel according
35c70103a528 ("btrfs: refactor find_device helper")

And, modify the compare condition from both devid and uuid to
devid or devid and uuid according
8f18cf13396c ("Btrfs: Make the resizer work based on shrinking and growing devices")

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-04-24 13:00:10 +02:00
Qu Wenruo f174972d2d btrfs-progs: volumes: Allow find_free_dev_extent() to return maximum hole size
Just as kernel find_free_dev_extent(), allow it to return maximum hole
size for us to build device list for later chunk allocator rework.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-24 01:37:17 +01:00
Qu Wenruo 12becfe122 btrfs-progs: Introduce btrfs_raid_array and related infrastructures
As part of the effort to unify code and behavior between btrfs-progs and
kernel, copy the btrfs_raid_array from kernel to btrfs-progs.

So later we can use the btrfs_raid_array[] to get needed raid info other
than manually do if-else branches.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-24 01:37:17 +01:00
Qu Wenruo e9097ad593 btrfs-progs: Refactor parameter of BTRFS_MAX_DEVS() from root to fs_info
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-24 01:37:17 +01:00
Gu Jinxiang 26072f584d btrfs-progs: Use fs_info instead of root for BTRFS_LEAF_DATA_SIZE
Do a cleanup. Also make it consistent with kernel.  Use fs_info instead
of root for BTRFS_LEAF_DATA_SIZE, since maybe in some situation we do
not know root, but just know fs_info.

Signed-off-by: Gu Jinxiang <gujx@cn.fujitsu.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-02-02 16:01:56 +01:00
Rosen Penev e4df433b8a btrfs-progs: treewide: Replace strerror(errno) with %m.
As btrfs is specific to Linux, %m can be used instead of strerror(errno)
in format strings. This has some size reduction benefits for embedded
systems.

glibc, musl, and uclibc-ng all support %m as a modifier to printf.
A quick glance at the BIONIC libc source indicates that it has
support for %m as well. BSDs and Windows do not but I do believe
them to be beyond the scope of btrfs-progs.

Compiled sizes on Ubuntu 16.04:

Before:
3916512 btrfs
233688  libbtrfs.so.0.1
4899    bcp
2367672 btrfs-convert
2208488 btrfs-corrupt-block
13302   btrfs-debugfs
2152160 btrfs-debug-tree
2136024 btrfs-find-root
2287592 btrfs-image
2144600 btrfs-map-logical
2130760 btrfs-select-super
2152608 btrfstune
2131760 btrfs-zero-log
2277752 mkfs.btrfs
9166    show-blocks

After:
3908744 btrfs
233256  libbtrfs.so.0.1
4899    bcp
2366560 btrfs-convert
2207432 btrfs-corrupt-block
13302   btrfs-debugfs
2151104 btrfs-debug-tree
2134968 btrfs-find-root
2281864 btrfs-image
2143536 btrfs-map-logical
2129704 btrfs-select-super
2151552 btrfstune
2130696 btrfs-zero-log
2276272 mkfs.btrfs
9166    show-blocks

Total savings: 23928 (24 kilo)bytes

Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-31 15:14:03 +01:00
Qu Wenruo d7f906c23a btrfs-progs: volumes: Remove unnecessary parameters when allocating device extent
@chunk_tree and @chunk_objectid of device extent is fixed to
BTRFS_CHUNK_TREE_OBJECTID and BTRFS_FIRST_CHUNK_TREE_OBJECTID
respectively.

There is no need to pass them as parameter explicitly.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-31 15:14:02 +01:00
Qu Wenruo a784003783 btrfs-progs: volumes: Remove unnecessary trans parameter
Remove @trans parameter for find_free_dev_extent_start() and its
callers.

The function itself is doing read-only tree search, no use of
transaction.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-31 15:14:02 +01:00
Qu Wenruo 1c9b58256a btrfs-progs: volumes: Make find_free_dev_extent_start static
The function is not used by anyone else outside of volumes.c, make it
static.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Su Yue <suy.fnst@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-31 15:14:02 +01:00
Nikolay Borisov 8075fd4da1 btrfs-progs: Replace usage of list_for_each with list_for_each_entry
There are a couple of places where instead of the more succinct
list_for_each_entry the code uses list_for_each. This results in
slightly more code with no additional benefit as well as no
coherent pattern. This patch makes the code uniform. No functional
changes.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Reviewed-by: Qu Wenruo <wqu@suse.com>
[ remove unused variable in uuid_search ]
Signed-off-by: David Sterba <dsterba@suse.com>
2018-01-31 15:14:01 +01:00
Qu Wenruo 06f56db9cb btrfs-progs: rescue: Introduce fix-device-size
Introduce new subcommand 'fix-device-size' to the rescue group, to fix
device size alignment-related problems.

Especially for people unable to mount their fs with super::total_bytes
mismatch, this tool will fix the problems and let the mount continue.

Reported-by: Asif Youssuff <yoasif@gmail.com>
Reported-by: Rich Rauenzahn <rrauenza@gmail.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
2017-11-14 15:59:00 +01:00
Qu Wenruo a450dc4bfe btrfs-progs: Introduce function to fix super block total bytes
Recent kernel (starting from v4.6) will refuse to mount if super block
total bytes is smaller than all devices' size.

This makes end user unable to do anything to their otherwise quite
healthy fs.

To fix such problem, introduce repair function to fix it on an unmounted
filesystem.

Reported-by: Asif Youssuff <yoasif@gmail.com>
Reported-by: Rich Rauenzahn <rrauenza@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-11-14 15:59:00 +01:00
Qu Wenruo ea9cd9df2b btrfs-progs: Introduce function to fix unaligned device size
Recent kernel introduced alignment check for dev item, however older
kernel doesn't align device size when adding new device or shrinking
existing device.

This makes noisy kernel warning every time when any DEV_ITEM gets updated.

Introduce function to fix device size on an unmounted filesystem.

Reported-by: Asif Youssuff <yoasif@gmail.com>
Reported-by: Rich Rauenzahn <rrauenza@gmail.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-11-14 15:59:00 +01:00
Qu Wenruo 66e485873c btrfs-progs: mkfs: avoid BUG_ON for chunk allocation when ENOSPC happens
When passing directory larger than block device using --rootdir
parameter, we get the following backtrace:

------
extent-tree.c:2693: btrfs_reserve_extent: BUG_ON `ret` triggered, value -28
./mkfs.btrfs(+0x1a05d)[0x557939e6b05d]
./mkfs.btrfs(btrfs_reserve_extent+0xb5a)[0x557939e710c8]
./mkfs.btrfs(+0xb0b6)[0x557939e5c0b6]
./mkfs.btrfs(main+0x15d5)[0x557939e5de04]
/usr/lib/libc.so.6(__libc_start_main+0xea)[0x7f83b101af6a]
./mkfs.btrfs(_start+0x2a)[0x557939e5af5a]
------

Nothing special, just BUG_ON() abusing from ancient code.

Fix them by using correct return.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Reviewed-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-11-14 15:59:00 +01:00
Liu Bo eebdf02321 btrfs-progs: do not add stale device into fs_devices
If one of btrfs' devices was pulled out and we've replaced it with a
new one, then they have the same uuid.

If that device gets reconnected, 'btrfs filesystem show' will show the
stale one instead of the new one, but on the kernel side btrfs has a fix
not to include the stale one, this could confuse users as people may
monitor btrfs by running that command.

This does the similar thing to what kernel side has done.

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
Reviewed-by: Anand Jain <anand.jain@oracle.com>
[ format string adjustments ]
Signed-off-by: David Sterba <dsterba@suse.com>
2017-11-14 15:59:00 +01:00
Qu Wenruo 02b58051e6 btrfs-progs: Refactor find_next_chunk to get rid of parameter root and objectid
Function find_next_chunk() is used to find next chunk start position,
which should only do search on chunk tree and objectid is set to
BTRFS_FIRST_CHUNK_TREE_OBJECTID.

So refactor the parameter list to get rid of @root, which should be
obtained from fs_info->chunk_root, and @objectid, which is set to
BTRFS_FIRST_CHUNK_TREE_OBJECTID.

Signed-off-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-25 15:17:48 +02:00
David Sterba 2030f49751 btrfs-progs: drop blocksize argument from btrfs_find_create_tree_block
Metadata blocks are always nodesize. When reading the
superblock::sys_array, the actual size of data is fixed to 4k and
smaller than nodesize, but otherwise everything works as before.

Signed-off-by: David Sterba <dsterba@suse.com>
2017-09-08 16:15:05 +02:00
Nikolay Borisov 960870cc57 btrfs-progs: Use already defined BTRFS_BLOCK_GROUP_PROFILE_MASK
Instead of opencoding it. No functional changes.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-08-24 19:06:54 +02:00
Justin Maggard e2fd79c243 btrfs-progs: Fix an infinite loop in btrfs_next_bg
I've run into a couple filesystems where btrfs-find-root would spin
indefinitely.

If the first cache extent start location is 0, we end up in an infinite
loop in btrfs_next_bg().  Fix it by checking for that situation, and
jumping to the next bg if necessary.

Fixes: e2e0dae9 (btrfs-progs: volume: Fix a bug causing btrfs-find-root to skip first chunk)
Signed-off-by: Justin Maggard <jmaggard@netgear.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-20 17:43:43 +02:00
Qu Wenruo 0544aafcbf btrfs-progs: Refactor chunk creation functions to use btrfs_fs_info
4 functions are involved in this refactor: btrfs_make_block_group()
btrfs_make_block_groups(), btrfs_alloc_chunk, btrfs_alloc_data_chunk().

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:54:16 +02:00
Qu Wenruo 9b3959d72a btrfs-progs: Refactor btrfs_add_device() to use btrfs_fs_info
BTW, there is a duplicated definition of btrfs_add_device() in
volumes.h, also remove it.

Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:53:54 +02:00
Qu Wenruo 7a36a1216b btrfs-progs: Refactor btrfs_chunk_readonly to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:53:50 +02:00
Qu Wenruo 37ddab31fd btrfs-progs: Refactor btrfs_add_system_chunk to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:53:44 +02:00
Qu Wenruo 5b4c9ccce9 btrfs-progs: Refactor btrfs_read_sys_array/chunk_tree to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:53:41 +02:00
Qu Wenruo de5d0cea24 btrfs-progs: Refactor btrfs_find_device to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:53:36 +02:00
Qu Wenruo 505639ee62 btrfs-progs: Refactor btrfs_check_chunk_valid to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:53:33 +02:00
Qu Wenruo 824a300ec8 btrfs-progs: Refactor btrfs_next_bg and its callers to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:52:37 +02:00
Qu Wenruo a30579b1a7 btrfs-progs: Refactor btrfs_num_copies to use btrfs_fs_info
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2017-07-12 17:52:15 +02:00