linux/include
Qu Wenruo 1418bae1c2 btrfs: qgroup: Move reserved data accounting from btrfs_delayed_ref_head to btrfs_qgroup_extent_record
[BUG]
Btrfs/139 will fail with a high probability if the testing machine (VM)
has only 2G RAM.

Resulting the final write success while it should fail due to EDQUOT,
and the fs will have quota exceeding the limit by 16K.

The simplified reproducer will be: (needs a 2G ram VM)

  $ mkfs.btrfs -f $dev
  $ mount $dev $mnt

  $ btrfs subv create $mnt/subv
  $ btrfs quota enable $mnt
  $ btrfs quota rescan -w $mnt
  $ btrfs qgroup limit -e 1G $mnt/subv

  $ for i in $(seq -w  1 8); do
  	xfs_io -f -c "pwrite 0 128M" $mnt/subv/file_$i > /dev/null
  	echo "file $i written" > /dev/kmsg
    done
  $ sync
  $ btrfs qgroup show -pcre --raw $mnt

The last pwrite will not trigger EDQUOT and final 'qgroup show' will
show something like:

  qgroupid         rfer         excl     max_rfer     max_excl parent  child
  --------         ----         ----     --------     -------- ------  -----
  0/5             16384        16384         none         none ---     ---
  0/256      1073758208   1073758208         none   1073741824 ---     ---

And 1073758208 is larger than
  > 1073741824.

[CAUSE]
It's a bug in btrfs qgroup data reserved space management.

For quota limit, we must ensure that:
  reserved (data + metadata) + rfer/excl <= limit

Since rfer/excl is only updated at transaction commmit time, reserved
space needs to be taken special care.

One important part of reserved space is data, and for a new data extent
written to disk, we still need to take the reserved space until
rfer/excl numbers get updated.

Originally when an ordered extent finishes, we migrate the reserved
qgroup data space from extent_io tree to delayed ref head of the data
extent, expecting delayed ref will only be cleaned up at commit
transaction time.

However for small RAM machine, due to memory pressure dirty pages can be
flushed back to disk without committing a transaction.

The related events will be something like:

  file 1 written
  btrfs_finish_ordered_io: ino=258 ordered offset=0 len=54947840
  btrfs_finish_ordered_io: ino=258 ordered offset=54947840 len=5636096
  btrfs_finish_ordered_io: ino=258 ordered offset=61153280 len=57344
  btrfs_finish_ordered_io: ino=258 ordered offset=61210624 len=8192
  btrfs_finish_ordered_io: ino=258 ordered offset=60583936 len=569344
  cleanup_ref_head: num_bytes=54947840
  cleanup_ref_head: num_bytes=5636096
  cleanup_ref_head: num_bytes=569344
  cleanup_ref_head: num_bytes=57344
  cleanup_ref_head: num_bytes=8192
  ^^^^^^^^^^^^^^^^ This will free qgroup data reserved space
  file 2 written
  ...
  file 8 written
  cleanup_ref_head: num_bytes=8192
  ...
  btrfs_commit_transaction  <<< the only transaction committed during
				the test

When file 2 is written, we have already freed 128M reserved qgroup data
space for ino 258. Thus later write won't trigger EDQUOT.

This allows us to write more data beyond qgroup limit.

In my 2G ram VM, it could reach about 1.2G before hitting EDQUOT.

[FIX]
By moving reserved qgroup data space from btrfs_delayed_ref_head to
btrfs_qgroup_extent_record, we can ensure that reserved qgroup data
space won't be freed half way before commit transaction, thus fix the
problem.

Fixes: f64d5ca868 ("btrfs: delayed_ref: Add new function to record reserved space into delayed ref")
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2019-02-25 14:13:39 +01:00
..
acpi Merge branch 'x86-cleanups-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2018-12-26 17:03:51 -08:00
asm-generic Rename include/{uapi => }/asm-generic/shmparam.h really 2019-02-12 16:33:18 -08:00
clocksource
crypto crypto: skcipher - remove remnants of internal IV generators 2018-12-23 11:52:45 +08:00
drm Merge tag 'drm-intel-fixes-2019-01-11' of git://anongit.freedesktop.org/drm/drm-intel into drm-fixes 2019-01-11 10:26:21 +01:00
dt-bindings ARM: SoC fixes for linux-5.0 2019-02-08 16:23:41 -08:00
keys KEYS: user: Align the payload buffer 2019-02-22 10:11:19 -08:00
kvm KVM: arm/arm64: vgic: Make vgic_cpu->ap_list_lock a raw_spinlock 2019-01-24 15:09:09 +01:00
linux Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2019-02-24 09:28:26 -08:00
math-emu
media media fixes for v4.20-rc8 2018-12-25 13:11:30 -08:00
memory
misc
net phonet: fix building with clang 2019-02-21 16:23:56 -08:00
pcmcia
ras
rdma RDMA/device: Expose ib_device_try_get(() 2019-01-21 14:33:08 -07:00
scsi 4.21 merge window pull request 2018-12-28 14:57:10 -08:00
soc ARM: SoC driver updates 2018-12-31 17:32:35 -08:00
sound ALSA: compress: Fix stop handling on compressed capture streams 2019-02-05 22:01:41 +01:00
target scsi: target: consistently null-terminate t10_wwn strings 2018-12-07 21:54:33 -05:00
trace btrfs: qgroup: Move reserved data accounting from btrfs_delayed_ref_head to btrfs_qgroup_extent_record 2019-02-25 14:13:39 +01:00
uapi btrfs: introduce new ioctl to unregister a btrfs device 2019-02-25 14:13:30 +01:00
video drm/exynos: fimd: Make pixel blend mode configurable 2018-12-14 15:46:15 +09:00
xen arm64/xen: fix xen-swiotlb cache flushing 2019-01-23 22:14:56 +01:00