linux/block
Ming Lei 2a5a24aa83 scsi: blk-mq: Return budget token from .get_budget callback
SCSI uses a global atomic variable to track queue depth for each
LUN/request queue.

This doesn't scale well when there are lots of CPU cores and the disk is
very fast. It has been observed that IOPS is affected a lot by tracking
queue depth via sdev->device_busy in the I/O path.

Return budget token from .get_budget callback. The budget token can be
passed to driver so that we can replace the atomic variable with
sbitmap_queue and alleviate the scaling problems that way.

Link: https://lore.kernel.org/r/20210122023317.687987-9-ming.lei@redhat.com
Cc: Omar Sandoval <osandov@fb.com>
Cc: Kashyap Desai <kashyap.desai@broadcom.com>
Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com>
Cc: Ewan D. Milne <emilne@redhat.com>
Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2021-03-04 17:36:59 -05:00
..
partitions for-5.12/block-2021-02-17 2021-02-21 11:02:48 -08:00
Kconfig blk-wbt: Remove obsolete multiqueue I/O scheduling comment 2020-09-01 16:49:26 -06:00
Kconfig.iosched treewide: replace '---help---' in Kconfig files with 'help' 2020-06-14 01:57:21 +09:00
Makefile blk-mq: merge blk-softirq.c into blk-mq.c 2020-06-24 09:15:56 -06:00
badblocks.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
bfq-cgroup.c bfq: fix blkio cgroup leakage v4 2020-08-18 07:48:08 -07:00
bfq-iosched.c block: get rid of the trace rq insert wrapper 2021-02-22 06:37:41 -07:00
bfq-iosched.h block, bfq: make waker-queue detection more robust 2021-01-25 14:18:37 -07:00
bfq-wf2q.c bfq: don't duplicate code for different paths 2021-01-24 18:20:47 -07:00
bio-integrity.c block: use bi_max_vecs to find the bvec pool 2021-02-08 08:33:16 -07:00
bio.c for-5.12/block-2021-02-17 2021-02-21 11:02:48 -08:00
blk-cgroup-rwstat.c blk-cgroup: separate out blkg_rwstat under CONFIG_BLK_CGROUP_RWSTAT 2019-11-07 12:28:13 -07:00
blk-cgroup-rwstat.h blk-cgroup: separate out blkg_rwstat under CONFIG_BLK_CGROUP_RWSTAT 2019-11-07 12:28:13 -07:00
blk-cgroup.c for-5.12/block-2021-02-17 2021-02-21 11:02:48 -08:00
blk-core.c block: get rid of the trace rq insert wrapper 2021-02-22 06:37:41 -07:00
blk-crypto-fallback.c block-crypto-fallback: use a bio_set for splitting bios 2021-02-24 08:55:00 -07:00
blk-crypto-internal.h block: make blk_crypto_rq_bio_prep() able to fail 2020-10-05 10:47:43 -06:00
blk-crypto.c dm: support key eviction from keyslot managers of underlying devices 2021-02-11 09:45:25 -05:00
blk-exec.c block: drop removed argument from kernel-doc of blk_execute_rq() 2021-01-29 07:43:29 -07:00
blk-flush.c block: use an on-stack bio in blkdev_issue_flush 2021-01-27 09:51:48 -07:00
blk-integrity.c block: remove the unused blk_integrity_merge_bio export 2020-10-06 07:29:53 -06:00
blk-ioc.c block: remove retry loop in ioc_release_fn() 2020-07-16 10:22:15 -06:00
blk-iocost.c blk-iocost: fix NULL iocg deref from racing against initialization 2021-01-05 11:33:32 -07:00
blk-iolatency.c block: Remove redundant 'return' statement 2020-10-08 07:59:48 -06:00
blk-lib.c block: move the start_sect field to struct block_device 2020-12-01 14:53:40 -07:00
blk-map.c block: Add bio_max_segs 2021-02-26 15:49:51 -07:00
blk-merge.c block: store a block_device pointer in struct bio 2021-01-24 18:17:20 -07:00
blk-mq-cpumap.c blk-mq: remove the calling of local_memory_node() 2020-10-20 07:08:17 -06:00
blk-mq-debugfs-zoned.c block: Cleanup license notice 2019-01-17 21:21:40 -07:00
blk-mq-debugfs.c block-5.11-2021-01-10 2021-01-10 12:53:08 -08:00
blk-mq-debugfs.h blk-mq: no need to check return value of debugfs_create functions 2019-06-13 03:00:30 -06:00
blk-mq-pci.c block: Fix blk_mq_*_map_queues() kernel-doc headers 2019-05-31 15:12:34 -06:00
blk-mq-rdma.c block: Fix blk_mq_*_map_queues() kernel-doc headers 2019-05-31 15:12:34 -06:00
blk-mq-sched.c scsi: blk-mq: Return budget token from .get_budget callback 2021-03-04 17:36:59 -05:00
blk-mq-sched.h block: get rid of the trace rq insert wrapper 2021-02-22 06:37:41 -07:00
blk-mq-sysfs.c blk-mq: move cancel of hctx->run_work to the front of blk_exit_queue 2020-10-09 12:46:28 -06:00
blk-mq-tag.c block-mq: fix comments in blk_mq_queue_tag_busy_iter 2020-09-29 08:11:00 -06:00
blk-mq-tag.h blk-mq: Relocate hctx_may_queue() 2020-09-03 15:20:47 -06:00
blk-mq-virtio.c blk-mq: Fix typo in comment 2020-03-17 20:55:21 +01:00
blk-mq.c scsi: blk-mq: Return budget token from .get_budget callback 2021-03-04 17:36:59 -05:00
blk-mq.h scsi: blk-mq: Return budget token from .get_budget callback 2021-03-04 17:36:59 -05:00
blk-pm.c scsi: block: Fix a race in the runtime power management code 2020-12-09 11:41:41 -05:00
blk-pm.h block: Remove unused blk_pm_*() function definitions 2021-02-22 06:33:48 -07:00
blk-rq-qos.c Revert "blk-rq-qos: remove redundant finish_wait to rq_qos_wait." 2020-07-15 09:33:37 -06:00
blk-rq-qos.h blk-rq-qos: fix first node deletion of rq_qos_del() 2019-10-15 10:13:13 -06:00
blk-settings.c blk-settings: align max_sectors on "logical_block_size" boundary 2021-02-23 19:25:36 -07:00
blk-stat.c blk-stat: make q->stats->lock irqsafe 2020-09-01 16:48:46 -06:00
blk-stat.h block: deactivate blk_stat timer in wbt_disable_default() 2018-12-12 06:47:51 -07:00
blk-sysfs.c block: fix potential IO hang when turning off io_poll 2021-02-22 06:40:02 -07:00
blk-throttle.c block: store a block_device pointer in struct bio 2021-01-24 18:17:20 -07:00
blk-timeout.c block: blk-timeout: delete duplicated word 2020-07-31 16:29:47 -06:00
blk-wbt.c blk: wbt: remove unused parameter from wbt_should_throttle 2021-01-26 13:13:00 -07:00
blk-wbt.h blk-wbt: remove wbt_update_limits 2020-05-29 16:30:39 -06:00
blk-zoned.c block: introduce blk_queue_clear_zone_settings() 2021-02-10 07:44:41 -07:00
blk.h block: introduce blk_queue_clear_zone_settings() 2021-02-10 07:44:41 -07:00
bounce.c block: memory allocations in bounce_clone_bio must not fail 2021-02-24 08:55:00 -07:00
bsg-lib.c block: drop double zeroing 2020-09-23 09:18:13 -06:00
bsg.c block: remove unnecessary argument from blk_execute_rq 2021-01-24 21:52:39 -07:00
cmdline-parser.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
elevator.c block: fix comment and add lockdep assert 2020-10-09 12:34:06 -06:00
genhd.c block: fix logging on capacity change 2021-02-23 19:28:51 -07:00
ioctl.c block: reopen the device in blkdev_reread_part 2021-02-23 19:23:22 -07:00
ioprio.c block: grant IOPRIO_CLASS_RT to CAP_SYS_NICE 2020-09-01 19:38:33 -06:00
keyslot-manager.c - Fix DM integrity's HMAC support to provide enhanced security of 2021-02-22 10:22:54 -08:00
kyber-iosched.c scsi: sbitmap: Move allocation hint into sbitmap 2021-03-04 17:36:59 -05:00
mq-deadline.c block: get rid of the trace rq insert wrapper 2021-02-22 06:37:41 -07:00
opal_proto.h block: sed-opal: Change the check condition for regular session validity 2020-03-12 08:00:10 -06:00
scsi_ioctl.c block: remove unnecessary argument from blk_execute_rq 2021-01-24 21:52:39 -07:00
sed-opal.c block: sed-opal: Change the check condition for regular session validity 2020-03-12 08:00:10 -06:00
t10-pi.c block: Allow t10-pi to be modular 2020-01-06 20:59:04 -07:00