linux/drivers/gpu/drm
Daniel Vetter 7741b547b6 drm/i915: Preallocate our mmu notifier workequeu to unbreak cpu hotplug deadlock
4.14-rc1 gained the fancy new cross-release support in lockdep, which
seems to have uncovered a few more rules about what is allowed and
isn't.

This one here seems to indicate that allocating a work-queue while
holding mmap_sem is a no-go, so let's try to preallocate it.

Of course another way to break this chain would be somewhere in the
cpu hotplug code, since this isn't the only trace we're finding now
which goes through msr_create_device.

Full lockdep splat:

======================================================
WARNING: possible circular locking dependency detected
4.14.0-rc1-CI-CI_DRM_3118+ #1 Tainted: G     U
------------------------------------------------------
prime_mmap/1551 is trying to acquire lock:
 (cpu_hotplug_lock.rw_sem){++++}, at: [<ffffffff8109dbb7>] apply_workqueue_attrs+0x17/0x50

but task is already holding lock:
 (&dev_priv->mm_lock){+.+.}, at: [<ffffffffa01a7b2a>] i915_gem_userptr_init__mmu_notifier+0x14a/0x270 [i915]

which lock already depends on the new lock.

the existing dependency chain (in reverse order) is:

-> #6 (&dev_priv->mm_lock){+.+.}:
       __lock_acquire+0x1420/0x15e0
       lock_acquire+0xb0/0x200
       __mutex_lock+0x86/0x9b0
       mutex_lock_nested+0x1b/0x20
       i915_gem_userptr_init__mmu_notifier+0x14a/0x270 [i915]
       i915_gem_userptr_ioctl+0x222/0x2c0 [i915]
       drm_ioctl_kernel+0x69/0xb0
       drm_ioctl+0x2f9/0x3d0
       do_vfs_ioctl+0x94/0x670
       SyS_ioctl+0x41/0x70
       entry_SYSCALL_64_fastpath+0x1c/0xb1

-> #5 (&mm->mmap_sem){++++}:
       __lock_acquire+0x1420/0x15e0
       lock_acquire+0xb0/0x200
       __might_fault+0x68/0x90
       _copy_to_user+0x23/0x70
       filldir+0xa5/0x120
       dcache_readdir+0xf9/0x170
       iterate_dir+0x69/0x1a0
       SyS_getdents+0xa5/0x140
       entry_SYSCALL_64_fastpath+0x1c/0xb1

-> #4 (&sb->s_type->i_mutex_key#5){++++}:
       down_write+0x3b/0x70
       handle_create+0xcb/0x1e0
       devtmpfsd+0x139/0x180
       kthread+0x152/0x190
       ret_from_fork+0x27/0x40

-> #3 ((complete)&req.done){+.+.}:
       __lock_acquire+0x1420/0x15e0
       lock_acquire+0xb0/0x200
       wait_for_common+0x58/0x210
       wait_for_completion+0x1d/0x20
       devtmpfs_create_node+0x13d/0x160
       device_add+0x5eb/0x620
       device_create_groups_vargs+0xe0/0xf0
       device_create+0x3a/0x40
       msr_device_create+0x2b/0x40
       cpuhp_invoke_callback+0xa3/0x840
       cpuhp_thread_fun+0x7a/0x150
       smpboot_thread_fn+0x18a/0x280
       kthread+0x152/0x190
       ret_from_fork+0x27/0x40

-> #2 (cpuhp_state){+.+.}:
       __lock_acquire+0x1420/0x15e0
       lock_acquire+0xb0/0x200
       cpuhp_issue_call+0x10b/0x170
       __cpuhp_setup_state_cpuslocked+0x134/0x2a0
       __cpuhp_setup_state+0x46/0x60
       page_writeback_init+0x43/0x67
       pagecache_init+0x3d/0x42
       start_kernel+0x3a8/0x3fc
       x86_64_start_reservations+0x2a/0x2c
       x86_64_start_kernel+0x6d/0x70
       verify_cpu+0x0/0xfb

-> #1 (cpuhp_state_mutex){+.+.}:
       __lock_acquire+0x1420/0x15e0
       lock_acquire+0xb0/0x200
       __mutex_lock+0x86/0x9b0
       mutex_lock_nested+0x1b/0x20
       __cpuhp_setup_state_cpuslocked+0x52/0x2a0
       __cpuhp_setup_state+0x46/0x60
       page_alloc_init+0x28/0x30
       start_kernel+0x145/0x3fc
       x86_64_start_reservations+0x2a/0x2c
       x86_64_start_kernel+0x6d/0x70
       verify_cpu+0x0/0xfb

-> #0 (cpu_hotplug_lock.rw_sem){++++}:
       check_prev_add+0x430/0x840
       __lock_acquire+0x1420/0x15e0
       lock_acquire+0xb0/0x200
       cpus_read_lock+0x3d/0xb0
       apply_workqueue_attrs+0x17/0x50
       __alloc_workqueue_key+0x1d8/0x4d9
       i915_gem_userptr_init__mmu_notifier+0x1fb/0x270 [i915]
       i915_gem_userptr_ioctl+0x222/0x2c0 [i915]
       drm_ioctl_kernel+0x69/0xb0
       drm_ioctl+0x2f9/0x3d0
       do_vfs_ioctl+0x94/0x670
       SyS_ioctl+0x41/0x70
       entry_SYSCALL_64_fastpath+0x1c/0xb1

other info that might help us debug this:

Chain exists of:
  cpu_hotplug_lock.rw_sem --> &mm->mmap_sem --> &dev_priv->mm_lock

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&dev_priv->mm_lock);
                               lock(&mm->mmap_sem);
                               lock(&dev_priv->mm_lock);
  lock(cpu_hotplug_lock.rw_sem);

 *** DEADLOCK ***

2 locks held by prime_mmap/1551:
 #0:  (&mm->mmap_sem){++++}, at: [<ffffffffa01a7b18>] i915_gem_userptr_init__mmu_notifier+0x138/0x270 [i915]
 #1:  (&dev_priv->mm_lock){+.+.}, at: [<ffffffffa01a7b2a>] i915_gem_userptr_init__mmu_notifier+0x14a/0x270 [i915]

stack backtrace:
CPU: 4 PID: 1551 Comm: prime_mmap Tainted: G     U          4.14.0-rc1-CI-CI_DRM_3118+ #1
Hardware name: Dell Inc. XPS 8300  /0Y2MRG, BIOS A06 10/17/2011
Call Trace:
 dump_stack+0x68/0x9f
 print_circular_bug+0x235/0x3c0
 ? lockdep_init_map_crosslock+0x20/0x20
 check_prev_add+0x430/0x840
 __lock_acquire+0x1420/0x15e0
 ? __lock_acquire+0x1420/0x15e0
 ? lockdep_init_map_crosslock+0x20/0x20
 lock_acquire+0xb0/0x200
 ? apply_workqueue_attrs+0x17/0x50
 cpus_read_lock+0x3d/0xb0
 ? apply_workqueue_attrs+0x17/0x50
 apply_workqueue_attrs+0x17/0x50
 __alloc_workqueue_key+0x1d8/0x4d9
 ? __lockdep_init_map+0x57/0x1c0
 i915_gem_userptr_init__mmu_notifier+0x1fb/0x270 [i915]
 i915_gem_userptr_ioctl+0x222/0x2c0 [i915]
 ? i915_gem_userptr_release+0x140/0x140 [i915]
 drm_ioctl_kernel+0x69/0xb0
 drm_ioctl+0x2f9/0x3d0
 ? i915_gem_userptr_release+0x140/0x140 [i915]
 ? __do_page_fault+0x2a4/0x570
 do_vfs_ioctl+0x94/0x670
 ? entry_SYSCALL_64_fastpath+0x5/0xb1
 ? __this_cpu_preempt_check+0x13/0x20
 ? trace_hardirqs_on_caller+0xe3/0x1b0
 SyS_ioctl+0x41/0x70
 entry_SYSCALL_64_fastpath+0x1c/0xb1
RIP: 0033:0x7fbb83c39587
RSP: 002b:00007fff188dc228 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: ffffffff81492963 RCX: 00007fbb83c39587
RDX: 00007fff188dc260 RSI: 00000000c0186473 RDI: 0000000000000003
RBP: ffffc90001487f88 R08: 0000000000000000 R09: 00007fff188dc2ac
R10: 00007fbb83efcb58 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000003 R14: 00000000c0186473 R15: 00007fff188dc2ac
 ? __this_cpu_preempt_check+0x13/0x20

Note that this also has the minor benefit of slightly reducing the
critical section where we hold mmap_sem.

v2: Set ret correctly when we raced with another thread.

v3: Use Chris' diff. Attach the right lockdep splat.

v4: Repaint in Tvrtko's colors (aka don't report ENOMEM if we race and
some other thread managed to not also get an ENOMEM and successfully
install the mmu notifier. Note that the kernel guarantees that small
allocations succeed, so this never actually happens).

Cc: Chris Wilson <chris@chris-wilson.co.uk>
Cc: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Cc: Joonas Lahtinen <joonas.lahtinen@linux.intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Sasha Levin <alexander.levin@verizon.com>
Cc: Marta Lofstedt <marta.lofstedt@intel.com>
Cc: Tejun Heo <tj@kernel.org>
References: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_3180/shard-hsw3/igt@prime_mmap@test_userptr.html
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=102939
Reviewed-by: Tvrtko Ursulin <tvrtko.ursulin@intel.com>
Signed-off-by: Daniel Vetter <daniel.vetter@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20171009164401.16035-1-daniel.vetter@ffwll.ch
2017-10-10 12:57:03 +02:00
..
amd Merge branch 'drm-next-4.15' of git://people.freedesktop.org/~agd5f/linux into drm-next 2017-09-28 08:37:02 +10:00
arc drm/arc: Use drm_gem_fb_create() 2017-09-02 14:23:06 +02:00
arm drm/arm/mali: Use drm_gem_fb_create() 2017-08-27 19:29:46 +02:00
armada drm/armada: Use correct path to trace include 2017-09-05 11:11:20 +02:00
ast Merge tag 'drm-misc-next-2017-08-16' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-08-17 07:33:41 +10:00
atmel-hlcdc drm/atmel-hlcdc: Use drm_gem_fb_create() 2017-08-27 19:30:15 +02:00
bochs drm/bochs: Use the drm_driver.dumb_destroy default 2017-08-16 20:18:55 +02:00
bridge Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
cirrus drm/cirrus: Use the drm_driver.dumb_destroy default 2017-08-16 20:14:22 +02:00
etnaviv mm: treewide: remove GFP_TEMPORARY allocation flag 2017-09-13 18:53:16 -07:00
exynos Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
fsl-dcu drm: Nuke drm_atomic_helper_connector_dpms 2017-08-08 14:48:48 +02:00
gma500 Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
hisilicon drm: kirin: Add mode_valid logic to avoid mode clocks we can't generate 2017-08-29 05:20:35 +10:00
i2c drm: i2c: tda998x: constify i2c_device_id 2017-08-22 08:33:44 +02:00
i810 drm/pci: Deprecate drm_pci_init/exit completely 2017-06-20 10:41:03 +02:00
i915 drm/i915: Preallocate our mmu notifier workequeu to unbreak cpu hotplug deadlock 2017-10-10 12:57:03 +02:00
imx Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
lib mm: treewide: remove GFP_TEMPORARY allocation flag 2017-09-13 18:53:16 -07:00
mediatek drm/mediatek: hdmi: clean up drm_bridge_add call 2017-08-21 08:49:24 +05:30
meson drm/meson: Use .dumb_map_offset and .dumb_destroy defaults 2017-08-16 20:11:43 +02:00
mga Merge airlied/drm-next into drm-misc-next 2017-07-26 13:43:33 +02:00
mgag200 drm/mgag200: Use the drm_driver.dumb_destroy default 2017-08-16 20:18:22 +02:00
msm mm: treewide: remove GFP_TEMPORARY allocation flag 2017-09-13 18:53:16 -07:00
mxsfb drm/mxsfb: Use .dumb_map_offset and .dumb_destroy defaults 2017-08-16 20:12:19 +02:00
nouveau dmi: Mark all struct dmi_system_id instances const 2017-09-14 11:59:30 +02:00
omapdrm drm/omap: work-around for omap3 display enable 2017-08-23 12:22:12 +03:00
panel drm/panel: Add driver for Seiko 43WVF1G panel 2017-08-18 17:10:45 +02:00
pl111 drm/pl111: Add handling of Versatile platforms 2017-09-10 23:58:42 +02:00
qxl drm/qxl: Use the drm_driver.dumb_destroy default 2017-08-16 20:15:38 +02:00
r128 drm/pci: Deprecate drm_pci_init/exit completely 2017-06-20 10:41:03 +02:00
radeon Merge branch 'drm-next-4.15' of git://people.freedesktop.org/~agd5f/linux into drm-next 2017-09-28 08:37:02 +10:00
rcar-du Merge tag 'drm-misc-next-2017-08-08' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-08-10 10:47:33 +10:00
rockchip Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
savage drm/pci: Deprecate drm_pci_init/exit completely 2017-06-20 10:41:03 +02:00
selftests mm: treewide: remove GFP_TEMPORARY allocation flag 2017-09-13 18:53:16 -07:00
shmobile drm/shmobile: Use .dumb_map_offset and .dumb_destroy defaults 2017-07-29 13:57:33 +02:00
sis drm/pci: Deprecate drm_pci_init/exit completely 2017-06-20 10:41:03 +02:00
sti drm/sti: Use drm_gem_fb_create() 2017-08-27 19:30:49 +02:00
stm drm/stm: fix warning about multiplication in condition 2017-09-08 10:29:11 +02:00
sun4i Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
tdfx drm/pci: Deprecate drm_pci_init/exit completely 2017-06-20 10:41:03 +02:00
tegra drm/tegra: Changes for v4.14-rc1 2017-08-21 17:37:33 +10:00
tilcdc drm/tilcdc: Use drm_gem_fb_create() 2017-09-09 18:00:58 +02:00
tinydrm drm/tinydrm: Drop driver registered message 2017-09-16 14:07:59 +02:00
ttm Merge branch 'drm-next-4.15' of git://people.freedesktop.org/~agd5f/linux into drm-next 2017-09-28 08:37:02 +10:00
tve200 drm/tve200: Clean up panel bridging 2017-09-11 20:32:56 +02:00
udl Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2017-09-04 12:21:28 -07:00
vc4 drm/vc4: clean up error handling on devm_kzalloc failure 2017-09-08 11:06:54 -07:00
vgem drm/vgem: switch to drm_*_get(), drm_*_put() helpers 2017-08-11 11:41:43 -04:00
via drm/pci: Deprecate drm_pci_init/exit completely 2017-06-20 10:41:03 +02:00
virtio drm/ttm: make ttm_mem_type_manager_func debug more useful 2017-08-17 15:45:59 -04:00
vmwgfx main drm pull request for 4.14 merge window 2017-09-03 17:02:26 -07:00
zte drm/zte: Use drm_gem_fb_create() 2017-08-27 19:31:06 +02:00
Kconfig Merge branch 'drm-next-4.15' of git://people.freedesktop.org/~agd5f/linux into drm-next 2017-09-28 08:37:02 +10:00
Makefile Merge branch 'drm-next-4.15' of git://people.freedesktop.org/~agd5f/linux into drm-next 2017-09-28 08:37:02 +10:00
ati_pcigart.c
drm_agpsupport.c
drm_atomic.c Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
drm_atomic_helper.c Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
drm_auth.c
drm_blend.c mm: treewide: remove GFP_TEMPORARY allocation flag 2017-09-13 18:53:16 -07:00
drm_bridge.c drm/bridge: change return type of drm_bridge_add function 2017-08-21 08:51:53 +05:30
drm_bufs.c switch compat_drm_mapbufs() to drm_ioctl_kernel() 2017-07-04 13:16:26 -04:00
drm_cache.c
drm_color_mgmt.c drm: More links for gamma support helpers 2017-06-20 12:13:11 +02:00
drm_connector.c drm: Drop drm_get_link_status_name() 2017-09-11 15:53:33 +03:00
drm_context.c
drm_crtc.c drm/crtc: Convert setcrtc ioctl locking to interruptible. 2017-09-13 09:52:05 +02:00
drm_crtc_helper.c drm: Handle properties in the core for atomic drivers 2017-08-08 14:45:09 +02:00
drm_crtc_helper_internal.h drm: Add drm_{crtc/encoder/connector}_mode_valid() 2017-05-30 08:37:24 +02:00
drm_crtc_internal.h drm: Handle properties in the core for atomic drivers 2017-08-08 14:45:09 +02:00
drm_debugfs.c
drm_debugfs_crc.c drm/atomic: Prepare drm_modeset_lock infrastructure for interruptible waiting, v2. 2017-09-13 09:50:52 +02:00
drm_dma.c
drm_dp_aux_dev.c drm_dp_aux_dev: switch to read_iter/write_iter 2017-07-08 20:51:46 -04:00
drm_dp_dual_mode_helper.c mm: treewide: remove GFP_TEMPORARY allocation flag 2017-09-13 18:53:16 -07:00
drm_dp_helper.c drm/dp: Don't trust drm_dp_downstream_id() 2017-07-21 17:45:26 +03:00
drm_dp_mst_topology.c drm/dp/mst: Sideband message transaction to power up/down nodes 2017-09-11 16:03:57 +03:00
drm_drv.c drm: Clean up drm_dev_unplug 2017-08-11 10:49:21 +02:00
drm_dumb_buffers.c drm/dumb-buffers: Add defaults for .dumb_map_offset and .dumb_destroy 2017-07-29 13:51:44 +02:00
drm_edid.c drm: handle override and firmware EDID at drm_do_get_edid() level 2017-09-19 17:49:25 +03:00
drm_edid_load.c drm: add backwards compatibility support for drm_kms_helper.edid_firmware 2017-09-19 18:11:45 +03:00
drm_encoder.c
drm_encoder_slave.c
drm_fb_cma_helper.c drm/fb-cma-helper: Use drm_gem_framebuffer_helper 2017-08-16 21:34:38 +02:00
drm_fb_helper.c drm/fb-helper: Avoid NULL ptr dereference in fb_set_suspend() 2017-09-02 14:37:59 +02:00
drm_file.c drm: Document device unplug infrastructure 2017-08-11 10:48:03 +02:00
drm_flip_work.c
drm_fourcc.c
drm_framebuffer.c Merge airlied/drm-next into drm-misc-next 2017-07-26 13:43:33 +02:00
drm_gem.c Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
drm_gem_cma_helper.c drm/gem-cma-helper: Remove drm_gem_cma_dumb_map_offset() 2017-08-16 20:21:24 +02:00
drm_gem_framebuffer_helper.c drm/gem-fb-helper: Use debug message on gem lookup failure 2017-09-16 14:06:53 +02:00
drm_global.c
drm_hashtab.c
drm_info.c
drm_internal.h drm/syncobj: Add a signal ioctl (v3) 2017-08-29 10:16:25 +10:00
drm_ioc32.c Merge airlied/drm-next into drm-misc-next 2017-07-26 13:43:33 +02:00
drm_ioctl.c drm/syncobj: Add a signal ioctl (v3) 2017-08-29 10:16:25 +10:00
drm_irq.c drm/doc: Polish irq helper documentation 2017-06-01 08:02:14 +02:00
drm_kms_helper_common.c drm: add backwards compatibility support for drm_kms_helper.edid_firmware 2017-09-19 18:11:45 +03:00
drm_legacy.h switch compat_drm_mapbufs() to drm_ioctl_kernel() 2017-07-04 13:16:26 -04:00
drm_lock.c
drm_memory.c
drm_mipi_dsi.c drm: Convert to using %pOF instead of full_name 2017-07-26 13:45:06 +02:00
drm_mm.c lib/interval_tree: fast overlap detection 2017-09-08 18:26:49 -07:00
drm_mode_config.c drm: Create a format/modifier blob 2017-08-01 17:50:06 +01:00
drm_mode_object.c drm: Make __drm_object_property_get_value() static 2017-09-11 15:53:33 +03:00
drm_modes.c drm/modes: Fix drm_mode_is_420_only() comment 2017-07-31 14:23:30 +02:00
drm_modeset_helper.c drm: Plumb modifiers through plane init 2017-08-01 17:50:06 +01:00
drm_modeset_lock.c drm/atomic: Prepare drm_modeset_lock infrastructure for interruptible waiting, v2. 2017-09-13 09:50:52 +02:00
drm_of.c drm: Convert to using %pOF instead of full_name 2017-07-26 13:45:06 +02:00
drm_panel.c
drm_pci.c drm/pci: Deprecate drm_pci_init/exit completely 2017-06-20 10:41:03 +02:00
drm_plane.c Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
drm_plane_helper.c
drm_prime.c
drm_print.c
drm_probe_helper.c drm: handle override and firmware EDID at drm_do_get_edid() level 2017-09-19 17:49:25 +03:00
drm_property.c drm: rename, adjust and export drm_atomic_replace_property_blob 2017-07-14 15:53:06 +02:00
drm_rect.c
drm_scatter.c
drm_scdc_helper.c Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
drm_simple_kms_helper.c drm: Plumb modifiers through plane init 2017-08-01 17:50:06 +01:00
drm_syncobj.c Merge tag 'drm-misc-next-2017-09-20' of git://anongit.freedesktop.org/git/drm-misc into drm-next 2017-09-28 05:46:15 +10:00
drm_sysfs.c
drm_trace.h drm: Use correct path to trace include 2017-09-05 11:11:18 +02:00
drm_trace_points.c
drm_vblank.c Merge airlied/drm-next into drm-misc-next 2017-07-26 13:43:33 +02:00
drm_vm.c Merge branch 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2017-09-04 12:21:28 -07:00
drm_vma_manager.c lib/interval_tree: fast overlap detection 2017-09-08 18:26:49 -07:00