Those callbacks are called with the media_device.graph_mutex held.
Add a note about that, as the code called by those notifiers should
not be touching in the mutex.
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Right now, the lock schema for media_device struct is messy,
since sometimes, it is protected via a spin lock, while, for
media graph traversal, it is protected by a mutex.
Solve this conflict by always using a mutex.
As a side effect, this prevents a bug when the media notifiers
is called at atomic context, while running the notifier callback:
BUG: sleeping function called from invalid context at mm/slub.c:1289
in_atomic(): 1, irqs_disabled(): 0, pid: 3479, name: modprobe
4 locks held by modprobe/3479:
#0: (&dev->mutex){......}, at: [<ffffffff81ce8933>] __driver_attach+0xa3/0x160
#1: (&dev->mutex){......}, at: [<ffffffff81ce8941>] __driver_attach+0xb1/0x160
#2: (register_mutex#5){+.+.+.}, at: [<ffffffffa10596c7>] usb_audio_probe+0x257/0x1c90 [snd_usb_audio]
#3: (&(&mdev->lock)->rlock){+.+.+.}, at: [<ffffffffa0e6051b>] media_device_register_entity+0x1cb/0x700 [media]
CPU: 2 PID: 3479 Comm: modprobe Not tainted 4.5.0-rc3+ #49
Hardware name: /NUC5i7RYB, BIOS RYBDWi35.86A.0350.2015.0812.1722 08/12/2015
0000000000000000 ffff8803b3f6f288 ffffffff81933901 ffff8803c4bae000
ffff8803c4bae5c8 ffff8803b3f6f2b0 ffffffff811c6af5 ffff8803c4bae000
ffffffff8285d7f6 0000000000000509 ffff8803b3f6f2f0 ffffffff811c6ce5
Call Trace:
[<ffffffff81933901>] dump_stack+0x85/0xc4
[<ffffffff811c6af5>] ___might_sleep+0x245/0x3a0
[<ffffffff811c6ce5>] __might_sleep+0x95/0x1a0
[<ffffffff8155aade>] kmem_cache_alloc_trace+0x20e/0x300
[<ffffffffa0e66e3d>] ? media_add_link+0x4d/0x140 [media]
[<ffffffffa0e66e3d>] media_add_link+0x4d/0x140 [media]
[<ffffffffa0e69931>] media_create_pad_link+0xa1/0x600 [media]
[<ffffffffa0fe11b3>] au0828_media_graph_notify+0x173/0x360 [au0828]
[<ffffffffa0e68a6a>] ? media_gobj_create+0x1ba/0x480 [media]
[<ffffffffa0e606fb>] media_device_register_entity+0x3ab/0x700 [media]
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The comment is for the wrong function. Fix it.
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Re-create the graph walk object as needed in order to have one large enough
available for all entities in the graph.
This enumeration is used for pipeline power management in the future.
[mchehab@osg.samsung.com: fix documentation bug:
" warning: bad line: graph_mutex"]
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Fix these compiler warnings:
media-git/include/media/media-device.h: In function 'media_device_pci_init':
media-git/include/media/media-device.h:610:9: warning: 'return' with a value, in function returning void
return NULL;
^
media-git/include/media/media-device.h: In function '__media_device_usb_init':
media-git/include/media/media-device.h:618:9: warning: 'return' with a value, in function returning void
return NULL;
^
Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Add new fields to struct media_device to add enable_source, and
disable_source handlers, and source_priv to stash driver private
data that is used to run these handlers. The enable_source handler
finds source entity for the passed in entity and checks if it is
available. When link is found, it activates it. Disable source
handler deactivates the link.
Bridge driver is expected to implement and set these handlers.
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Add new interfaces to register and unregister entity_notify
hook to media device. These interfaces allow drivers to add
hooks to take appropriate actions when new entities get added
to a shared media device. For example, au0828 bridge driver
registers an entity_notify hook to create links as needed
between media graph nodes.
[mchehab@osg.samsung.com: simple comments should be /* and not /**]
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Right now, media_device_pci_init and media_device_usb_init does
media_device allocation internaly. That preents its usage when
the media_device struct is embedded on some other structure.
Move memory allocation outside it, to make it more generic.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Those ancillary functions could be called even when compiled
without V4L2 support, as warned by ktest build robot:
All errors (new ones prefixed by >>):
>> ERROR: "__v4l2_mc_usb_media_device_init" [drivers/media/usb/dvb-usb/dvb-usb.ko] undefined!
>> ERROR: "__v4l2_mc_usb_media_device_init" [drivers/media/usb/dvb-usb-v2/dvb_usb_v2.ko] undefined!
>> ERROR: "__v4l2_mc_usb_media_device_init" [drivers/media/usb/au0828/au0828.ko] undefined!
Also, there's nothing there that are specific to V4L2. So, move
those ancillary functions to MC core.
No functional changes. Just function rename.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
On USB drivers, the dev struct is usually filled with the USB
device. That would mean that the name of the driver specified
by media_device.dev.driver.name would be "usb", instead of the
name of the actual driver that created the media entity.
Add an optional field at the internal struct to allow drivers
to override the driver name.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Instead of using one u32 counter per type for object IDs, use
just one counter. With such change, it makes sense to simplify
the debug logs too.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Several parameters added by the media_ent_enum patches
were declared with wrong argument names:
include/media/media-device.h:333: warning: No description found for parameter 'entity_internal_idx_max'
include/media/media-device.h:354: warning: No description found for parameter 'ent_enum'
include/media/media-device.h:354: warning: Excess function parameter 'e' description in 'media_entity_enum_init'
include/media/media-device.h:333: warning: No description found for parameter 'entity_internal_idx_max'
include/media/media-device.h:354: warning: No description found for parameter 'ent_enum'
include/media/media-device.h:354: warning: Excess function parameter 'e' description in 'media_entity_enum_init'
include/media/media-entity.h:397: warning: No description found for parameter 'ent_enum'
include/media/media-entity.h:397: warning: Excess function parameter 'e' description in 'media_entity_enum_zero'
include/media/media-entity.h:409: warning: No description found for parameter 'ent_enum'
include/media/media-entity.h:409: warning: Excess function parameter 'e' description in 'media_entity_enum_set'
include/media/media-entity.h:424: warning: No description found for parameter 'ent_enum'
include/media/media-entity.h:424: warning: Excess function parameter 'e' description in 'media_entity_enum_clear'
include/media/media-entity.h:441: warning: No description found for parameter 'ent_enum'
include/media/media-entity.h:441: warning: Excess function parameter 'e' description in 'media_entity_enum_test'
include/media/media-entity.h:458: warning: No description found for parameter 'ent_enum'
include/media/media-entity.h:458: warning: Excess function parameter 'e' description in 'media_entity_enum_test_and_set'
include/media/media-entity.h:474: warning: No description found for parameter 'ent_enum'
include/media/media-entity.h:474: warning: Excess function parameter 'e' description in 'media_entity_enum_empty'
include/media/media-entity.h:474: warning: Excess function parameter 'entity' description in 'media_entity_enum_empty'
include/media/media-entity.h:489: warning: No description found for parameter 'ent_enum1'
include/media/media-entity.h:489: warning: No description found for parameter 'ent_enum2'
include/media/media-entity.h:489: warning: Excess function parameter 'e' description in 'media_entity_enum_intersects'
include/media/media-entity.h:489: warning: Excess function parameter 'f' description in 'media_entity_enum_intersects'
Fix them.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Some exported functions were still documented at the .c file,
instead of documenting at the .h one.
Move the documentation to the right place, as we only use headers
at media device-drivers.xml DocBook.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
This is useful in e.g. knowing whether certain operations have already
been performed for an entity. The users include the framework itself (for
graph walking) and a number of drivers.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The internal index can be used internally by the framework in order to keep
track of entities for a purpose or another. The internal index is constant
while it's registered to a media device, but the same index may be re-used
once the entity having that index is unregistered.
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The media device node is registered and so made visible to user-space
before entities are registered and links created which means that the
media graph obtained by user-space could be only partially enumerated
if that happens too early before all the graph has been created.
To avoid this race condition, split the media init and registration
in separate functions and only register the media device node when
all the pending subdevices have been registered, either explicitly
by the driver or asynchronously using v4l2_async_register_subdev().
The media_device_register() had a check for drivers not filling dev
and model fields but all drivers in mainline set them and not doing
it will be a driver bug so change the function return to void and
add a BUG_ON() for dev being NULL instead.
Also, add a media_device_cleanup() function that will destroy the
graph_mutex that is initialized in media_device_init().
[mchehab@osg.samsung.com: Fix compilation if !CONFIG_MEDIA_CONTROLLER
and remove two warnings added by this changeset]
Suggested-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Acked-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Improve the documentation to let it clear that the entity function
must be initialized.
Suggested-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Now that we moved the content of the media-framework.txt into
the kerneldoc documentation, move the per-function specific
documentation to the corresponding functions and clean it up.
It would be good if we had already the markdown kernel-doc
patches merged upstream, but, while we doesn't have it,
let's make it less ugly at device-drivers.xml.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Instead of using a text file, let's put it together with the
struct documentation for the Media Controller.
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Some parts of the media controller are using mutexes while
others are using spin locks in order to protect creation
and removal of elements in the graph. That's wrong!
Also, the V4L2 core can remove graph elements on non-interactive
context:
BUG: sleeping function called from invalid context at include/linux/sched.h:2776
Fix it by always using spin locks for graph element addition/removal,
just like entity creation/removal is protected at media-device.c
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Every time a graph object is added or removed, the version
of the topology changes. That's a requirement for the new
MEDIA_IOC_G_TOPOLOGY, in order to allow userspace to know
that the topology has changed after a previous call to it.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The MC next gen API sends objects to userspace grouped by
their types.
In the case of pads and links, in order to improve performance
and have a simpler code, the best is to store them also on
separate linked lists at MC.
If we don't do that, we would need this kind of interaction
to send data to userspace (code is in structured english):
for each entity:
for each pad:
store pads
for each entity:
for each link:
store link
for each interface:
for each link:
store link
With would require one nested loop for pads and two nested
loops for links. By using separate linked lists for them,
just one loop would be enough.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Let's control the topology changes inside the graph_object. So, move the
addition and removal of interfaces/entities from the mdev lists to
media_gobj_init() and media_gobj_remove().
The main reason is that mdev should have lists for all object types, as
the new MC api will require to store objects in separate places.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Just like we do with entities, use a similar macro for the
interfaces loop.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
The media device should list the interface objects, so add a linked list
for those interfaces in struct media_device.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Interfaces are different than entities: they represent a
Kernel<->userspace interaction, while entities represent a
piece of hardware/firmware/software that executes a function.
Let's distinguish them by creating a separate structure to
store the interfaces.
Later patches should change the existing drivers and logic
to split the current interface embedded inside the entity
structure (device nodes) into a separate object of the graph.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Just like entities and pads, links also need to have unique
Object IDs along a given media controller.
So, let's add a media_gobj inside it and initialize
the object then a new link is created.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
PADs also need unique object IDs that won't conflict with
the entity object IDs.
The pad objects are currently created via media_entity_init()
and, once created, never change.
While this will likely change in the future in order to
support dynamic changes, for now we'll keep PADs as arrays
and initialize the media_gobj embedded structs when
registering the entity.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
As entities are graph objects, let's embed media_gobj
on it. That ensures an unique ID for entities that can be
global along the entire media controller.
For now, we'll keep the already existing entity ID. Such
field need to be dropped at some point, but for now, let's
not do this, to avoid needing to review all drivers and
the userspace apps.
Acked-by: Hans Verkuil <hans.verkuil@cisco.com>
Tested-by: Javier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Change to define Media Controller API when CONFIG_MEDIA_CONTROLLER
is enabled. Define stubs for CONFIG_MEDIA_CONTROLLER disabled case.
This will help avoid drivers needing to enclose Media Controller
code within ifdef CONFIG_MEDIA_CONTROLLER block.
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Add new media controller API to allocate media device as a
device resource. When a media device is created on the main
struct device which is the parent device for the interface
device, it will be available to all drivers associated with
that interface. For example, if a usb media device driver
creates the media device on the main struct device which is
common for all the drivers that control the media device,
including the non-media ALSA driver, media controller API
can be used to share access to the resources on the media
device. This new interface provides the above described
feature. A second interface that finds and returns the media
device is added to allow drivers to find the media device
created by any of the drivers associated with the device.
Signed-off-by: Shuah Khan <shuahkh@osg.samsung.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
mdev->fops->owner is actually the owner of the very same module which
implements media_device_register(), so it can't be unloaded anyway. Instead,
use THIS_MODULE through a macro as does video_register_device().
Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <m.chehab@samsung.com>
Currently the media device link_notify callback is invoked before the
actual change of state of a link when the link is being enabled, and
after the actual change of state when the link is being disabled.
This doesn't allow a media device driver to perform any operations
on a full graph before a link is disabled, as well as performing
any tasks on a modified graph right after a link's state is changed.
This patch modifies signature of the link_notify callback. This
callback is now called always before and after a link's state change.
To distinguish the notifications a 'notification' argument is added
to the link_notify callback: MEDIA_DEV_NOTIFY_PRE_LINK_CH indicates
notification before link's state change and
MEDIA_DEV_NOTIFY_POST_LINK_CH corresponds to a notification after
link flags change.
[mchehab@redhat.com: whitespace cleanups]
Signed-off-by: Sylwester Nawrocki <s.nawrocki@samsung.com>
Signed-off-by: Kyungmin Park <kyungmin.park@samsung.com>
Acked-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Sakari Ailus <sakari.ailus@iki.fi>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
The <linux/device.h> header includes a lot of stuff, and
it in turn gets a lot of use just for the basic "struct device"
which appears so often.
Clean up the users as follows:
1) For those headers only needing "struct device" as a pointer
in fcn args, replace the include with exactly that.
2) For headers not really using anything from device.h, simply
delete the include altogether.
3) For headers relying on getting device.h implicitly before
being included themselves, now explicitly include device.h
4) For files in which doing #1 or #2 uncovers an implicit
dependency on some other header, fix by explicitly adding
the required header(s).
Any C files that were implicitly relying on device.h to be
present have already been dealt with in advance.
Total removals from #1 and #2: 51. Total additions coming
from #3: 9. Total other implicit dependencies from #4: 7.
As of 3.3-rc1, there were 110, so a net removal of 42 gives
about a 38% reduction in device.h presence in include/*
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Create the following ioctl and implement it at the media device level to
setup links.
- MEDIA_IOC_SETUP_LINK: Modify the properties of a given link
The only property that can currently be modified is the ENABLED link
flag to enable/disable a link. Links marked with the IMMUTABLE link flag
can not be enabled or disabled.
Enabling or disabling a link has effects on entities' use count. Those
changes are automatically propagated through the graph.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Stanimir Varbanov <svarbanov@mm-sol.com>
Signed-off-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Due to the wide differences between drivers regarding power management
needs, the media controller does not implement power management.
However, the media_entity structure includes a use_count field that
media drivers can use to track the number of users of every entity for
power management needs.
The use_count field is owned by media drivers and must not be touched by
entity drivers. Access to the field must be protected by the media
device graph_mutex lock.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
As video hardware pipelines become increasingly complex and
configurable, the current hardware description through v4l2 subdevices
reaches its limits. In addition to enumerating and configuring
subdevices, video camera drivers need a way to discover and modify at
runtime how those subdevices are connected. This is done through new
elements called entities, pads and links.
An entity is a basic media hardware building block. It can correspond to
a large variety of logical blocks such as physical hardware devices
(CMOS sensor for instance), logical hardware devices (a building block
in a System-on-Chip image processing pipeline), DMA channels or physical
connectors.
A pad is a connection endpoint through which an entity can interact with
other entities. Data (not restricted to video) produced by an entity
flows from the entity's output to one or more entity inputs. Pads should
not be confused with physical pins at chip boundaries.
A link is a point-to-point oriented connection between two pads, either
on the same entity or on different entities. Data flows from a source
pad to a sink pad.
Links are stored in the source entity. To make backwards graph walk
faster, a copy of all links is also stored in the sink entity. The copy
is known as a backlink and is only used to help graph traversal.
The entity API is made of three functions:
- media_entity_init() initializes an entity. The caller must provide an
array of pads as well as an estimated number of links. The links array
is allocated dynamically and will be reallocated if it grows beyond the
initial estimate.
- media_entity_cleanup() frees resources allocated for an entity. It
must be called during the cleanup phase after unregistering the entity
and before freeing it.
- media_entity_create_link() creates a link between two entities. An
entry in the link array of each entity is allocated and stores pointers
to source and sink pads.
When a media device is unregistered, all its entities are unregistered
automatically.
The code is based on Hans Verkuil <hverkuil@xs4all.nl> initial work.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Sakari Ailus <sakari.ailus@iki.fi>
Acked-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
The media_device structure abstracts functions common to all kind of
media devices (v4l2, dvb, alsa, ...). It manages media entities and
offers a userspace API to discover and configure the media device
internal topology.
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: Hans Verkuil <hverkuil@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab@redhat.com>
Mauro Carvalho Chehab