diff options
Diffstat (limited to 'Documentation/media/kapi')
| -rw-r--r-- | Documentation/media/kapi/dtv-core.rst | 132 | ||||
| -rw-r--r-- | Documentation/media/kapi/mc-core.rst | 263 | ||||
| -rw-r--r-- | Documentation/media/kapi/rc-core.rst | 14 | ||||
| -rw-r--r-- | Documentation/media/kapi/v4l2-controls.rst | 826 | ||||
| -rw-r--r-- | Documentation/media/kapi/v4l2-core.rst | 36 | ||||
| -rw-r--r-- | Documentation/media/kapi/v4l2-framework.rst | 1262 | ||||
| -rw-r--r-- | Documentation/media/kapi/videobuf.rst | 402 |
7 files changed, 2935 insertions, 0 deletions
diff --git a/Documentation/media/kapi/dtv-core.rst b/Documentation/media/kapi/dtv-core.rst new file mode 100644 index 000000000000..11da77e141ed --- /dev/null +++ b/Documentation/media/kapi/dtv-core.rst @@ -0,0 +1,132 @@ +Digital TV (DVB) devices +------------------------ + +Digital TV Common functions +--------------------------- + +.. kernel-doc:: drivers/media/dvb-core/dvb_math.h + +.. kernel-doc:: drivers/media/dvb-core/dvb_ringbuffer.h + +.. kernel-doc:: drivers/media/dvb-core/dvbdev.h + + + +.. kernel-doc:: drivers/media/dvb-core/dvb_math.h + :export: drivers/media/dvb-core/dvb_math.c + +.. kernel-doc:: drivers/media/dvb-core/dvbdev.h + :export: drivers/media/dvb-core/dvbdev.c + + + +Digital TV Frontend kABI +------------------------ + +Digital TV Frontend +~~~~~~~~~~~~~~~~~~~ + +The Digital TV Frontend kABI defines a driver-internal interface for +registering low-level, hardware specific driver to a hardware independent +frontend layer. It is only of interest for Digital TV device driver writers. +The header file for this API is named dvb_frontend.h and located in +drivers/media/dvb-core. + +Before using the Digital TV frontend core, the bridge driver should attach +the frontend demod, tuner and SEC devices and call +:cpp:func:`dvb_register_frontend()`, +in order to register the new frontend at the subsystem. At device +detach/removal, the bridge driver should call +:cpp:func:`dvb_unregister_frontend()` to +remove the frontend from the core and then :cpp:func:`dvb_frontend_detach()` +to free the memory allocated by the frontend drivers. + +The drivers should also call :cpp:func:`dvb_frontend_suspend()` as part of +their handler for the :c:type:`device_driver`.\ ``suspend()``, and +:cpp:func:`dvb_frontend_resume()` as +part of their handler for :c:type:`device_driver`.\ ``resume()``. + +A few other optional functions are provided to handle some special cases. + +.. kernel-doc:: drivers/media/dvb-core/dvb_frontend.h + + +Digital TV Demux kABI +--------------------- + +Digital TV Demux +~~~~~~~~~~~~~~~~ + +The Kernel Digital TV Demux kABI defines a driver-internal interface for +registering low-level, hardware specific driver to a hardware independent +demux layer. It is only of interest for Digital TV device driver writers. +The header file for this kABI is named demux.h and located in +drivers/media/dvb-core. + +The demux kABI should be implemented for each demux in the system. It is +used to select the TS source of a demux and to manage the demux resources. +When the demux client allocates a resource via the demux kABI, it receives +a pointer to the kABI of that resource. + +Each demux receives its TS input from a DVB front-end or from memory, as +set via this demux kABI. In a system with more than one front-end, the kABI +can be used to select one of the DVB front-ends as a TS source for a demux, +unless this is fixed in the HW platform. + +The demux kABI only controls front-ends regarding to their connections with +demuxes; the kABI used to set the other front-end parameters, such as +tuning, are devined via the Digital TV Frontend kABI. + +The functions that implement the abstract interface demux should be defined +static or module private and registered to the Demux core for external +access. It is not necessary to implement every function in the struct +&dmx_demux. For example, a demux interface might support Section filtering, +but not PES filtering. The kABI client is expected to check the value of any +function pointer before calling the function: the value of ``NULL`` means +that the function is not available. + +Whenever the functions of the demux API modify shared data, the +possibilities of lost update and race condition problems should be +addressed, e.g. by protecting parts of code with mutexes. + +Note that functions called from a bottom half context must not sleep. +Even a simple memory allocation without using ``GFP_ATOMIC`` can result in a +kernel thread being put to sleep if swapping is needed. For example, the +Linux Kernel calls the functions of a network device interface from a +bottom half context. Thus, if a demux kABI function is called from network +device code, the function must not sleep. + + + +Demux Callback API +------------------ + +Demux Callback +~~~~~~~~~~~~~~ + +This kernel-space API comprises the callback functions that deliver filtered +data to the demux client. Unlike the other DVB kABIs, these functions are +provided by the client and called from the demux code. + +The function pointers of this abstract interface are not packed into a +structure as in the other demux APIs, because the callback functions are +registered and used independent of each other. As an example, it is possible +for the API client to provide several callback functions for receiving TS +packets and no callbacks for PES packets or sections. + +The functions that implement the callback API need not be re-entrant: when +a demux driver calls one of these functions, the driver is not allowed to +call the function again before the original call returns. If a callback is +triggered by a hardware interrupt, it is recommended to use the Linux +bottom half mechanism or start a tasklet instead of making the callback +function call directly from a hardware interrupt. + +This mechanism is implemented by :cpp:func:`dmx_ts_cb()` and :cpp:func:`dmx_section_cb()` +callbacks. + +.. kernel-doc:: drivers/media/dvb-core/demux.h + +Digital TV Conditional Access kABI +---------------------------------- + +.. kernel-doc:: drivers/media/dvb-core/dvb_ca_en50221.h diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst new file mode 100644 index 000000000000..c1fe0d69207d --- /dev/null +++ b/Documentation/media/kapi/mc-core.rst @@ -0,0 +1,263 @@ +Media Controller devices +------------------------ + +Media Controller +~~~~~~~~~~~~~~~~ + +The media controller userspace API is documented in +:ref:`the Media Controller uAPI book <media_common>`. This document focus +on the kernel-side implementation of the media framework. + +Abstract media device model +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Discovering a device internal topology, and configuring it at runtime, is one +of the goals of the media framework. To achieve this, hardware devices are +modelled as an oriented graph of building blocks called entities connected +through pads. + +An entity is a basic media hardware building block. It can correspond to +a large variety of logical blocks such as physical hardware devices +(CMOS sensor for instance), logical hardware devices (a building block +in a System-on-Chip image processing pipeline), DMA channels or physical +connectors. + +A pad is a connection endpoint through which an entity can interact with +other entities. Data (not restricted to video) produced by an entity +flows from the entity's output to one or more entity inputs. Pads should +not be confused with physical pins at chip boundaries. + +A link is a point-to-point oriented connection between two pads, either +on the same entity or on different entities. Data flows from a source +pad to a sink pad. + +Media device +^^^^^^^^^^^^ + +A media device is represented by a :c:type:`struct media_device <media_device>` +instance, defined in ``include/media/media-device.h``. +Allocation of the structure is handled by the media device driver, usually by +embedding the :c:type:`media_device` instance in a larger driver-specific +structure. + +Drivers register media device instances by calling +:cpp:func:`__media_device_register()` via the macro ``media_device_register()`` +and unregistered by calling :cpp:func:`media_device_unregister()`. + +Entities +^^^^^^^^ + +Entities are represented by a :c:type:`struct media_entity <media_entity>` +instance, defined in ``include/media/media-entity.h``. The structure is usually +embedded into a higher-level structure, such as +:ref:`v4l2_subdev` or :ref:`video_device` +instances, although drivers can allocate entities directly. + +Drivers initialize entity pads by calling +:cpp:func:`media_entity_pads_init()`. + +Drivers register entities with a media device by calling +:cpp:func:`media_device_register_entity()` +and unregistred by calling +:cpp:func:`media_device_unregister_entity()`. + +Interfaces +^^^^^^^^^^ + +Interfaces are represented by a +:c:type:`struct media_interface <media_interface>` instance, defined in +``include/media/media-entity.h``. Currently, only one type of interface is +defined: a device node. Such interfaces are represented by a +:c:type:`struct media_intf_devnode <media_intf_devnode>`. + +Drivers initialize and create device node interfaces by calling +:cpp:func:`media_devnode_create()` +and remove them by calling: +:cpp:func:`media_devnode_remove()`. + +Pads +^^^^ +Pads are represented by a :c:type:`struct media_pad <media_pad>` instance, +defined in ``include/media/media-entity.h``. Each entity stores its pads in +a pads array managed by the entity driver. Drivers usually embed the array in +a driver-specific structure. + +Pads are identified by their entity and their 0-based index in the pads +array. + +Both information are stored in the :c:type:`struct media_pad`, making the +:c:type:`media_pad` pointer the canonical way to store and pass link references. + +Pads have flags that describe the pad capabilities and state. + +``MEDIA_PAD_FL_SINK`` indicates that the pad supports sinking data. +``MEDIA_PAD_FL_SOURCE`` indicates that the pad supports sourcing data. + +.. note:: + + One and only one of ``MEDIA_PAD_FL_SINK`` or ``MEDIA_PAD_FL_SOURCE`` must + be set for each pad. + +Links +^^^^^ + +Links are represented by a :c:type:`struct media_link <media_link>` instance, +defined in ``include/media/media-entity.h``. There are two types of links: + +**1. pad to pad links**: + +Associate two entities via their PADs. Each entity has a list that points +to all links originating at or targeting any of its pads. +A given link is thus stored twice, once in the source entity and once in +the target entity. + +Drivers create pad to pad links by calling: +:cpp:func:`media_create_pad_link()` and remove with +:cpp:func:`media_entity_remove_links()`. + +**2. interface to entity links**: + +Associate one interface to a Link. + +Drivers create interface to entity links by calling: +:cpp:func:`media_create_intf_link()` and remove with +:cpp:func:`media_remove_intf_links()`. + +.. note:: + + Links can only be created after having both ends already created. + +Links have flags that describe the link capabilities and state. The +valid values are described at :cpp:func:`media_create_pad_link()` and +:cpp:func:`media_create_intf_link()`. + +Graph traversal +^^^^^^^^^^^^^^^ + +The media framework provides APIs to iterate over entities in a graph. + +To iterate over all entities belonging to a media device, drivers can use +the media_device_for_each_entity macro, defined in +``include/media/media-device.h``. + +.. code-block:: c + + struct media_entity *entity; + + media_device_for_each_entity(entity, mdev) { + // entity will point to each entity in turn + ... + } + +Drivers might also need to iterate over all entities in a graph that can be +reached only through enabled links starting at a given entity. The media +framework provides a depth-first graph traversal API for that purpose. + +.. note:: + + Graphs with cycles (whether directed or undirected) are **NOT** + supported by the graph traversal API. To prevent infinite loops, the graph + traversal code limits the maximum depth to ``MEDIA_ENTITY_ENUM_MAX_DEPTH``, + currently defined as 16. + +Drivers initiate a graph traversal by calling +:cpp:func:`media_entity_graph_walk_start()` + +The graph structure, provided by the caller, is initialized to start graph +traversal at the given entity. + +Drivers can then retrieve the next entity by calling +:cpp:func:`media_entity_graph_walk_next()` + +When the graph traversal is complete the function will return ``NULL``. + +Graph traversal can be interrupted at any moment. No cleanup function call +is required and the graph structure can be freed normally. + +Helper functions can be used to find a link between two given pads, or a pad +connected to another pad through an enabled link +:cpp:func:`media_entity_find_link()` and +:cpp:func:`media_entity_remote_pad()`. + +Use count and power handling +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Due to the wide differences between drivers regarding power management +needs, the media controller does not implement power management. However, +the :c:type:`struct media_entity <media_entity>` includes a ``use_count`` +field that media drivers +can use to track the number of users of every entity for power management +needs. + +The :c:type:`media_entity<media_entity>`.\ ``use_count`` field is owned by +media drivers and must not be +touched by entity drivers. Access to the field must be protected by the +:c:type:`media_device`.\ ``graph_mutex`` lock. + +Links setup +^^^^^^^^^^^ + +Link properties can be modified at runtime by calling +:cpp:func:`media_entity_setup_link()`. + +Pipelines and media streams +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When starting streaming, drivers must notify all entities in the pipeline to +prevent link states from being modified during streaming by calling +:cpp:func:`media_entity_pipeline_start()`. + +The function will mark all entities connected to the given entity through +enabled links, either directly or indirectly, as streaming. + +The :c:type:`struct media_pipeline <media_pipeline>` instance pointed to by +the pipe argument will be stored in every entity in the pipeline. +Drivers should embed the :c:type:`struct media_pipeline <media_pipeline>` +in higher-level pipeline structures and can then access the +pipeline through the :c:type:`struct media_entity <media_entity>` +pipe field. + +Calls to :cpp:func:`media_entity_pipeline_start()` can be nested. +The pipeline pointer must be identical for all nested calls to the function. + +:cpp:func:`media_entity_pipeline_start()` may return an error. In that case, +it will clean up any of the changes it did by itself. + +When stopping the stream, drivers must notify the entities with +:cpp:func:`media_entity_pipeline_stop()`. + +If multiple calls to :cpp:func:`media_entity_pipeline_start()` have been +made the same number of :cpp:func:`media_entity_pipeline_stop()` calls +are required to stop streaming. +The :c:type:`media_entity`.\ ``pipe`` field is reset to ``NULL`` on the last +nested stop call. + +Link configuration will fail with ``-EBUSY`` by default if either end of the +link is a streaming entity. Links that can be modified while streaming must +be marked with the ``MEDIA_LNK_FL_DYNAMIC`` flag. + +If other operations need to be disallowed on streaming entities (such as +changing entities configuration parameters) drivers can explicitly check the +media_entity stream_count field to find out if an entity is streaming. This +operation must be done with the media_device graph_mutex held. + +Link validation +^^^^^^^^^^^^^^^ + +Link validation is performed by :cpp:func:`media_entity_pipeline_start()` +for any entity which has sink pads in the pipeline. The +:c:type:`media_entity`.\ ``link_validate()`` callback is used for that +purpose. In ``link_validate()`` callback, entity driver should check +that the properties of the source pad of the connected entity and its own +sink pad match. It is up to the type of the entity (and in the end, the +properties of the hardware) what matching actually means. + +Subsystems should facilitate link validation by providing subsystem specific +helper functions to provide easy access for commonly needed information, and +in the end provide a way to use driver-specific callbacks. + +.. kernel-doc:: include/media/media-device.h + +.. kernel-doc:: include/media/media-devnode.h + +.. kernel-doc:: include/media/media-entity.h diff --git a/Documentation/media/kapi/rc-core.rst b/Documentation/media/kapi/rc-core.rst new file mode 100644 index 000000000000..a45895886257 --- /dev/null +++ b/Documentation/media/kapi/rc-core.rst @@ -0,0 +1,14 @@ +Remote Controller devices +------------------------- + +Remote Controller core +~~~~~~~~~~~~~~~~~~~~~~ + +.. kernel-doc:: include/media/rc-core.h + +.. kernel-doc:: include/media/rc-map.h + +LIRC +~~~~ + +.. kernel-doc:: include/media/lirc_dev.h diff --git a/Documentation/media/kapi/v4l2-controls.rst b/Documentation/media/kapi/v4l2-controls.rst new file mode 100644 index 000000000000..8ff9ee806042 --- /dev/null +++ b/Documentation/media/kapi/v4l2-controls.rst @@ -0,0 +1,826 @@ +V4L2 Controls +============= + +Introduction +------------ + +The V4L2 control API seems simple enough, but quickly becomes very hard to +implement correctly in drivers. But much of the code needed to handle controls +is actually not driver specific and can be moved to the V4L core framework. + +After all, the only part that a driver developer is interested in is: + +1) How do I add a control? +2) How do I set the control's value? (i.e. s_ctrl) + +And occasionally: + +3) How do I get the control's value? (i.e. g_volatile_ctrl) +4) How do I validate the user's proposed control value? (i.e. try_ctrl) + +All the rest is something that can be done centrally. + +The control framework was created in order to implement all the rules of the +V4L2 specification with respect to controls in a central place. And to make +life as easy as possible for the driver developer. + +Note that the control framework relies on the presence of a struct v4l2_device +for V4L2 drivers and struct v4l2_subdev for sub-device drivers. + + +Objects in the framework +------------------------ + +There are two main objects: + +The v4l2_ctrl object describes the control properties and keeps track of the +control's value (both the current value and the proposed new value). + +v4l2_ctrl_handler is the object that keeps track of controls. It maintains a +list of v4l2_ctrl objects that it owns and another list of references to +controls, possibly to controls owned by other handlers. + + +Basic usage for V4L2 and sub-device drivers +------------------------------------------- + +1) Prepare the driver: + +1.1) Add the handler to your driver's top-level struct: + +.. code-block:: none + + struct foo_dev { + ... + struct v4l2_ctrl_handler ctrl_handler; + ... + }; + + struct foo_dev *foo; + +1.2) Initialize the handler: + +.. code-block:: none + + v4l2_ctrl_handler_init(&foo->ctrl_handler, nr_of_controls); + +The second argument is a hint telling the function how many controls this +handler is expected to handle. It will allocate a hashtable based on this +information. It is a hint only. + +1.3) Hook the control handler into the driver: + +1.3.1) For V4L2 drivers do this: + +.. code-block:: none + + struct foo_dev { + ... + struct v4l2_device v4l2_dev; + ... + struct v4l2_ctrl_handler ctrl_handler; + ... + }; + + foo->v4l2_dev.ctrl_handler = &foo->ctrl_handler; + +Where foo->v4l2_dev is of type struct v4l2_device. + +Finally, remove all control functions from your v4l2_ioctl_ops (if any): +vidioc_queryctrl, vidioc_query_ext_ctrl, vidioc_querymenu, vidioc_g_ctrl, +vidioc_s_ctrl, vidioc_g_ext_ctrls, vidioc_try_ext_ctrls and vidioc_s_ext_ctrls. +Those are now no longer needed. + +1.3.2) For sub-device drivers do this: + +.. code-block:: none + + struct foo_dev { + ... + struct v4l2_subdev sd; + ... + struct v4l2_ctrl_handler ctrl_handler; + ... + }; + + foo->sd.ctrl_handler = &foo->ctrl_handler; + +Where foo->sd is of type struct v4l2_subdev. + +And set all core control ops in your struct v4l2_subdev_core_ops to these +helpers: + +.. code-block:: none + + .queryctrl = v4l2_subdev_queryctrl, + .querymenu = v4l2_subdev_querymenu, + .g_ctrl = v4l2_subdev_g_ctrl, + .s_ctrl = v4l2_subdev_s_ctrl, + .g_ext_ctrls = v4l2_subdev_g_ext_ctrls, + .try_ext_ctrls = v4l2_subdev_try_ext_ctrls, + .s_ext_ctrls = v4l2_subdev_s_ext_ctrls, + +Note: this is a temporary solution only. Once all V4L2 drivers that depend +on subdev drivers are converted to the control framework these helpers will +no longer be needed. + +1.4) Clean up the handler at the end: + +.. code-block:: none + + v4l2_ctrl_handler_free(&foo->ctrl_handler); + + +2) Add controls: + +You add non-menu controls by calling v4l2_ctrl_new_std: + +.. code-block:: none + + struct v4l2_ctrl *v4l2_ctrl_new_std(struct v4l2_ctrl_handler *hdl, + const struct v4l2_ctrl_ops *ops, + u32 id, s32 min, s32 max, u32 step, s32 def); + +Menu and integer menu controls are added by calling v4l2_ctrl_new_std_menu: + +.. code-block:: none + + struct v4l2_ctrl *v4l2_ctrl_new_std_menu(struct v4l2_ctrl_handler *hdl, + const struct v4l2_ctrl_ops *ops, + u32 id, s32 max, s32 skip_mask, s32 def); + +Menu controls with a driver specific menu are added by calling +v4l2_ctrl_new_std_menu_items: + +.. code-block:: none + + struct v4l2_ctrl *v4l2_ctrl_new_std_menu_items( + struct v4l2_ctrl_handler *hdl, + const struct v4l2_ctrl_ops *ops, u32 id, s32 max, + s32 skip_mask, s32 def, const char * const *qmenu); + +Integer menu controls with a driver specific menu can be added by calling +v4l2_ctrl_new_int_menu: + +.. code-block:: none + + struct v4l2_ctrl *v4l2_ctrl_new_int_menu(struct v4l2_ctrl_handler *hdl, + const struct v4l2_ctrl_ops *ops, + u32 id, s32 max, s32 def, const s64 *qmenu_int); + +These functions are typically called right after the v4l2_ctrl_handler_init: + +.. code-block:: none + + static const s64 exp_bias_qmenu[] = { + -2, -1, 0, 1, 2 + }; + static const char * const test_pattern[] = { + "Disabled", + "Vertical Bars", + "Solid Black", + "Solid White", + }; + + v4l2_ctrl_handler_init(&foo->ctrl_handler, nr_of_controls); + v4l2_ctrl_new_std(&foo->ctrl_handler, &foo_ctrl_ops, + V4L2_CID_BRIGHTNESS, 0, 255, 1, 128); + v4l2_ctrl_new_std(&foo->ctrl_handler, &foo_ctrl_ops, + V4L2_CID_CONTRAST, 0, 255, 1, 128); + v4l2_ctrl_new_std_menu(&foo->ctrl_handler, &foo_ctrl_ops, + V4L2_CID_POWER_LINE_FREQUENCY, + V4L2_CID_POWER_LINE_FREQUENCY_60HZ, 0, + V4L2_CID_POWER_LINE_FREQUENCY_DISABLED); + v4l2_ctrl_new_int_menu(&foo->ctrl_handler, &foo_ctrl_ops, + V4L2_CID_EXPOSURE_BIAS, + ARRAY_SIZE(exp_bias_qmenu) - 1, + ARRAY_SIZE(exp_bias_qmenu) / 2 - 1, + exp_bias_qmenu); + v4l2_ctrl_new_std_menu_items(&foo->ctrl_handler, &foo_ctrl_ops, + V4L2_CID_TEST_PATTERN, ARRAY_SIZE(test_pattern) - 1, 0, + 0, test_pattern); + ... + if (foo->ctrl_handler.error) { + int err = foo->ctrl_handler.error; + + v4l2_ctrl_handler_free(&foo->ctrl_handler); + return err; + } + +The v4l2_ctrl_new_std function returns the v4l2_ctrl pointer to the new +control, but if you do not need to access the pointer outside the control ops, +then there is no need to store it. + +The v4l2_ctrl_new_std function will fill in most fields based on the control +ID except for the min, max, step and default values. These are passed in the +last four arguments. These values are driver specific while control attributes +like type, name, flags are all global. The control's current value will be set +to the default value. + +The v4l2_ctrl_new_std_menu function is very similar but it is used for menu +controls. There is no min argument since that is always 0 for menu controls, +and instead of a step there is a skip_mask argument: if bit X is 1, then menu +item X is skipped. + +The v4l2_ctrl_new_int_menu function creates a new standard integer menu +control with driver-specific items in the menu. It differs from +v4l2_ctrl_new_std_menu in that it doesn't have the mask argument and takes +as the last argument an array of signed 64-bit integers that form an exact +menu item list. + +The v4l2_ctrl_new_std_menu_items function is very similar to +v4l2_ctrl_new_std_menu but takes an extra parameter qmenu, which is the driver +specific menu for an otherwise standard menu control. A good example for this +control is the test pattern control for capture/display/sensors devices that +have the capability to generate test patterns. These test patterns are hardware +specific, so the contents of the menu will vary from device to device. + +Note that if something fails, the function will return NULL or an error and +set ctrl_handler->error to the error code. If ctrl_handler->error was already +set, then it will just return and do nothing. This is also true for +v4l2_ctrl_handler_init if it cannot allocate the internal data structure. + +This makes it easy to init the handler and just add all controls and only check +the error code at the end. Saves a lot of repetitive error checking. + +It is recommended to add controls in ascending control ID order: it will be +a bit faster that way. + +3) Optionally force initial control setup: + +.. code-block:: none + + v4l2_ctrl_handler_setup(&foo->ctrl_handler); + +This will call s_ctrl for all controls unconditionally. Effectively this +initializes the hardware to the default control values. It is recommended +that you do this as this ensures that both the internal data structures and +the hardware are in sync. + +4) Finally: implement the v4l2_ctrl_ops + +.. code-block:: none + + static const struct v4l2_ctrl_ops foo_ctrl_ops = { + .s_ctrl = foo_s_ctrl, + }; + +Usually all you need is s_ctrl: + +.. code-block:: none + + static int foo_s_ctrl(struct v4l2_ctrl *ctrl) + { + struct foo *state = container_of(ctrl->handler, struct foo, ctrl_handler); + + switch (ctrl->id) { + case V4L2_CID_BRIGHTNESS: + write_reg(0x123, ctrl->val); + break; + case V4L2_CID_CONTRAST: + write_reg(0x456, ctrl->val); + break; + } + return 0; + } + +The control ops are called with the v4l2_ctrl pointer as argument. +The new control value has already been validated, so all you need to do is +to actually update the hardware registers. + +You're done! And this is sufficient for most of the drivers we have. No need +to do any validation of control values, or implement QUERYCTRL, QUERY_EXT_CTRL +and QUERYMENU. And G/S_CTRL as well as G/TRY/S_EXT_CTRLS are automatically supported. + + +.. note:: + + The remainder sections deal with more advanced controls topics and scenarios. + In practice the basic usage as described above is sufficient for most drivers. + + +Inheriting Controls +------------------- + +When a sub-device is registered with a V4L2 driver by calling +v4l2_device_register_subdev() and the ctrl_handler fields of both v4l2_subdev +and v4l2_device are set, then the controls of the subdev will become +automatically available in the V4L2 driver as well. If the subdev driver +contains controls that already exist in the V4L2 driver, then those will be +skipped (so a V4L2 driver can always override a subdev control). + +What happens here is that v4l2_device_register_subdev() calls +v4l2_ctrl_add_handler() adding the controls of the subdev to the controls +of v4l2_device. + + +Accessing Control Values +------------------------ + +The following union is used inside the control framework to access control +values: + +.. code-block:: none + + union v4l2_ctrl_ptr { + s32 *p_s32; + s64 *p_s64; + char *p_char; + void *p; + }; + +The v4l2_ctrl struct contains these fields that can be used to access both +current and new values: + +.. code-block:: none + + s32 val; + struct { + s32 val; + } cur; + + + union v4l2_ctrl_ptr p_new; + union v4l2_ctrl_ptr p_cur; + +If the control has a simple s32 type type, then: + +.. code-block:: none + + &ctrl->val == ctrl->p_new.p_s32 + &ctrl->cur.val == ctrl->p_cur.p_s32 + +For all other types use ctrl->p_cur.p<something>. Basically the val +and cur.val fields can be considered an alias since these are used so often. + +Within the control ops you can freely use these. The val and cur.val speak for +themselves. The p_char pointers point to character buffers of length +ctrl->maximum + 1, and are always 0-terminated. + +Unless the control is marked volatile the p_cur field points to the the +current cached control value. When you create a new control this value is made +identical to the default value. After calling v4l2_ctrl_handler_setup() this +value is passed to the hardware. It is generally a good idea to call this +function. + +Whenever a new value is set that new value is automatically cached. This means +that most drivers do not need to implement the g_volatile_ctrl() op. The +exception is for controls that return a volatile register such as a signal +strength read-out that changes continuously. In that case you will need to +implement g_volatile_ctrl like this: + +.. code-block:: none + + static int foo_g_volatile_ctrl(struct v4l2_ctrl *ctrl) + { + switch (ctrl->id) { + case V4L2_CID_BRIGHTNESS: + ctrl->val = read_reg(0x123); + break; + } + } + +Note that you use the 'new value' union as well in g_volatile_ctrl. In general +controls that need to implement g_volatile_ctrl are read-only controls. If they +are not, a V4L2_EVENT_CTRL_CH_VALUE will not be generated when the control +changes. + +To mark a control as volatile you have to set V4L2_CTRL_FLAG_VOLATILE: + +.. code-block:: none + + ctrl = v4l2_ctrl_new_std(&sd->ctrl_handler, ...); + if (ctrl) + ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE; + +For try/s_ctrl the new values (i.e. as passed by the user) are filled in and +you can modify them in try_ctrl or set them in s_ctrl. The 'cur' union +contains the current value, which you can use (but not change!) as well. + +If s_ctrl returns 0 (OK), then the control framework will copy the new final +values to the 'cur' union. + +While in g_volatile/s/try_ctrl you can access the value of all controls owned +by the same handler since the handler's lock is held. If you need to access +the value of controls owned by other handlers, then you have to be very careful +not to introduce deadlocks. + +Outside of the control ops you have to go through to helper functions to get +or set a single control value safely in your driver: + +.. code-block:: none + + s32 v4l2_ctrl_g_ctrl(struct v4l2_ctrl *ctrl); + int v4l2_ctrl_s_ctrl(struct v4l2_ctrl *ctrl, s32 val); + +These functions go through the control framework just as VIDIOC_G/S_CTRL ioctls +do. Don't use these inside the control ops g_volatile/s/try_ctrl, though, that +will result in a deadlock since these helpers lock the handler as well. + +You can also take the handler lock yourself: + +.. code-block:: none + + mutex_lock(&state->ctrl_handler.lock); + pr_info("String value is '%s'\n", ctrl1->p_cur.p_char); + pr_info("Integer value is '%s'\n", ctrl2->cur.val); + mutex_unlock(&state->ctrl_handler.lock); + + +Menu Controls +------------- + +The v4l2_ctrl struct contains this union: + +.. code-block:: none + + union { + u32 step; + u32 menu_skip_mask; + }; + +For menu controls menu_skip_mask is used. What it does is that it allows you +to easily exclude certain menu items. This is used in the VIDIOC_QUERYMENU +implementation where you can return -EINVAL if a certain menu item is not +present. Note that VIDIOC_QUERYCTRL always returns a step value of 1 for +menu controls. + +A good example is the MPEG Audio Layer II Bitrate menu control where the +menu is a list of standardized possible bitrates. But in practice hardware +implementations will only support a subset of those. By setting the skip +mask you can tell the framework which menu items should be skipped. Setting +it to 0 means that all menu items are supported. + +You set this mask either through the v4l2_ctrl_config struct for a custom +control, or by calling v4l2_ctrl_new_std_menu(). + + +Custom Controls +--------------- + +Driver specific controls can be created using v4l2_ctrl_new_custom(): + +.. code-block:: none + + static const struct v4l2_ctrl_config ctrl_filter = { + .ops = &ctrl_custom_ops, + .id = V4L2_CID_MPEG_CX2341X_VIDEO_SPATIAL_FILTER, + .name = "Spatial Filter", + .type = V4L2_CTRL_TYPE_INTEGER, + .flags = V4L2_CTRL_FLAG_SLIDER, + .max = 15, + .step = 1, + }; + + ctrl = v4l2_ctrl_new_custom(&foo->ctrl_handler, &ctrl_filter, NULL); + +The last argument is the priv pointer which can be set to driver-specific +private data. + +The v4l2_ctrl_config struct also has a field to set the is_private flag. + +If the name field is not set, then the framework will assume this is a standard +control and will fill in the name, type and flags fields accordingly. + + +Active and Grabbed Controls +--------------------------- + +If you get more complex relationships between controls, then you may have to +activate and deactivate controls. For example, if the Chroma AGC control is +on, then the Chroma Gain control is inactive. That is, you may set it, but +the value will not be used by the hardware as long as the automatic gain +control is on. Typically user interfaces can disable such input fields. + +You can set the 'active' status using v4l2_ctrl_activate(). By default all +controls are active. Note that the framework does not check for this flag. +It is meant purely for GUIs. The function is typically called from within +s_ctrl. + +The other flag is the 'grabbed' flag. A grabbed control means that you cannot +change it because it is in use by some resource. Typical examples are MPEG +bitrate controls that cannot be changed while capturing is in progress. + +If a control is set to 'grabbed' using v4l2_ctrl_grab(), then the framework +will return -EBUSY if an attempt is made to set this control. The +v4l2_ctrl_grab() function is typically called from the driver when it +starts or stops streaming. + + +Control Clusters +---------------- + +By default all controls are independent from the others. But in more +complex scenarios you can get dependencies from one control to another. +In that case you need to 'cluster' them: + +.. code-block:: none + + struct foo { + struct v4l2_ctrl_handler ctrl_handler; + #define AUDIO_CL_VOLUME (0) + #define AUDIO_CL_MUTE (1) + struct v4l2_ctrl *audio_cluster[2]; + ... + }; + + state->audio_cluster[AUDIO_CL_VOLUME] = + v4l2_ctrl_new_std(&state->ctrl_handler, ...); + state->audio_cluster[AUDIO_CL_MUTE] = + v4l2_ctrl_new_std(&state->ctrl_handler, ...); + v4l2_ctrl_cluster(ARRAY_SIZE(state->audio_cluster), state->audio_cluster); + +From now on whenever one or more of the controls belonging to the same +cluster is set (or 'gotten', or 'tried'), only the control ops of the first +control ('volume' in this example) is called. You effectively create a new +composite control. Similar to how a 'struct' works in C. + +So when s_ctrl is called with V4L2_CID_AUDIO_VOLUME as argument, you should set +all two controls belonging to the audio_cluster: + +.. code-block:: none + + static int foo_s_ctrl(struct v4l2_ctrl *ctrl) + { + struct foo *state = container_of(ctrl->handler, struct foo, ctrl_handler); + + switch (ctrl->id) { + case V4L2_CID_AUDIO_VOLUME: { + struct v4l2_ctrl *mute = ctrl->cluster[AUDIO_CL_MUTE]; + + write_reg(0x123, mute->val ? 0 : ctrl->val); + break; + } + case V4L2_CID_CONTRAST: + write_reg(0x456, ctrl->val); + break; + } + return 0; + } + +In the example above the following are equivalent for the VOLUME case: + +.. code-block:: none + + ctrl == ctrl->cluster[AUDIO_CL_VOLUME] == state->audio_cluster[AUDIO_CL_VOLUME] + ctrl->cluster[AUDIO_CL_MUTE] == state->audio_cluster[AUDIO_CL_MUTE] + +In practice using cluster arrays like this becomes very tiresome. So instead +the following equivalent method is used: + +.. code-block:: none + + struct { + /* audio cluster */ + struct v4l2_ctrl *volume; + struct v4l2_ctrl *mute; + }; + +The anonymous struct is used to clearly 'cluster' these two control pointers, +but it serves no other purpose. The effect is the same as creating an +array with two control pointers. So you can just do: + +.. code-block:: none + + state->volume = v4l2_ctrl_new_std(&state->ctrl_handler, ...); + state->mute = v4l2_ctrl_new_std(&state->ctrl_handler, ...); + v4l2_ctrl_cluster(2, &state->volume); + +And in foo_s_ctrl you can use these pointers directly: state->mute->val. + +Note that controls in a cluster may be NULL. For example, if for some +reason mute was never added (because the hardware doesn't support that +particular feature), then mute will be NULL. So in that case we have a +cluster of 2 controls, of which only 1 is actually instantiated. The +only restriction is that the first control of the cluster must always be +present, since that is the 'master' control of the cluster. The master +control is the one that identifies the cluster and that provides the +pointer to the v4l2_ctrl_ops struct that is used for that cluster. + +Obviously, all controls in the cluster array must be initialized to either +a valid control or to NULL. + +In rare cases you might want to know which controls of a cluster actually +were set explicitly by the user. For this you can check the 'is_new' flag of +each control. For example, in the case of a volume/mute cluster the 'is_new' +flag of the mute control would be set if the user called VIDIOC_S_CTRL for +mute only. If the user would call VIDIOC_S_EXT_CTRLS for both mute and volume +controls, then the 'is_new' flag would be 1 for both controls. + +The 'is_new' flag is always 1 when called from v4l2_ctrl_handler_setup(). + + +Handling autogain/gain-type Controls with Auto Clusters +------------------------------------------------------- + +A common type of control cluster is one that handles 'auto-foo/foo'-type +controls. Typical examples are autogain/gain, autoexposure/exposure, +autowhitebalance/red balance/blue balance. In all cases you have one control +that determines whether another control is handled automatically by the hardware, +or whether it is under manual control from the user. + +If the cluster is in automatic mode, then the manual controls should be +marked inactive and volatile. When the volatile controls are read the +g_volatile_ctrl operation should return the value that the hardware's automatic +mode set up automatically. + +If the cluster is put in manual mode, then the manual controls should become +active again and the volatile flag is cleared (so g_volatile_ctrl is no longer +called while in manual mode). In addition just before switching to manual mode +the current values as determined by the auto mode are copied as the new manual +values. + +Finally the V4L2_CTRL_FLAG_UPDATE should be set for the auto control since +changing that control affects the control flags of the manual controls. + +In order to simplify this a special variation of v4l2_ctrl_cluster was +introduced: + +.. code-block:: none + + void v4l2_ctrl_auto_cluster(unsigned ncontrols, struct v4l2_ctrl **controls, + u8 manual_val, bool set_volatile); + +The first two arguments are identical to v4l2_ctrl_cluster. The third argument +tells the framework which value switches the cluster into manual mode. The +last argument will optionally set V4L2_CTRL_FLAG_VOLATILE for the non-auto controls. +If it is false, then the manual controls are never volatile. You would typically +use that if the hardware does not give you the option to read back to values as +determined by the auto mode (e.g. if autogain is on, the hardware doesn't allow +you to obtain the current gain value). + +The first control of the cluster is assumed to be the 'auto' control. + +Using this function will ensure that you don't need to handle all the complex +flag and volatile handling. + + +VIDIOC_LOG_STATUS Support +------------------------- + +This ioctl allow you to dump the current status of a driver to the kernel log. +The v4l2_ctrl_handler_log_status(ctrl_handler, prefix) can be used to dump the +value of the controls owned by the given handler to the log. You can supply a +prefix as well. If the prefix didn't end with a space, then ': ' will be added +for you. + + +Different Handlers for Different Video Nodes +-------------------------------------------- + +Usually the V4L2 driver has just one control handler that is global for +all video nodes. But you can also specify different control handlers for +different video nodes. You can do that by manually setting the ctrl_handler +field of struct video_device. + +That is no problem if there are no subdevs involved but if there are, then +you need to block the automatic merging of subdev controls to the global +control handler. You do that by simply setting the ctrl_handler field in +struct v4l2_device to NULL. Now v4l2_device_register_subdev() will no longer +merge subdev controls. + +After each subdev was added, you will then have to call v4l2_ctrl_add_handler +manually to add the subdev's control handler (sd->ctrl_handler) to the desired +control handler. This control handler may be specific to the video_device or +for a subset of video_device's. For example: the radio device nodes only have +audio controls, while the video and vbi device nodes share the same control +handler for the audio and video controls. + +If you want to have one handler (e.g. for a radio device node) have a subset +of another handler (e.g. for a video device node), then you should first add +the controls to the first handler, add the other controls to the second +handler and finally add the first handler to the second. For example: + +.. code-block:: none + + v4l2_ctrl_new_std(&radio_ctrl_handler, &radio_ops, V4L2_CID_AUDIO_VOLUME, ...); + v4l2_ctrl_new_std(&radio_ctrl_handler, &radio_ops, V4L2_CID_AUDIO_MUTE, ...); + v4l2_ctrl_new_std(&video_ctrl_handler, &video_ops, V4L2_CID_BRIGHTNESS, ...); + v4l2_ctrl_new_std(&video_ctrl_handler, &video_ops, V4L2_CID_CONTRAST, ...); + v4l2_ctrl_add_handler(&video_ctrl_handler, &radio_ctrl_handler, NULL); + +The last argument to v4l2_ctrl_add_handler() is a filter function that allows +you to filter which controls will be added. Set it to NULL if you want to add +all controls. + +Or you can add specific controls to a handler: + +.. code-block:: none + + volume = v4l2_ctrl_new_std(&video_ctrl_handler, &ops, V4L2_CID_AUDIO_VOLUME, ...); + v4l2_ctrl_new_std(&video_ctrl_handler, &ops, V4L2_CID_BRIGHTNESS, ...); + v4l2_ctrl_new_std(&video_ctrl_handler, &ops, V4L2_CID_CONTRAST, ...); + +What you should not do is make two identical controls for two handlers. +For example: + +.. code-block:: none + + v4l2_ctrl_new_std(&radio_ctrl_handler, &radio_ops, V4L2_CID_AUDIO_MUTE, ...); + v4l2_ctrl_new_std(&video_ctrl_handler, &video_ops, V4L2_CID_AUDIO_MUTE, ...); + +This would be bad since muting the radio would not change the video mute +control. The rule is to have one control for each hardware 'knob' that you +can twiddle. + + +Finding Controls +---------------- + +Normally you have created the controls yourself and you can store the struct +v4l2_ctrl pointer into your own struct. + +But sometimes you need to find a control from another handler that you do +not own. For example, if you have to find a volume control from a subdev. + +You can do that by calling v4l2_ctrl_find: + +.. code-block:: none + + struct v4l2_ctrl *volume; + + volume = v4l2_ctrl_find(sd->ctrl_handler, V4L2_CID_AUDIO_VOLUME); + +Since v4l2_ctrl_find will lock the handler you have to be careful where you +use it. For example, this is not a good idea: + +.. code-block:: none + + struct v4l2_ctrl_handler ctrl_handler; + + v4l2_ctrl_new_std(&ctrl_handler, &video_ops, V4L2_CID_BRIGHTNESS, ...); + v4l2_ctrl_new_std(&ctrl_handler, &video_ops, V4L2_CID_CONTRAST, ...); + +...and in video_ops.s_ctrl: + +.. code-block:: none + + case V4L2_CID_BRIGHTNESS: + contrast = v4l2_find_ctrl(&ctrl_handler, V4L2_CID_CONTRAST); + ... + +When s_ctrl is called by the framework the ctrl_handler.lock is already taken, so +attempting to find another control from the same handler will deadlock. + +It is recommended not to use this function from inside the control ops. + + +Inheriting Controls +------------------- + +When one control handler is added to another using v4l2_ctrl_add_handler, then +by default all controls from one are merged to the other. But a subdev might +have low-level controls that make sense for some advanced embedded system, but +not when it is used in consumer-level hardware. In that case you want to keep +those low-level controls local to the subdev. You can do this by simply +setting the 'is_private' flag of the control to 1: + +.. code-block:: none + + static const struct v4l2_ctrl_config ctrl_private = { + .ops = &ctrl_custom_ops, + .id = V4L2_CID_..., + .name = "Some Private Control", + .type = V4L2_CTRL_TYPE_INTEGER, + .max = 15, + .step = 1, + .is_private = 1, + }; + + ctrl = v4l2_ctrl_new_custom(&foo->ctrl_handler, &ctrl_private, NULL); + +These controls will now be skipped when v4l2_ctrl_add_handler is called. + + +V4L2_CTRL_TYPE_CTRL_CLASS Controls +---------------------------------- + +Controls of this type can be used by GUIs to get the name of the control class. +A fully featured GUI can make a dialog with multiple tabs with each tab +containing the controls belonging to a particular control class. The name of +each tab can be found by querying a special control with ID <control class | 1>. + +Drivers do not have to care about this. The framework will automatically add +a control of this type whenever the first control belonging to a new control +class is added. + + +Adding Notify Callbacks +----------------------- + +Sometimes the platform or bridge driver needs to be notified when a control +from a sub-device driver changes. You can set a notify callback by calling +this function: + +.. code-block:: none + + void v4l2_ctrl_notify(struct v4l2_ctrl *ctrl, + void (*notify)(struct v4l2_ctrl *ctrl, void *priv), void *priv); + +Whenever the give control changes value the notify callback will be called +with a pointer to the control and the priv pointer that was passed with +v4l2_ctrl_notify. Note that the control's handler lock is held when the +notify function is called. + +There can be only one notify function per control handler. Any attempt +to set another notify function will cause a WARN_ON. diff --git a/Documentation/media/kapi/v4l2-core.rst b/Documentation/media/kapi/v4l2-core.rst new file mode 100644 index 000000000000..a1b73e8d6795 --- /dev/null +++ b/Documentation/media/kapi/v4l2-core.rst @@ -0,0 +1,36 @@ +Video2Linux devices +------------------- + +.. kernel-doc:: include/media/tuner.h + +.. kernel-doc:: include/media/tuner-types.h + +.. kernel-doc:: include/media/tveeprom.h + +.. kernel-doc:: include/media/v4l2-async.h + +.. kernel-doc:: include/media/v4l2-ctrls.h + +.. kernel-doc:: include/media/v4l2-dv-timings.h + +.. kernel-doc:: include/media/v4l2-event.h + +.. kernel-doc:: include/media/v4l2-flash-led-class.h + +.. kernel-doc:: include/media/v4l2-mc.h + +.. kernel-doc:: include/media/v4l2-mediabus.h + +.. kernel-doc:: include/media/v4l2-mem2mem.h + +.. kernel-doc:: include/media/v4l2-of.h + +.. kernel-doc:: include/media/v4l2-rect.h + +.. kernel-doc:: include/media/v4l2-subdev.h + +.. kernel-doc:: include/media/videobuf2-core.h + +.. kernel-doc:: include/media/videobuf2-v4l2.h + +.. kernel-doc:: include/media/videobuf2-memops.h diff --git a/Documentation/media/kapi/v4l2-framework.rst b/Documentation/media/kapi/v4l2-framework.rst new file mode 100644 index 000000000000..740875ddfcec --- /dev/null +++ b/Documentation/media/kapi/v4l2-framework.rst @@ -0,0 +1,1262 @@ +Overview of the V4L2 driver framework +===================================== + +This text documents the various structures provided by the V4L2 framework and +their relationships. + + +Introduction +------------ + +The V4L2 drivers tend to be very complex due to the complexity of the +hardware: most devices have multiple ICs, export multiple device nodes in +/dev, and create also non-V4L2 devices such as DVB, ALSA, FB, I2C and input +(IR) devices. + +Especially the fact that V4L2 drivers have to setup supporting ICs to +do audio/video muxing/encoding/decoding makes it more complex than most. +Usually these ICs are connected to the main bridge driver through one or +more I2C busses, but other busses can also be used. Such devices are +called 'sub-devices'. + +For a long time the framework was limited to the video_device struct for +creating V4L device nodes and video_buf for handling the video buffers +(note that this document does not discuss the video_buf framework). + +This meant that all drivers had to do the setup of device instances and +connecting to sub-devices themselves. Some of this is quite complicated +to do right and many drivers never did do it correctly. + +There is also a lot of common code that could never be refactored due to +the lack of a framework. + +So this framework sets up the basic building blocks that all drivers +need and this same framework should make it much easier to refactor +common code into utility functions shared by all drivers. + +A good example to look at as a reference is the v4l2-pci-skeleton.c +source that is available in samples/v4l/. It is a skeleton driver for +a PCI capture card, and demonstrates how to use the V4L2 driver +framework. It can be used as a template for real PCI video capture driver. + +Structure of a driver +--------------------- + +All drivers have the following structure: + +1) A struct for each device instance containing the device state. + +2) A way of initializing and commanding sub-devices (if any). + +3) Creating V4L2 device nodes (/dev/videoX, /dev/vbiX and /dev/radioX) + and keeping track of device-node specific data. + +4) Filehandle-specific structs containing per-filehandle data; + +5) video buffer handling. + +This is a rough schematic of how it all relates: + +.. code-block:: none + + device instances + | + +-sub-device instances + | + \-V4L2 device nodes + | + \-filehandle instances + + +Structure of the framework +-------------------------- + +The framework closely resembles the driver structure: it has a v4l2_device +struct for the device instance data, a v4l2_subdev struct to refer to +sub-device instances, the video_device struct stores V4L2 device node data +and the v4l2_fh struct keeps track of filehandle instances. + +The V4L2 framework also optionally integrates with the media framework. If a +driver sets the struct v4l2_device mdev field, sub-devices and video nodes +will automatically appear in the media framework as entities. + + +struct v4l2_device +------------------ + +Each device instance is represented by a struct v4l2_device (v4l2-device.h). +Very simple devices can just allocate this struct, but most of the time you +would embed this struct inside a larger struct. + +You must register the device instance: + +.. code-block:: none + + v4l2_device_register(struct device *dev, struct v4l2_device *v4l2_dev); + +Registration will initialize the v4l2_device struct. If the dev->driver_data +field is NULL, it will be linked to v4l2_dev. + +Drivers that want integration with the media device framework need to set +dev->driver_data manually to point to the driver-specific device structure +that embed the struct v4l2_device instance. This is achieved by a +dev_set_drvdata() call before registering the V4L2 device instance. They must +also set the struct v4l2_device mdev field to point to a properly initialized +and registered media_device instance. + +If v4l2_dev->name is empty then it will be set to a value derived from dev +(driver name followed by the bus_id, to be precise). If you set it up before +calling v4l2_device_register then it will be untouched. If dev is NULL, then +you **must** setup v4l2_dev->name before calling v4l2_device_register. + +You can use v4l2_device_set_name() to set the name based on a driver name and +a driver-global atomic_t instance. This will generate names like ivtv0, ivtv1, +etc. If the name ends with a digit, then it will insert a dash: cx18-0, +cx18-1, etc. This function returns the instance number. + +The first 'dev' argument is normally the struct device pointer of a pci_dev, +usb_interface or platform_device. It is rare for dev to be NULL, but it happens +with ISA devices or when one device creates multiple PCI devices, thus making +it impossible to associate v4l2_dev with a particular parent. + +You can also supply a notify() callback that can be called by sub-devices to +notify you of events. Whether you need to set this depends on the sub-device. +Any notifications a sub-device supports must be defined in a header in +include/media/<subdevice>.h. + +You unregister with: + +.. code-block:: none + + v4l2_device_unregister(struct v4l2_device *v4l2_dev); + +If the dev->driver_data field points to v4l2_dev, it will be reset to NULL. +Unregistering will also automatically unregister all subdevs from the device. + +If you have a hotpluggable device (e.g. a USB device), then when a disconnect +happens the parent device becomes invalid. Since v4l2_device has a pointer to +that parent device it has to be cleared as well to mark that the parent is +gone. To do this call: + +.. code-block:: none + + v4l2_device_disconnect(struct v4l2_device *v4l2_dev); + +This does *not* unregister the subdevs, so you still need to call the +v4l2_device_unregister() function for that. If your driver is not hotpluggable, +then there is no need to call v4l2_device_disconnect(). + +Sometimes you need to iterate over all devices registered by a specific +driver. This is usually the case if multiple device drivers use the same +hardware. E.g. the ivtvfb driver is a framebuffer driver that uses the ivtv +hardware. The same is true for alsa drivers for example. + +You can iterate over all registered devices as follows: + +.. code-block:: none + + static int callback(struct device *dev, void *p) + { + struct v4l2_device *v4l2_dev = dev_get_drvdata(dev); + + /* test if this device was inited */ + if (v4l2_dev == NULL) + return 0; + ... + return 0; + } + + int iterate(void *p) + { + struct device_driver *drv; + int err; + + /* Find driver 'ivtv' on the PCI bus. + pci_bus_type is a global. For USB busses use usb_bus_type. */ + drv = driver_find("ivtv", &pci_bus_type); + /* iterate over all ivtv device instances */ + err = driver_for_each_device(drv, NULL, p, callback); + put_driver(drv); + return err; + } + +Sometimes you need to keep a running counter of the device instance. This is +commonly used to map a device instance to an index of a module option array. + +The recommended approach is as follows: + +.. code-block:: none + + static atomic_t drv_instance = ATOMIC_INIT(0); + + static int drv_probe(struct pci_dev *pdev, const struct pci_device_id *pci_id) + { + ... + state->instance = atomic_inc_return(&drv_instance) - 1; + } + +If you have multiple device nodes then it can be difficult to know when it is +safe to unregister v4l2_device for hotpluggable devices. For this purpose +v4l2_device has refcounting support. The refcount is increased whenever +video_register_device is called and it is decreased whenever that device node +is released. When the refcount reaches zero, then the v4l2_device release() +callback is called. You can do your final cleanup there. + +If other device nodes (e.g. ALSA) are created, then you can increase and +decrease the refcount manually as well by calling: + +.. code-block:: none + + void v4l2_device_get(struct v4l2_device *v4l2_dev); + +or: + +.. code-block:: none + + int v4l2_device_put(struct v4l2_device *v4l2_dev); + +Since the initial refcount is 1 you also need to call v4l2_device_put in the +disconnect() callback (for USB devices) or in the remove() callback (for e.g. +PCI devices), otherwise the refcount will never reach 0. + +struct v4l2_subdev +------------------ + +Many drivers need to communicate with sub-devices. These devices can do all +sort of tasks, but most commonly they handle audio and/or video muxing, +encoding or decoding. For webcams common sub-devices are sensors and camera +controllers. + +Usually these are I2C devices, but not necessarily. In order to provide the +driver with a consistent interface to these sub-devices the v4l2_subdev struct +(v4l2-subdev.h) was created. + +Each sub-device driver must have a v4l2_subdev struct. This struct can be +stand-alone for simple sub-devices or it might be embedded in a larger struct +if more state information needs to be stored. Usually there is a low-level +device struct (e.g. i2c_client) that contains the device data as setup +by the kernel. It is recommended to store that pointer in the private +data of v4l2_subdev using v4l2_set_subdevdata(). That makes it easy to go +from a v4l2_subdev to the actual low-level bus-specific device data. + +You also need a way to go from the low-level struct to v4l2_subdev. For the +common i2c_client struct the i2c_set_clientdata() call is used to store a +v4l2_subdev pointer, for other busses you may have to use other methods. + +Bridges might also need to store per-subdev private data, such as a pointer to +bridge-specific per-subdev private data. The v4l2_subdev structure provides +host private data for that purpose that can be accessed with +v4l2_get_subdev_hostdata() and v4l2_set_subdev_hostdata(). + +From the bridge driver perspective you load the sub-device module and somehow +obtain the v4l2_subdev pointer. For i2c devices this is easy: you call +i2c_get_clientdata(). For other busses something similar needs to be done. +Helper functions exists for sub-devices on an I2C bus that do most of this +tricky work for you. + +Each v4l2_subdev contains function pointers that sub-device drivers can +implement (or leave NULL if it is not applicable). Since sub-devices can do +so many different things and you do not want to end up with a huge ops struct +of which only a handful of ops are commonly implemented, the function pointers +are sorted according to category and each category has its own ops struct. + +The top-level ops struct contains pointers to the category ops structs, which +may be NULL if the subdev driver does not support anything from that category. + +It looks like this: + +.. code-block:: none + + struct v4l2_subdev_core_ops { + int (*log_status)(struct v4l2_subdev *sd); + int (*init)(struct v4l2_subdev *sd, u32 val); + ... + }; + + struct v4l2_subdev_tuner_ops { + ... + }; + + struct v4l2_subdev_audio_ops { + ... + }; + + struct v4l2_subdev_video_ops { + ... + }; + + struct v4l2_subdev_pad_ops { + ... + }; + + struct v4l2_subdev_ops { + const struct v4l2_subdev_core_ops *core; + const struct v4l2_subdev_tuner_ops *tuner; + const struct v4l2_subdev_audio_ops *audio; + const struct v4l2_subdev_video_ops *video; + const struct v4l2_subdev_pad_ops *video; + }; + +The core ops are common to all subdevs, the other categories are implemented +depending on the sub-device. E.g. a video device is unlikely to support the +audio ops and vice versa. + +This setup limits the number of function pointers while still making it easy +to add new ops and categories. + +A sub-device driver initializes the v4l2_subdev struct using: + +.. code-block:: none + + v4l2_subdev_init(sd, &ops); + +Afterwards you need to initialize subdev->name with a unique name and set the +module owner. This is done for you if you use the i2c helper functions. + +If integration with the media framework is needed, you must initialize the +media_entity struct embedded in the v4l2_subdev struct (entity field) by +calling media_entity_pads_init(), if the entity has pads: + +.. code-block:: none + + struct media_pad *pads = &my_sd->pads; + int err; + + err = media_entity_pads_init(&sd->entity, npads, pads); + +The pads array must have been previously initialized. There is no need to +manually set the struct media_entity function and name fields, but the +revision field must be initialized if needed. + +A reference to the entity will be automatically acquired/released when the +subdev device node (if any) is opened/closed. + +Don't forget to cleanup the media entity before the sub-device is destroyed: + +.. code-block:: none + + media_entity_cleanup(&sd->entity); + +If the subdev driver intends to process video and integrate with the media +framework, it must implement format related functionality using +v4l2_subdev_pad_ops instead of v4l2_subdev_video_ops. + +In that case, the subdev driver may set the link_validate field to provide +its own link validation function. The link validation function is called for +every link in the pipeline where both of the ends of the links are V4L2 +sub-devices. The driver is still responsible for validating the correctness +of the format configuration between sub-devices and video nodes. + +If link_validate op is not set, the default function +v4l2_subdev_link_validate_default() is used instead. This function ensures +that width, height and the media bus pixel code are equal on both source and +sink of the link. Subdev drivers are also free to use this function to +perform the checks mentioned above in addition to their own checks. + +There are currently two ways to register subdevices with the V4L2 core. The +first (traditional) possibility is to have subdevices registered by bridge +drivers. This can be done when the bridge driver has the complete information +about subdevices connected to it and knows exactly when to register them. This +is typically the case for internal subdevices, like video data processing units +within SoCs or complex PCI(e) boards, camera sensors in USB cameras or connected +to SoCs, which pass information about them to bridge drivers, usually in their +platform data. + +There are however also situations where subdevices have to be registered +asynchronously to bridge devices. An example of such a configuration is a Device +Tree based system where information about subdevices is made available to the +system independently from the bridge devices, e.g. when subdevices are defined +in DT as I2C device nodes. The API used in this second case is described further +below. + +Using one or the other registration method only affects the probing process, the +run-time bridge-subdevice interaction is in both cases the same. + +In the synchronous case a device (bridge) driver needs to register the +v4l2_subdev with the v4l2_device: + +.. code-block:: none + + int err = v4l2_device_register_subdev(v4l2_dev, sd); + +This can fail if the subdev module disappeared before it could be registered. +After this function was called successfully the subdev->dev field points to +the v4l2_device. + +If the v4l2_device parent device has a non-NULL mdev field, the sub-device +entity will be automatically registered with the media device. + +You can unregister a sub-device using: + +.. code-block:: none + + v4l2_device_unregister_subdev(sd); + +Afterwards the subdev module can be unloaded and sd->dev == NULL. + +You can call an ops function either directly: + +.. code-block:: none + + err = sd->ops->core->g_std(sd, &norm); + +but it is better and easier to use this macro: + +.. code-block:: none + + err = v4l2_subdev_call(sd, core, g_std, &norm); + +The macro will to the right NULL pointer checks and returns -ENODEV if subdev +is NULL, -ENOIOCTLCMD if either subdev->core or subdev->core->g_std is +NULL, or the actual result of the subdev->ops->core->g_std ops. + +It is also possible to call all or a subset of the sub-devices: + +.. code-block:: none + + v4l2_device_call_all(v4l2_dev, 0, core, g_std, &norm); + +Any subdev that does not support this ops is skipped and error results are +ignored. If you want to check for errors use this: + +.. code-block:: none + + err = v4l2_device_call_until_err(v4l2_dev, 0, core, g_std, &norm); + +Any error except -ENOIOCTLCMD will exit the loop with that error. If no +errors (except -ENOIOCTLCMD) occurred, then 0 is returned. + +The second argument to both calls is a group ID. If 0, then all subdevs are +called. If non-zero, then only those whose group ID match that value will +be called. Before a bridge driver registers a subdev it can set sd->grp_id +to whatever value it wants (it's 0 by default). This value is owned by the +bridge driver and the sub-device driver will never modify or use it. + +The group ID gives the bridge driver more control how callbacks are called. +For example, there may be multiple audio chips on a board, each capable of +changing the volume. But usually only one will actually be used when the +user want to change the volume. You can set the group ID for that subdev to +e.g. AUDIO_CONTROLLER and specify that as the group ID value when calling +v4l2_device_call_all(). That ensures that it will only go to the subdev +that needs it. + +If the sub-device needs to notify its v4l2_device parent of an event, then +it can call v4l2_subdev_notify(sd, notification, arg). This macro checks +whether there is a notify() callback defined and returns -ENODEV if not. +Otherwise the result of the notify() call is returned. + +The advantage of using v4l2_subdev is that it is a generic struct and does +not contain any knowledge about the underlying hardware. So a driver might +contain several subdevs that use an I2C bus, but also a subdev that is +controlled through GPIO pins. This distinction is only relevant when setting +up the device, but once the subdev is registered it is completely transparent. + + +In the asynchronous case subdevice probing can be invoked independently of the +bridge driver availability. The subdevice driver then has to verify whether all +the requirements for a successful probing are satisfied. This can include a +check for a master clock availability. If any of the conditions aren't satisfied +the driver might decide to return -EPROBE_DEFER to request further reprobing +attempts. Once all conditions are met the subdevice shall be registered using +the v4l2_async_register_subdev() function. Unregistration is performed using +the v4l2_async_unregister_subdev() call. Subdevices registered this way are +stored in a global list of subdevices, ready to be picked up by bridge drivers. + +Bridge drivers in turn have to register a notifier object with an array of +subdevice descriptors that the bridge device needs for its operation. This is +performed using the v4l2_async_notifier_register() call. To unregister the +notifier the driver has to call v4l2_async_notifier_unregister(). The former of +the two functions takes two arguments: a pointer to struct v4l2_device and a +pointer to struct v4l2_async_notifier. The latter contains a pointer to an array +of pointers to subdevice descriptors of type struct v4l2_async_subdev type. The +V4L2 core will then use these descriptors to match asynchronously registered +subdevices to them. If a match is detected the .bound() notifier callback is +called. After all subdevices have been located the .complete() callback is +called. When a subdevice is removed from the system the .unbind() method is +called. All three callbacks are optional. + + +V4L2 sub-device userspace API +----------------------------- + +Beside exposing a kernel API through the v4l2_subdev_ops structure, V4L2 +sub-devices can also be controlled directly by userspace applications. + +Device nodes named v4l-subdevX can be created in /dev to access sub-devices +directly. If a sub-device supports direct userspace configuration it must set +the V4L2_SUBDEV_FL_HAS_DEVNODE flag before being registered. + +After registering sub-devices, the v4l2_device driver can create device nodes +for all registered sub-devices marked with V4L2_SUBDEV_FL_HAS_DEVNODE by calling +v4l2_device_register_subdev_nodes(). Those device nodes will be automatically +removed when sub-devices are unregistered. + +The device node handles a subset of the V4L2 API. + +VIDIOC_QUERYCTRL +VIDIOC_QUERYMENU +VIDIOC_G_CTRL +VIDIOC_S_CTRL +VIDIOC_G_EXT_CTRLS +VIDIOC_S_EXT_CTRLS +VIDIOC_TRY_EXT_CTRLS + + The controls ioctls are identical to the ones defined in V4L2. They + behave identically, with the only exception that they deal only with + controls implemented in the sub-device. Depending on the driver, those + controls can be also be accessed through one (or several) V4L2 device + nodes. + +VIDIOC_DQEVENT +VIDIOC_SUBSCRIBE_EVENT +VIDIOC_UNSUBSCRIBE_EVENT + + The events ioctls are identical to the ones defined in V4L2. They + behave identically, with the only exception that they deal only with + events generated by the sub-device. Depending on the driver, those + events can also be reported by one (or several) V4L2 device nodes. + + Sub-device drivers that want to use events need to set the + V4L2_SUBDEV_USES_EVENTS v4l2_subdev::flags and initialize + v4l2_subdev::nevents to events queue depth before registering the + sub-device. After registration events can be queued as usual on the + v4l2_subdev::devnode device node. + + To properly support events, the poll() file operation is also + implemented. + +Private ioctls + + All ioctls not in the above list are passed directly to the sub-device + driver through the core::ioctl operation. + + +I2C sub-device drivers +---------------------- + +Since these drivers are so common, special helper functions are available to +ease the use of these drivers (v4l2-common.h). + +The recommended method of adding v4l2_subdev support to an I2C driver is to +embed the v4l2_subdev struct into the state struct that is created for each +I2C device instance. Very simple devices have no state struct and in that case +you can just create a v4l2_subdev directly. + +A typical state struct would look like this (where 'chipname' is replaced by +the name of the chip): + +.. code-block:: none + + struct chipname_state { + struct v4l2_subdev sd; + ... /* additional state fields */ + }; + +Initialize the v4l2_subdev struct as follows: + +.. code-block:: none + + v4l2_i2c_subdev_init(&state->sd, client, subdev_ops); + +This function will fill in all the fields of v4l2_subdev and ensure that the +v4l2_subdev and i2c_client both point to one another. + +You should also add a helper inline function to go from a v4l2_subdev pointer +to a chipname_state struct: + +.. code-block:: none + + static inline struct chipname_state *to_state(struct v4l2_subdev *sd) + { + return container_of(sd, struct chipname_state, sd); + } + +Use this to go from the v4l2_subdev struct to the i2c_client struct: + +.. code-block:: none + + struct i2c_client *client = v4l2_get_subdevdata(sd); + +And this to go from an i2c_client to a v4l2_subdev struct: + +.. code-block:: none + + struct v4l2_subdev *sd = i2c_get_clientdata(client); + +Make sure to call v4l2_device_unregister_subdev(sd) when the remove() callback +is called. This will unregister the sub-device from the bridge driver. It is +safe to call this even if the sub-device was never registered. + +You need to do this because when the bridge driver destroys the i2c adapter +the remove() callbacks are called of the i2c devices on that adapter. +After that the corresponding v4l2_subdev structures are invalid, so they +have to be unregistered first. Calling v4l2_device_unregister_subdev(sd) +from the remove() callback ensures that this is always done correctly. + + +The bridge driver also has some helper functions it can use: + +.. code-block:: none + + struct v4l2_subdev *sd = v4l2_i2c_new_subdev(v4l2_dev, adapter, + "module_foo", "chipid", 0x36, NULL); + +This loads the given module (can be NULL if no module needs to be loaded) and +calls i2c_new_device() with the given i2c_adapter and chip/address arguments. +If all goes well, then it registers the subdev with the v4l2_device. + +You can also use the last argument of v4l2_i2c_new_subdev() to pass an array +of possible I2C addresses that it should probe. These probe addresses are +only used if the previous argument is 0. A non-zero argument means that you +know the exact i2c address so in that case no probing will take place. + +Both functions return NULL if something went wrong. + +Note that the chipid you pass to v4l2_i2c_new_subdev() is usually +the same as the module name. It allows you to specify a chip variant, e.g. +"saa7114" or "saa7115". In general though the i2c driver autodetects this. +The use of chipid is something that needs to be looked at more closely at a +later date. It differs between i2c drivers and as such can be confusing. +To see which chip variants are supported you can look in the i2c driver code +for the i2c_device_id table. This lists all the possibilities. + +There are two more helper functions: + +v4l2_i2c_new_subdev_cfg: this function adds new irq and platform_data +arguments and has both 'addr' and 'probed_addrs' arguments: if addr is not +0 then that will be used (non-probing variant), otherwise the probed_addrs +are probed. + +For example: this will probe for address 0x10: + +.. code-block:: none + + struct v4l2_subdev *sd = v4l2_i2c_new_subdev_cfg(v4l2_dev, adapter, + "module_foo", "chipid", 0, NULL, 0, I2C_ADDRS(0x10)); + +v4l2_i2c_new_subdev_board uses an i2c_board_info struct which is passed +to the i2c driver and replaces the irq, platform_data and addr arguments. + +If the subdev supports the s_config core ops, then that op is called with +the irq and platform_data arguments after the subdev was setup. The older +v4l2_i2c_new_(probed\_)subdev functions will call s_config as well, but with +irq set to 0 and platform_data set to NULL. + +struct video_device +------------------- + +The actual device nodes in the /dev directory are created using the +video_device struct (v4l2-dev.h). This struct can either be allocated +dynamically or embedded in a larger struct. + +To allocate it dynamically use: + +.. code-block:: none + + struct video_device *vdev = video_device_alloc(); + + if (vdev == NULL) + return -ENOMEM; + + vdev->release = video_device_release; + +If you embed it in a larger struct, then you must set the release() +callback to your own function: + +.. code-block:: none + + struct video_device *vdev = &my_vdev->vdev; + + vdev->release = my_vdev_release; + +The release callback must be set and it is called when the last user +of the video device exits. + +The default video_device_release() callback just calls kfree to free the +allocated memory. + +There is also a video_device_release_empty() function that does nothing +(is empty) and can be used if the struct is embedded and there is nothing +to do when it is released. + +You should also set these fields: + +- v4l2_dev: must be set to the v4l2_device parent device. + +- name: set to something descriptive and unique. + +- vfl_dir: set this to VFL_DIR_RX for capture devices (VFL_DIR_RX has value 0, + so this is normally already the default), set to VFL_DIR_TX for output + devices and VFL_DIR_M2M for mem2mem (codec) devices. + +- fops: set to the v4l2_file_operations struct. + +- ioctl_ops: if you use the v4l2_ioctl_ops to simplify ioctl maintenance + (highly recommended to use this and it might become compulsory in the + future!), then set this to your v4l2_ioctl_ops struct. The vfl_type and + vfl_dir fields are used to disable ops that do not match the type/dir + combination. E.g. VBI ops are disabled for non-VBI nodes, and output ops + are disabled for a capture device. This makes it possible to provide + just one v4l2_ioctl_ops struct for both vbi and video nodes. + +- lock: leave to NULL if you want to do all the locking in the driver. + Otherwise you give it a pointer to a struct mutex_lock and before the + unlocked_ioctl file operation is called this lock will be taken by the + core and released afterwards. See the next section for more details. + +- queue: a pointer to the struct vb2_queue associated with this device node. + If queue is non-NULL, and queue->lock is non-NULL, then queue->lock is + used for the queuing ioctls (VIDIOC_REQBUFS, CREATE_BUFS, QBUF, DQBUF, + QUERYBUF, PREPARE_BUF, STREAMON and STREAMOFF) instead of the lock above. + That way the vb2 queuing framework does not have to wait for other ioctls. + This queue pointer is also used by the vb2 helper functions to check for + queuing ownership (i.e. is the filehandle calling it allowed to do the + operation). + +- prio: keeps track of the priorities. Used to implement VIDIOC_G/S_PRIORITY. + If left to NULL, then it will use the struct v4l2_prio_state in v4l2_device. + If you want to have a separate priority state per (group of) device node(s), + then you can point it to your own struct v4l2_prio_state. + +- dev_parent: you only set this if v4l2_device was registered with NULL as + the parent device struct. This only happens in cases where one hardware + device has multiple PCI devices that all share the same v4l2_device core. + + The cx88 driver is an example of this: one core v4l2_device struct, but + it is used by both a raw video PCI device (cx8800) and a MPEG PCI device + (cx8802). Since the v4l2_device cannot be associated with two PCI devices + at the same time it is setup without a parent device. But when the struct + video_device is initialized you *do* know which parent PCI device to use and + so you set dev_device to the correct PCI device. + +If you use v4l2_ioctl_ops, then you should set .unlocked_ioctl to video_ioctl2 +in your v4l2_file_operations struct. + +Do not use .ioctl! This is deprecated and will go away in the future. + +In some cases you want to tell the core that a function you had specified in +your v4l2_ioctl_ops should be ignored. You can mark such ioctls by calling this +function before video_device_register is called: + +.. code-block:: none + + void v4l2_disable_ioctl(struct video_device *vdev, unsigned int cmd); + +This tends to be needed if based on external factors (e.g. which card is +being used) you want to turns off certain features in v4l2_ioctl_ops without +having to make a new struct. + +The v4l2_file_operations struct is a subset of file_operations. The main +difference is that the inode argument is omitted since it is never used. + +If integration with the media framework is needed, you must initialize the +media_entity struct embedded in the video_device struct (entity field) by +calling media_entity_pads_init(): + +.. code-block:: none + + struct media_pad *pad = &my_vdev->pad; + int err; + + err = media_entity_pads_init(&vdev->entity, 1, pad); + +The pads array must have been previously initialized. There is no need to +manually set the struct media_entity type and name fields. + +A reference to the entity will be automatically acquired/released when the +video device is opened/closed. + +ioctls and locking +------------------ + +The V4L core provides optional locking services. The main service is the +lock field in struct video_device, which is a pointer to a mutex. If you set +this pointer, then that will be used by unlocked_ioctl to serialize all ioctls. + +If you are using the videobuf2 framework, then there is a second lock that you +can set: video_device->queue->lock. If set, then this lock will be used instead +of video_device->lock to serialize all queuing ioctls (see the previous section +for the full list of those ioctls). + +The advantage of using a different lock for the queuing ioctls is that for some +drivers (particularly USB drivers) certain commands such as setting controls +can take a long time, so you want to use a separate lock for the buffer queuing +ioctls. That way your VIDIOC_DQBUF doesn't stall because the driver is busy +changing the e.g. exposure of the webcam. + +Of course, you can always do all the locking yourself by leaving both lock +pointers at NULL. + +If you use the old videobuf then you must pass the video_device lock to the +videobuf queue initialize function: if videobuf has to wait for a frame to +arrive, then it will temporarily unlock the lock and relock it afterwards. If +your driver also waits in the code, then you should do the same to allow other +processes to access the device node while the first process is waiting for +something. + +In the case of videobuf2 you will need to implement the wait_prepare and +wait_finish callbacks to unlock/lock if applicable. If you use the queue->lock +pointer, then you can use the helper functions vb2_ops_wait_prepare/finish. + +The implementation of a hotplug disconnect should also take the lock from +video_device before calling v4l2_device_disconnect. If you are also using +video_device->queue->lock, then you have to first lock video_device->queue->lock +followed by video_device->lock. That way you can be sure no ioctl is running +when you call v4l2_device_disconnect. + +video_device registration +------------------------- + +Next you register the video device: this will create the character device +for you. + +.. code-block:: none + + err = video_register_device(vdev, VFL_TYPE_GRABBER, -1); + if (err) { + video_device_release(vdev); /* or kfree(my_vdev); */ + return err; + } + +If the v4l2_device parent device has a non-NULL mdev field, the video device +entity will be automatically registered with the media device. + +Which device is registered depends on the type argument. The following +types exist: + +VFL_TYPE_GRABBER: videoX for video input/output devices +VFL_TYPE_VBI: vbiX for vertical blank data (i.e. closed captions, teletext) +VFL_TYPE_RADIO: radioX for radio tuners +VFL_TYPE_SDR: swradioX for Software Defined Radio tuners + +The last argument gives you a certain amount of control over the device +device node number used (i.e. the X in videoX). Normally you will pass -1 +to let the v4l2 framework pick the first free number. But sometimes users +want to select a specific node number. It is common that drivers allow +the user to select a specific device node number through a driver module +option. That number is then passed to this function and video_register_device +will attempt to select that device node number. If that number was already +in use, then the next free device node number will be selected and it +will send a warning to the kernel log. + +Another use-case is if a driver creates many devices. In that case it can +be useful to place different video devices in separate ranges. For example, +video capture devices start at 0, video output devices start at 16. +So you can use the last argument to specify a minimum device node number +and the v4l2 framework will try to pick the first free number that is equal +or higher to what you passed. If that fails, then it will just pick the +first free number. + +Since in this case you do not care about a warning about not being able +to select the specified device node number, you can call the function +video_register_device_no_warn() instead. + +Whenever a device node is created some attributes are also created for you. +If you look in /sys/class/video4linux you see the devices. Go into e.g. +video0 and you will see 'name', 'dev_debug' and 'index' attributes. The 'name' +attribute is the 'name' field of the video_device struct. The 'dev_debug' attribute +can be used to enable core debugging. See the next section for more detailed +information on this. + +The 'index' attribute is the index of the device node: for each call to +video_register_device() the index is just increased by 1. The first video +device node you register always starts with index 0. + +Users can setup udev rules that utilize the index attribute to make fancy +device names (e.g. 'mpegX' for MPEG video capture device nodes). + +After the device was successfully registered, then you can use these fields: + +- vfl_type: the device type passed to video_register_device. +- minor: the assigned device minor number. +- num: the device node number (i.e. the X in videoX). +- index: the device index number. + +If the registration failed, then you need to call video_device_release() +to free the allocated video_device struct, or free your own struct if the +video_device was embedded in it. The vdev->release() callback will never +be called if the registration failed, nor should you ever attempt to +unregister the device if the registration failed. + +video device debugging +---------------------- + +The 'dev_debug' attribute that is created for each video, vbi, radio or swradio +device in /sys/class/video4linux/<devX>/ allows you to enable logging of +file operations. + +It is a bitmask and the following bits can be set: + +.. code-block:: none + + 0x01: Log the ioctl name and error code. VIDIOC_(D)QBUF ioctls are only logged + if bit 0x08 is also set. + 0x02: Log the ioctl name arguments and error code. VIDIOC_(D)QBUF ioctls are + only logged if bit 0x08 is also set. + 0x04: Log the file operations open, release, read, write, mmap and + get_unmapped_area. The read and write operations are only logged if + bit 0x08 is also set. + 0x08: Log the read and write file operations and the VIDIOC_QBUF and + VIDIOC_DQBUF ioctls. + 0x10: Log the poll file operation. + +video_device cleanup +-------------------- + +When the video device nodes have to be removed, either during the unload +of the driver or because the USB device was disconnected, then you should +unregister them: + +.. code-block:: none + + video_unregister_device(vdev); + +This will remove the device nodes from sysfs (causing udev to remove them +from /dev). + +After video_unregister_device() returns no new opens can be done. However, +in the case of USB devices some application might still have one of these +device nodes open. So after the unregister all file operations (except +release, of course) will return an error as well. + +When the last user of the video device node exits, then the vdev->release() +callback is called and you can do the final cleanup there. + +Don't forget to cleanup the media entity associated with the video device if +it has been initialized: + +.. code-block:: none + + media_entity_cleanup(&vdev->entity); + +This can be done from the release callback. + + +video_device helper functions +----------------------------- + +There are a few useful helper functions: + +- file/video_device private data + +You can set/get driver private data in the video_device struct using: + +.. code-block:: none + + void *video_get_drvdata(struct video_device *vdev); + void video_set_drvdata(struct video_device *vdev, void *data); + +Note that you can safely call video_set_drvdata() before calling +video_register_device(). + +And this function: + +.. code-block:: none + + struct video_device *video_devdata(struct file *file); + +returns the video_device belonging to the file struct. + +The video_drvdata function combines video_get_drvdata with video_devdata: + +.. code-block:: none + + void *video_drvdata(struct file *file); + +You can go from a video_device struct to the v4l2_device struct using: + +.. code-block:: none + + struct v4l2_device *v4l2_dev = vdev->v4l2_dev; + +- Device node name + +The video_device node kernel name can be retrieved using + +.. code-block:: none + + const char *video_device_node_name(struct video_device *vdev); + +The name is used as a hint by userspace tools such as udev. The function +should be used where possible instead of accessing the video_device::num and +video_device::minor fields. + + +video buffer helper functions +----------------------------- + +The v4l2 core API provides a set of standard methods (called "videobuf") +for dealing with video buffers. Those methods allow a driver to implement +read(), mmap() and overlay() in a consistent way. There are currently +methods for using video buffers on devices that supports DMA with +scatter/gather method (videobuf-dma-sg), DMA with linear access +(videobuf-dma-contig), and vmalloced buffers, mostly used on USB drivers +(videobuf-vmalloc). + +Please see Documentation/video4linux/videobuf for more information on how +to use the videobuf layer. + +struct v4l2_fh +-------------- + +struct v4l2_fh provides a way to easily keep file handle specific data +that is used by the V4L2 framework. New drivers must use struct v4l2_fh +since it is also used to implement priority handling (VIDIOC_G/S_PRIORITY). + +The users of v4l2_fh (in the V4L2 framework, not the driver) know +whether a driver uses v4l2_fh as its file->private_data pointer by +testing the V4L2_FL_USES_V4L2_FH bit in video_device->flags. This bit is +set whenever v4l2_fh_init() is called. + +struct v4l2_fh is allocated as a part of the driver's own file handle +structure and file->private_data is set to it in the driver's open +function by the driver. + +In many cases the struct v4l2_fh will be embedded in a larger structure. +In that case you should call v4l2_fh_init+v4l2_fh_add in open() and +v4l2_fh_del+v4l2_fh_exit in release(). + +Drivers can extract their own file handle structure by using the container_of +macro. Example: + +.. code-block:: none + + struct my_fh { + int blah; + struct v4l2_fh fh; + }; + + ... + + int my_open(struct file *file) + { + struct my_fh *my_fh; + struct video_device *vfd; + int ret; + + ... + + my_fh = kzalloc(sizeof(*my_fh), GFP_KERNEL); + + ... + + v4l2_fh_init(&my_fh->fh, vfd); + + ... + + file->private_data = &my_fh->fh; + v4l2_fh_add(&my_fh->fh); + return 0; + } + + int my_release(struct file *file) + { + struct v4l2_fh *fh = file->private_data; + struct my_fh *my_fh = container_of(fh, struct my_fh, fh); + + ... + v4l2_fh_del(&my_fh->fh); + v4l2_fh_exit(&my_fh->fh); + kfree(my_fh); + return 0; + } + +Below is a short description of the v4l2_fh functions used: + +.. code-block:: none + + void v4l2_fh_init(struct v4l2_fh *fh, struct video_device *vdev) + + Initialise the file handle. This *MUST* be performed in the driver's + v4l2_file_operations->open() handler. + +.. code-block:: none + + void v4l2_fh_add(struct v4l2_fh *fh) + + Add a v4l2_fh to video_device file handle list. Must be called once the + file handle is completely initialized. + +.. code-block:: none + + void v4l2_fh_del(struct v4l2_fh *fh) + + Unassociate the file handle from video_device(). The file handle + exit function may now be called. + +.. code-block:: none + + void v4l2_fh_exit(struct v4l2_fh *fh) + + Uninitialise the file handle. After uninitialisation the v4l2_fh + memory can be freed. + + +If struct v4l2_fh is not embedded, then you can use these helper functions: + +.. code-block:: none + + int v4l2_fh_open(struct file *filp) + + This allocates a struct v4l2_fh, initializes it and adds it to the struct + video_device associated with the file struct. + +.. code-block:: none + + int v4l2_fh_release(struct file *filp) + + This deletes it from the struct video_device associated with the file + struct, uninitialised the v4l2_fh and frees it. + +These two functions can be plugged into the v4l2_file_operation's open() and +release() ops. + + +Several drivers need to do something when the first file handle is opened and +when the last file handle closes. Two helper functions were added to check +whether the v4l2_fh struct is the only open filehandle of the associated +device node: + +.. code-block:: none + + int v4l2_fh_is_singular(struct v4l2_fh *fh) + + Returns 1 if the file handle is the only open file handle, else 0. + +.. code-block:: none + + int v4l2_fh_is_singular_file(struct file *filp) + + Same, but it calls v4l2_fh_is_singular with filp->private_data. + + +V4L2 events +----------- + +The V4L2 events provide a generic way to pass events to user space. +The driver must use v4l2_fh to be able to support V4L2 events. + +Events are defined by a type and an optional ID. The ID may refer to a V4L2 +object such as a control ID. If unused, then the ID is 0. + +When the user subscribes to an event the driver will allocate a number of +kevent structs for that event. So every (type, ID) event tuple will have +its own set of kevent structs. This guarantees that if a driver is generating +lots of events of one type in a short time, then that will not overwrite +events of another type. + +But if you get more events of one type than the number of kevents that were +reserved, then the oldest event will be dropped and the new one added. + +Furthermore, the internal struct v4l2_subscribed_event has merge() and +replace() callbacks which drivers can set. These callbacks are called when +a new event is raised and there is no more room. The replace() callback +allows you to replace the payload of the old event with that of the new event, +merging any relevant data from the old payload into the new payload that +replaces it. It is called when this event type has only one kevent struct +allocated. The merge() callback allows you to merge the oldest event payload +into that of the second-oldest event payload. It is called when there are two +or more kevent structs allocated. + +This way no status information is lost, just the intermediate steps leading +up to that state. + +A good example of these replace/merge callbacks is in v4l2-event.c: +ctrls_replace() and ctrls_merge() callbacks for the control event. + +Note: these callbacks can be called from interrupt context, so they must be +fast. + +Useful functions: + +.. code-block:: none + + void v4l2_event_queue(struct video_device *vdev, const struct v4l2_event *ev) + + Queue events to video device. The driver's only responsibility is to fill + in the type and the data fields. The other fields will be filled in by + V4L2. + +.. code-block:: none + + int v4l2_event_subscribe(struct v4l2_fh *fh, + struct v4l2_event_subscription *sub, unsigned elems, + const struct v4l2_subscribed_event_ops *ops) + + The video_device->ioctl_ops->vidioc_subscribe_event must check the driver + is able to produce events with specified event id. Then it calls + v4l2_event_subscribe() to subscribe the event. + + The elems argument is the size of the event queue for this event. If it is 0, + then the framework will fill in a default value (this depends on the event + type). + + The ops argument allows the driver to specify a number of callbacks: + * add: called when a new listener gets added (subscribing to the same + event twice will only cause this callback to get called once) + * del: called when a listener stops listening + * replace: replace event 'old' with event 'new'. + * merge: merge event 'old' into event 'new'. + All 4 callbacks are optional, if you don't want to specify any callbacks + the ops argument itself maybe NULL. + +.. code-block:: none + + int v4l2_event_unsubscribe(struct v4l2_fh *fh, + struct v4l2_event_subscription *sub) + + vidioc_unsubscribe_event in struct v4l2_ioctl_ops. A driver may use + v4l2_event_unsubscribe() directly unless it wants to be involved in + unsubscription process. + + The special type V4L2_EVENT_ALL may be used to unsubscribe all events. The + drivers may want to handle this in a special way. + +.. code-block:: none + + int v4l2_event_pending(struct v4l2_fh *fh) + + Returns the number of pending events. Useful when implementing poll. + +Events are delivered to user space through the poll system call. The driver +can use v4l2_fh->wait (a wait_queue_head_t) as the argument for poll_wait(). + +There are standard and private events. New standard events must use the +smallest available event type. The drivers must allocate their events from +their own class starting from class base. Class base is +V4L2_EVENT_PRIVATE_START + n * 1000 where n is the lowest available number. +The first event type in the class is reserved for future use, so the first +available event type is 'class base + 1'. + +An example on how the V4L2 events may be used can be found in the OMAP +3 ISP driver (drivers/media/platform/omap3isp). + +A subdev can directly send an event to the v4l2_device notify function with +V4L2_DEVICE_NOTIFY_EVENT. This allows the bridge to map the subdev that sends +the event to the video node(s) associated with the subdev that need to be +informed about such an event. + +V4L2 clocks +----------- + +Many subdevices, like camera sensors, TV decoders and encoders, need a clock +signal to be supplied by the system. Often this clock is supplied by the +respective bridge device. The Linux kernel provides a Common Clock Framework for +this purpose. However, it is not (yet) available on all architectures. Besides, +the nature of the multi-functional (clock, data + synchronisation, I2C control) +connection of subdevices to the system might impose special requirements on the +clock API usage. E.g. V4L2 has to support clock provider driver unregistration +while a subdevice driver is holding a reference to the clock. For these reasons +a V4L2 clock helper API has been developed and is provided to bridge and +subdevice drivers. + +The API consists of two parts: two functions to register and unregister a V4L2 +clock source: v4l2_clk_register() and v4l2_clk_unregister() and calls to control +a clock object, similar to the respective generic clock API calls: +v4l2_clk_get(), v4l2_clk_put(), v4l2_clk_enable(), v4l2_clk_disable(), +v4l2_clk_get_rate(), and v4l2_clk_set_rate(). Clock suppliers have to provide +clock operations that will be called when clock users invoke respective API +methods. + +It is expected that once the CCF becomes available on all relevant +architectures this API will be removed. diff --git a/Documentation/media/kapi/videobuf.rst b/Documentation/media/kapi/videobuf.rst new file mode 100644 index 000000000000..01156728203c --- /dev/null +++ b/Documentation/media/kapi/videobuf.rst @@ -0,0 +1,402 @@ +Videobuf Framework +================== + +Author: Jonathan Corbet <corbet@lwn.net> + +Current as of 2.6.33 + +.. note:: + + The videobuf framework was deprecated in favor of videobuf2. Shouldn't + be used on new drivers. + +Introduction +------------ + +The videobuf layer functions as a sort of glue layer between a V4L2 driver +and user space. It handles the allocation and management of buffers for +the storage of video frames. There is a set of functions which can be used +to implement many of the standard POSIX I/O system calls, including read(), +poll(), and, happily, mmap(). Another set of functions can be used to +implement the bulk of the V4L2 ioctl() calls related to streaming I/O, +including buffer allocation, queueing and dequeueing, and streaming +control. Using videobuf imposes a few design decisions on the driver +author, but the payback comes in the form of reduced code in the driver and +a consistent implementation of the V4L2 user-space API. + +Buffer types +------------ + +Not all video devices use the same kind of buffers. In fact, there are (at +least) three common variations: + + - Buffers which are scattered in both the physical and (kernel) virtual + address spaces. (Almost) all user-space buffers are like this, but it + makes great sense to allocate kernel-space buffers this way as well when + it is possible. Unfortunately, it is not always possible; working with + this kind of buffer normally requires hardware which can do + scatter/gather DMA operations. + + - Buffers which are physically scattered, but which are virtually + contiguous; buffers allocated with vmalloc(), in other words. These + buffers are just as hard to use for DMA operations, but they can be + useful in situations where DMA is not available but virtually-contiguous + buffers are convenient. + + - Buffers which are physically contiguous. Allocation of this kind of + buffer can be unreliable on fragmented systems, but simpler DMA + controllers cannot deal with anything else. + +Videobuf can work with all three types of buffers, but the driver author +must pick one at the outset and design the driver around that decision. + +[It's worth noting that there's a fourth kind of buffer: "overlay" buffers +which are located within the system's video memory. The overlay +functionality is considered to be deprecated for most use, but it still +shows up occasionally in system-on-chip drivers where the performance +benefits merit the use of this technique. Overlay buffers can be handled +as a form of scattered buffer, but there are very few implementations in +the kernel and a description of this technique is currently beyond the +scope of this document.] + +Data structures, callbacks, and initialization +---------------------------------------------- + +Depending on which type of buffers are being used, the driver should +include one of the following files: + +.. code-block:: none + + <media/videobuf-dma-sg.h> /* Physically scattered */ + <media/videobuf-vmalloc.h> /* vmalloc() buffers */ + <media/videobuf-dma-contig.h> /* Physically contiguous */ + +The driver's data structure describing a V4L2 device should include a +struct videobuf_queue instance for the management of the buffer queue, +along with a list_head for the queue of available buffers. There will also +need to be an interrupt-safe spinlock which is used to protect (at least) +the queue. + +The next step is to write four simple callbacks to help videobuf deal with +the management of buffers: + +.. code-block:: none + + struct videobuf_queue_ops { + int (*buf_setup)(struct videobuf_queue *q, + unsigned int *count, unsigned int *size); + int (*buf_prepare)(struct videobuf_queue *q, + struct videobuf_buffer *vb, + enum v4l2_field field); + void (*buf_queue)(struct videobuf_queue *q, + struct videobuf_buffer *vb); + void (*buf_release)(struct videobuf_queue *q, + struct videobuf_buffer *vb); + }; + +buf_setup() is called early in the I/O process, when streaming is being +initiated; its purpose is to tell videobuf about the I/O stream. The count +parameter will be a suggested number of buffers to use; the driver should +check it for rationality and adjust it if need be. As a practical rule, a +minimum of two buffers are needed for proper streaming, and there is +usually a maximum (which cannot exceed 32) which makes sense for each +device. The size parameter should be set to the expected (maximum) size +for each frame of data. + +Each buffer (in the form of a struct videobuf_buffer pointer) will be +passed to buf_prepare(), which should set the buffer's size, width, height, +and field fields properly. If the buffer's state field is +VIDEOBUF_NEEDS_INIT, the driver should pass it to: + +.. code-block:: none + + int videobuf_iolock(struct videobuf_queue* q, struct videobuf_buffer *vb, + struct v4l2_framebuffer *fbuf); + +Among other things, this call will usually allocate memory for the buffer. +Finally, the buf_prepare() function should set the buffer's state to +VIDEOBUF_PREPARED. + +When a buffer is queued for I/O, it is passed to buf_queue(), which should +put it onto the driver's list of available buffers and set its state to +VIDEOBUF_QUEUED. Note that this function is called with the queue spinlock +held; if it tries to acquire it as well things will come to a screeching +halt. Yes, this is the voice of experience. Note also that videobuf may +wait on the first buffer in the queue; placing other buffers in front of it +could again gum up the works. So use list_add_tail() to enqueue buffers. + +Finally, buf_release() is called when a buffer is no longer intended to be +used. The driver should ensure that there is no I/O active on the buffer, +then pass it to the appropriate free routine(s): + +.. code-block:: none + + /* Scatter/gather drivers */ + int videobuf_dma_unmap(struct videobuf_queue *q, + struct videobuf_dmabuf *dma); + int videobuf_dma_free(struct videobuf_dmabuf *dma); + + /* vmalloc drivers */ + void videobuf_vmalloc_free (struct videobuf_buffer *buf); + + /* Contiguous drivers */ + void videobuf_dma_contig_free(struct videobuf_queue *q, + struct videobuf_buffer *buf); + +One way to ensure that a buffer is no longer under I/O is to pass it to: + +.. code-block:: none + + int videobuf_waiton(struct videobuf_buffer *vb, int non_blocking, int intr); + +Here, vb is the buffer, non_blocking indicates whether non-blocking I/O +should be used (it should be zero in the buf_release() case), and intr +controls whether an interruptible wait is used. + +File operations +--------------- + +At this point, much of the work is done; much of the rest is slipping +videobuf calls into the implementation of the other driver callbacks. The +first step is in the open() function, which must initialize the +videobuf queue. The function to use depends on the type of buffer used: + +.. code-block:: none + + void videobuf_queue_sg_init(struct videobuf_queue *q, + struct videobuf_queue_ops *ops, + struct device *dev, + spinlock_t *irqlock, + enum v4l2_buf_type type, + enum v4l2_field field, + unsigned int msize, + void *priv); + + void videobuf_queue_vmalloc_init(struct videobuf_queue *q, + struct videobuf_queue_ops *ops, + struct device *dev, + spinlock_t *irqlock, + enum v4l2_buf_type type, + enum v4l2_field field, + unsigned int msize, + void *priv); + + void videobuf_queue_dma_contig_init(struct videobuf_queue *q, + struct videobuf_queue_ops *ops, + struct device *dev, + spinlock_t *irqlock, + enum v4l2_buf_type type, + enum v4l2_field field, + unsigned int msize, + void *priv); + +In each case, the parameters are the same: q is the queue structure for the +device, ops is the set of callbacks as described above, dev is the device +structure for this video device, irqlock is an interrupt-safe spinlock to +protect access to the data structures, type is the buffer type used by the +device (cameras will use V4L2_BUF_TYPE_VIDEO_CAPTURE, for example), field +describes which field is being captured (often V4L2_FIELD_NONE for +progressive devices), msize is the size of any containing structure used +around struct videobuf_buffer, and priv is a private data pointer which +shows up in the priv_data field of struct videobuf_queue. Note that these +are void functions which, evidently, are immune to failure. + +V4L2 capture drivers can be written to support either of two APIs: the +read() system call and the rather more complicated streaming mechanism. As +a general rule, it is necessary to support both to ensure that all +applications have a chance of working with the device. Videobuf makes it +easy to do that with the same code. To implement read(), the driver need +only make a call to one of: + +.. code-block:: none + + ssize_t videobuf_read_one(struct videobuf_queue *q, + char __user *data, size_t count, + loff_t *ppos, int nonblocking); + + ssize_t videobuf_read_stream(struct videobuf_queue *q, + char __user *data, size_t count, + loff_t *ppos, int vbihack, int nonblocking); + +Either one of these functions will read frame data into data, returning the +amount actually read; the difference is that videobuf_read_one() will only +read a single frame, while videobuf_read_stream() will read multiple frames +if they are needed to satisfy the count requested by the application. A +typical driver read() implementation will start the capture engine, call +one of the above functions, then stop the engine before returning (though a +smarter implementation might leave the engine running for a little while in +anticipation of another read() call happening in the near future). + +The poll() function can usually be implemented with a direct call to: + +.. code-block:: none + + unsigned int videobuf_poll_stream(struct file *file, + struct videobuf_queue *q, + poll_table *wait); + +Note that the actual wait queue eventually used will be the one associated +with the first available buffer. + +When streaming I/O is done to kernel-space buffers, the driver must support +the mmap() system call to enable user space to access the data. In many +V4L2 drivers, the often-complex mmap() implementation simplifies to a +single call to: + +.. code-block:: none + + int videobuf_mmap_mapper(struct videobuf_queue *q, + struct vm_area_struct *vma); + +Everything else is handled by the videobuf code. + +The release() function requires two separate videobuf calls: + +.. code-block:: none + + void videobuf_stop(struct videobuf_queue *q); + int videobuf_mmap_free(struct videobuf_queue *q); + +The call to videobuf_stop() terminates any I/O in progress - though it is +still up to the driver to stop the capture engine. The call to +videobuf_mmap_free() will ensure that all buffers have been unmapped; if +so, they will all be passed to the buf_release() callback. If buffers +remain mapped, videobuf_mmap_free() returns an error code instead. The +purpose is clearly to cause the closing of the file descriptor to fail if +buffers are still mapped, but every driver in the 2.6.32 kernel cheerfully +ignores its return value. + +ioctl() operations +------------------ + +The V4L2 API includes a very long list of driver callbacks to respond to +the many ioctl() commands made available to user space. A number of these +- those associated with streaming I/O - turn almost directly into videobuf +calls. The relevant helper functions are: + +.. code-block:: none + + int videobuf_reqbufs(struct videobuf_queue *q, + struct v4l2_requestbuffers *req); + int videobuf_querybuf(struct videobuf_queue *q, struct v4l2_buffer *b); + int videobuf_qbuf(struct videobuf_queue *q, struct v4l2_buffer *b); + int videobuf_dqbuf(struct videobuf_queue *q, struct v4l2_buffer *b, + int nonblocking); + int videobuf_streamon(struct videobuf_queue *q); + int videobuf_streamoff(struct videobuf_queue *q); + +So, for example, a VIDIOC_REQBUFS call turns into a call to the driver's +vidioc_reqbufs() callback which, in turn, usually only needs to locate the +proper struct videobuf_queue pointer and pass it to videobuf_reqbufs(). +These support functions can replace a great deal of buffer management +boilerplate in a lot of V4L2 drivers. + +The vidioc_streamon() and vidioc_streamoff() functions will be a bit more +complex, of course, since they will also need to deal with starting and +stopping the capture engine. + +Buffer allocation +----------------- + +Thus far, we have talked about buffers, but have not looked at how they are +allocated. The scatter/gather case is the most complex on this front. For +allocation, the driver can leave buffer allocation entirely up to the +videobuf layer; in this case, buffers will be allocated as anonymous +user-space pages and will be very scattered indeed. If the application is +using user-space buffers, no allocation is needed; the videobuf layer will +take care of calling get_user_pages() and filling in the scatterlist array. + +If the driver needs to do its own memory allocation, it should be done in +the vidioc_reqbufs() function, *after* calling videobuf_reqbufs(). The +first step is a call to: + +.. code-block:: none + + struct videobuf_dmabuf *videobuf_to_dma(struct videobuf_buffer *buf); + +The returned videobuf_dmabuf structure (defined in +<media/videobuf-dma-sg.h>) includes a couple of relevant fields: + +.. code-block:: none + + struct scatterlist *sglist; + int sglen; + +The driver must allocate an appropriately-sized scatterlist array and +populate it with pointers to the pieces of the allocated buffer; sglen +should be set to the length of the array. + +Drivers using the vmalloc() method need not (and cannot) concern themselves +with buffer allocation at all; videobuf will handle those details. The +same is normally true of contiguous-DMA drivers as well; videobuf will +allocate the buffers (with dma_alloc_coherent()) when it sees fit. That +means that these drivers may be trying to do high-order allocations at any +time, an operation which is not always guaranteed to work. Some drivers +play tricks by allocating DMA space at system boot time; videobuf does not +currently play well with those drivers. + +As of 2.6.31, contiguous-DMA drivers can work with a user-supplied buffer, +as long as that buffer is physically contiguous. Normal user-space +allocations will not meet that criterion, but buffers obtained from other +kernel drivers, or those contained within huge pages, will work with these +drivers. + +Filling the buffers +------------------- + +The final part of a videobuf implementation has no direct callback - it's +the portion of the code which actually puts frame data into the buffers, +usually in response to interrupts from the device. For all types of +drivers, this process works approximately as follows: + + - Obtain the next available buffer and make sure that somebody is actually + waiting for it. + + - Get a pointer to the memory and put video data there. + + - Mark the buffer as done and wake up the process waiting for it. + +Step (1) above is done by looking at the driver-managed list_head structure +- the one which is filled in the buf_queue() callback. Because starting +the engine and enqueueing buffers are done in separate steps, it's possible +for the engine to be running without any buffers available - in the +vmalloc() case especially. So the driver should be prepared for the list +to be empty. It is equally possible that nobody is yet interested in the +buffer; the driver should not remove it from the list or fill it until a +process is waiting on it. That test can be done by examining the buffer's +done field (a wait_queue_head_t structure) with waitqueue_active(). + +A buffer's state should be set to VIDEOBUF_ACTIVE before being mapped for +DMA; that ensures that the videobuf layer will not try to do anything with +it while the device is transferring data. + +For scatter/gather drivers, the needed memory pointers will be found in the +scatterlist structure described above. Drivers using the vmalloc() method +can get a memory pointer with: + +.. code-block:: none + + void *videobuf_to_vmalloc(struct videobuf_buffer *buf); + +For contiguous DMA drivers, the function to use is: + +.. code-block:: none + + dma_addr_t videobuf_to_dma_contig(struct videobuf_buffer *buf); + +The contiguous DMA API goes out of its way to hide the kernel-space address +of the DMA buffer from drivers. + +The final step is to set the size field of the relevant videobuf_buffer +structure to the actual size of the captured image, set state to +VIDEOBUF_DONE, then call wake_up() on the done queue. At this point, the +buffer is owned by the videobuf layer and the driver should not touch it +again. + +Developers who are interested in more information can go into the relevant +header files; there are a few low-level functions declared there which have +not been talked about here. Also worthwhile is the vivi driver +(drivers/media/platform/vivi.c), which is maintained as an example of how V4L2 +drivers should be written. Vivi only uses the vmalloc() API, but it's good +enough to get started with. Note also that all of these calls are exported +GPL-only, so they will not be available to non-GPL kernel modules. |