3.5. Buffers¶
A buffer contains data exchanged by application and driver using one of
the Streaming I/O methods. In the multi-planar API, the data is held in
planes, while the buffer structure acts as a container for the planes.
Only pointers to buffers (planes) are exchanged, the data itself is not
copied. These pointers, together with meta-information like timestamps
or field parity, are stored in a struct v4l2_buffer
,
argument to the ioctl VIDIOC_QUERYBUF,
VIDIOC_QBUF and
VIDIOC_DQBUF ioctl. In the multi-planar API,
some plane-specific members of struct v4l2_buffer
,
such as pointers and sizes for each plane, are stored in
struct v4l2_plane
instead. In that case,
struct v4l2_buffer
contains an array of plane structures.
Dequeued video buffers come with timestamps. The driver decides at which
part of the frame and with which clock the timestamp is taken. Please
see flags in the masks V4L2_BUF_FLAG_TIMESTAMP_MASK
and
V4L2_BUF_FLAG_TSTAMP_SRC_MASK
in Buffer Flags. These flags
are always valid and constant across all buffers during the whole video
stream. Changes in these flags may take place as a side effect of
VIDIOC_S_INPUT or
VIDIOC_S_OUTPUT however. The
V4L2_BUF_FLAG_TIMESTAMP_COPY
timestamp type which is used by e.g. on
mem-to-mem devices is an exception to the rule: the timestamp source
flags are copied from the OUTPUT video buffer to the CAPTURE video
buffer.
3.5.1. Interactions between formats, controls and buffers¶
V4L2 exposes parameters that influence the buffer size, or the way data is
laid out in the buffer. Those parameters are exposed through both formats and
controls. One example of such a control is the V4L2_CID_ROTATE
control
that modifies the direction in which pixels are stored in the buffer, as well
as the buffer size when the selected format includes padding at the end of
lines.
The set of information needed to interpret the content of a buffer (e.g. the pixel format, the line stride, the tiling orientation or the rotation) is collectively referred to in the rest of this section as the buffer layout.
Controls that can modify the buffer layout shall set the
V4L2_CTRL_FLAG_MODIFY_LAYOUT
flag.
Modifying formats or controls that influence the buffer size or layout require
the stream to be stopped. Any attempt at such a modification while the stream
is active shall cause the ioctl setting the format or the control to return
the EBUSY
error code. In that case drivers shall also set the
V4L2_CTRL_FLAG_GRABBED
flag when calling
VIDIOC_QUERYCTRL()
or VIDIOC_QUERY_EXT_CTRL()
for such a
control while the stream is active.
Note
The VIDIOC_S_SELECTION()
ioctl can, depending on the hardware (for
instance if the device doesn’t include a scaler), modify the format in
addition to the selection rectangle. Similarly, the
VIDIOC_S_INPUT()
, VIDIOC_S_OUTPUT()
, VIDIOC_S_STD()
and VIDIOC_S_DV_TIMINGS()
ioctls can also modify the format and
selection rectangles. When those ioctls result in a buffer size or layout
change, drivers shall handle that condition as they would handle it in the
VIDIOC_S_FMT()
ioctl in all cases described in this section.
Controls that only influence the buffer layout can be modified at any time
when the stream is stopped. As they don’t influence the buffer size, no
special handling is needed to synchronize those controls with buffer
allocation and the V4L2_CTRL_FLAG_GRABBED
flag is cleared once the
stream is stopped.
Formats and controls that influence the buffer size interact with buffer
allocation. The simplest way to handle this is for drivers to always require
buffers to be reallocated in order to change those formats or controls. In
that case, to perform such changes, userspace applications shall first stop
the video stream with the VIDIOC_STREAMOFF()
ioctl if it is running
and free all buffers with the VIDIOC_REQBUFS()
ioctl if they are
allocated. After freeing all buffers the V4L2_CTRL_FLAG_GRABBED
flag
for controls is cleared. The format or controls can then be modified, and
buffers shall then be reallocated and the stream restarted. A typical ioctl
sequence is
VIDIOC_STREAMOFF
VIDIOC_REQBUFS(0)
VIDIOC_S_EXT_CTRLS
VIDIOC_S_FMT
VIDIOC_REQBUFS(n)
VIDIOC_QBUF
VIDIOC_STREAMON
The second VIDIOC_REQBUFS()
call will take the new format and control
value into account to compute the buffer size to allocate. Applications can
also retrieve the size by calling the VIDIOC_G_FMT()
ioctl if needed.
Note
The API doesn’t mandate the above order for control (3.) and format (4.) changes. Format and controls can be set in a different order, or even interleaved, depending on the device and use case. For instance some controls might behave differently for different pixel formats, in which case the format might need to be set first.
When reallocation is required, any attempt to modify format or controls that
influences the buffer size while buffers are allocated shall cause the format
or control set ioctl to return the EBUSY
error. Any attempt to queue a
buffer too small for the current format or controls shall cause the
VIDIOC_QBUF()
ioctl to return a EINVAL
error.
Buffer reallocation is an expensive operation. To avoid that cost, drivers can (and are encouraged to) allow format or controls that influence the buffer size to be changed with buffers allocated. In that case, a typical ioctl sequence to modify format and controls is
VIDIOC_STREAMOFF
VIDIOC_S_EXT_CTRLS
VIDIOC_S_FMT
VIDIOC_QBUF
VIDIOC_STREAMON
For this sequence to operate correctly, queued buffers need to be large enough
for the new format or controls. Drivers shall return a ENOSPC
error in
response to format change (VIDIOC_S_FMT()
) or control changes
(VIDIOC_S_CTRL()
or VIDIOC_S_EXT_CTRLS()
) if buffers too small
for the new format are currently queued. As a simplification, drivers are
allowed to return a EBUSY
error from these ioctls if any buffer is
currently queued, without checking the queued buffers sizes.
Additionally, drivers shall return a EINVAL
error from the
VIDIOC_QBUF()
ioctl if the buffer being queued is too small for the
current format or controls. Together, these requirements ensure that queued
buffers will always be large enough for the configured format and controls.
Userspace applications can query the buffer size required for a given format
and controls by first setting the desired control values and then trying the
desired format. The VIDIOC_TRY_FMT()
ioctl will return the required
buffer size.
VIDIOC_S_EXT_CTRLS(x)
VIDIOC_S_EXT_CTRLS(y)
The VIDIOC_CREATE_BUFS()
ioctl can then be used to allocate buffers
based on the queried sizes (for instance by allocating a set of buffers large
enough for all the desired formats and controls, or by allocating separate set
of appropriately sized buffers for each use case).
-
type v4l2_buffer¶
3.5.2. struct v4l2_buffer¶
__u32 |
|
Number of the buffer, set by the application except when calling
VIDIOC_DQBUF, then it is set by the
driver. This field can range from zero to the number of buffers
allocated with the ioctl VIDIOC_REQBUFS ioctl
(struct |
__u32 |
|
Type of the buffer, same as struct
|
__u32 |
|
The number of bytes occupied by the data in the buffer. It depends
on the negotiated data format and may change with each buffer for
compressed variable size data like JPEG images. Drivers must set
this field when |
__u32 |
|
Flags set by the application or driver, see Buffer Flags. |
__u32 |
|
Indicates the field order of the image in the buffer, see
|
struct timeval |
|
For capture streams this is time when the first data byte was
captured, as returned by the |
struct |
|
When the |
__u32 |
|
Set by the driver, counting the frames (not fields!) in sequence. This field is set for both input and output devices. |
In Note This may count the frames received e.g. over USB, without taking into account the frames dropped by the remote hardware due to limited compression throughput or bus bandwidth. These devices identify by not enumerating any video standards, see Video Standards. |
||
__u32 |
|
This field must be set by applications and/or drivers in
accordance with the selected I/O method. See |
union { |
|
|
__u32 |
|
For the single-planar API and when |
unsigned long |
|
For the single-planar API and when |
|
When using the multi-planar API, contains a userspace pointer to
an array of struct |
|
int |
|
For the single-plane API and when |
} |
||
__u32 |
|
Size of the buffer (not the payload) in bytes for the
single-planar API. This is set by the driver based on the calls to
ioctl VIDIOC_REQBUFS and/or
ioctl VIDIOC_CREATE_BUFS. For the
multi-planar API the application sets this to the number of
elements in the |
__u32 |
|
A place holder for future extensions. Drivers and applications must set this to 0. |
__u32 |
|
The file descriptor of the request to queue the buffer to. If the flag
The Applications should not set If the device does not support requests, then |
-
type v4l2_plane¶
3.5.3. struct v4l2_plane¶
__u32 |
|
The number of bytes occupied by data in the plane (its payload).
Drivers must set this field when Note Note that the actual image data starts at |
__u32 |
|
Size in bytes of the plane (not its payload). This is set by the driver based on the calls to ioctl VIDIOC_REQBUFS and/or ioctl VIDIOC_CREATE_BUFS. |
union { |
|
|
__u32 |
|
When the memory type in the containing struct
|
unsigned long |
|
When the memory type in the containing struct
|
int |
|
When the memory type in the containing struct
|
} |
||
__u32 |
|
Offset in bytes to video data in the plane. Drivers must set this
field when Note That data_offset is included in |
__u32 |
|
Reserved for future use. Should be zeroed by drivers and applications. |
-
type v4l2_buf_type¶
3.5.4. enum v4l2_buf_type¶
|
1 |
Buffer of a single-planar video capture stream, see Video Capture Interface. |
|
9 |
Buffer of a multi-planar video capture stream, see Video Capture Interface. |
|
2 |
Buffer of a single-planar video output stream, see Video Output Interface. |
|
10 |
Buffer of a multi-planar video output stream, see Video Output Interface. |
|
3 |
Buffer for video overlay, see Video Overlay Interface. |
|
4 |
Buffer of a raw VBI capture stream, see Raw VBI Data Interface. |
|
5 |
Buffer of a raw VBI output stream, see Raw VBI Data Interface. |
|
6 |
Buffer of a sliced VBI capture stream, see Sliced VBI Data Interface. |
|
7 |
Buffer of a sliced VBI output stream, see Sliced VBI Data Interface. |
|
8 |
Buffer for video output overlay (OSD), see Video Output Overlay Interface. |
|
11 |
Buffer for Software Defined Radio (SDR) capture stream, see Software Defined Radio Interface (SDR). |
|
12 |
Buffer for Software Defined Radio (SDR) output stream, see Software Defined Radio Interface (SDR). |
|
13 |
Buffer for metadata capture, see Metadata Interface. |
|
14 |
Buffer for metadata output, see Metadata Interface. |
3.5.5. Buffer Flags¶
|
0x00000001 |
The buffer resides in device memory and has been mapped into the application’s address space, see Streaming I/O (Memory Mapping) for details. Drivers set or clear this flag when the ioctl VIDIOC_QUERYBUF, ioctl VIDIOC_QBUF, VIDIOC_DQBUF or VIDIOC_DQBUF ioctl is called. Set by the driver. |
|
0x00000002 |
Internally drivers maintain two buffer queues, an incoming and
outgoing queue. When this flag is set, the buffer is currently on
the incoming queue. It automatically moves to the outgoing queue
after the buffer has been filled (capture devices) or displayed
(output devices). Drivers set or clear this flag when the
|
|
0x00000004 |
When this flag is set, the buffer is currently on the outgoing
queue, ready to be dequeued from the driver. Drivers set or clear
this flag when the |
|
0x00000040 |
When this flag is set, the buffer has been dequeued successfully,
although the data might have been corrupted. This is recoverable,
streaming may continue as normal and the buffer may be reused
normally. Drivers set this flag when the |
|
0x00000080 |
This buffer is part of a request that hasn’t been queued yet. |
|
0x00000008 |
Drivers set or clear this flag when calling the |
|
0x00000010 |
Similar to |
|
0x00000020 |
Similar to |
|
0x00000100 |
The |
|
0x00000400 |
The buffer has been prepared for I/O and can be queued by the application. Drivers set or clear this flag when the ioctl VIDIOC_QUERYBUF, VIDIOC_PREPARE_BUF, ioctl VIDIOC_QBUF, VIDIOC_DQBUF or VIDIOC_DQBUF ioctl is called. |
|
0x00000800 |
Caches do not have to be invalidated for this buffer. Typically applications shall use this flag if the data captured in the buffer is not going to be touched by the CPU, instead the buffer will, probably, be passed on to a DMA-capable hardware unit for further processing or output. This flag is ignored unless the queue is used for memory mapping streaming I/O and reports V4L2_BUF_CAP_SUPPORTS_MMAP_CACHE_HINTS capability. |
|
0x00001000 |
Caches do not have to be cleaned for this buffer. Typically applications shall use this flag for output buffers if the data in this buffer has not been created by the CPU but by some DMA-capable unit, in which case caches have not been used. This flag is ignored unless the queue is used for memory mapping streaming I/O and reports V4L2_BUF_CAP_SUPPORTS_MMAP_CACHE_HINTS capability. |
|
0x00000200 |
Only valid if struct |
|
0x00100000 |
Last buffer produced by the hardware. mem2mem codec drivers set
this flag on the capture queue for the last buffer when the
ioctl VIDIOC_QUERYBUF or
VIDIOC_DQBUF ioctl is called. Due to
hardware limitations, the last buffer may be empty. In this case
the driver will set the |
|
0x00800000 |
The |
|
0x0000e000 |
Mask for timestamp types below. To test the timestamp type, mask out bits not belonging to timestamp type by performing a logical and operation with buffer flags and timestamp mask. |
|
0x00000000 |
Unknown timestamp type. This type is used by drivers before Linux
3.9 and may be either monotonic (see below) or realtime (wall
clock). Monotonic clock has been favoured in embedded systems
whereas most of the drivers use the realtime clock. Either kinds
of timestamps are available in user space via
|
|
0x00002000 |
The buffer timestamp has been taken from the |
|
0x00004000 |
The CAPTURE buffer timestamp has been taken from the corresponding OUTPUT buffer. This flag applies only to mem2mem devices. |
|
0x00070000 |
Mask for timestamp sources below. The timestamp source defines the
point of time the timestamp is taken in relation to the frame.
Logical ‘and’ operation between the |
|
0x00000000 |
End Of Frame. The buffer timestamp has been taken when the last pixel of the frame has been received or the last pixel of the frame has been transmitted. In practice, software generated timestamps will typically be read from the clock a small amount of time after the last pixel has been received or transmitten, depending on the system and other activity in it. |
|
0x00010000 |
Start Of Exposure. The buffer timestamp has been taken when the
exposure of the frame has begun. This is only valid for the
|
3.5.6. enum v4l2_memory¶
|
1 |
The buffer is used for memory mapping I/O. |
|
2 |
The buffer is used for user pointer I/O. |
|
3 |
[to do] |
|
4 |
The buffer is used for DMA shared buffer I/O. |
3.5.6.1. Memory Consistency Flags¶
|
0x00000001 |
A buffer is allocated either in coherent (it will be automatically coherent between the CPU and the bus) or non-coherent memory. The latter can provide performance gains, for instance the CPU cache sync/flush operations can be avoided if the buffer is accessed by the corresponding device only and the CPU does not read/write to/from that buffer. However, this requires extra care from the driver – it must guarantee memory consistency by issuing a cache flush/sync when consistency is needed. If this flag is set V4L2 will attempt to allocate the buffer in non-coherent memory. The flag takes effect only if the buffer is used for memory mapping I/O and the queue reports the V4L2_BUF_CAP_SUPPORTS_MMAP_CACHE_HINTS capability. |
3.5.7. Timecodes¶
The v4l2_buffer_timecode
structure is designed to hold a
SMPTE 12M or similar timecode.
(struct timeval
timestamps are stored in the struct
v4l2_buffer
timestamp
field.)
-
type v4l2_timecode¶
3.5.7.1. struct v4l2_timecode¶
__u32 |
|
Frame rate the timecodes are based on, see Timecode Types. |
__u32 |
|
Timecode flags, see Timecode Flags. |
__u8 |
|
Frame count, 0 … 23/24/29/49/59, depending on the type of timecode. |
__u8 |
|
Seconds count, 0 … 59. This is a binary, not BCD number. |
__u8 |
|
Minutes count, 0 … 59. This is a binary, not BCD number. |
__u8 |
|
Hours count, 0 … 29. This is a binary, not BCD number. |
__u8 |
|
The “user group” bits from the timecode. |
3.5.7.2. Timecode Types¶
|
1 |
24 frames per second, i. e. film. |
|
2 |
25 frames per second, i. e. PAL or SECAM video. |
|
3 |
30 frames per second, i. e. NTSC video. |
|
4 |
|
|
5 |
3.5.7.3. Timecode Flags¶
|
0x0001 |
Indicates “drop frame” semantics for counting frames in 29.97 fps material. When set, frame numbers 0 and 1 at the start of each minute, except minutes 0, 10, 20, 30, 40, 50 are omitted from the count. |
|
0x0002 |
The “color frame” flag. |
|
0x000C |
Field mask for the “binary group flags”. |
|
0x0000 |
Unspecified format. |
|
0x0008 |
8-bit ISO characters. |