37. Camera subsystem

37.1 Introduction

The camera subsystem is built around a double path:

When plugging a camera sensor, it is recommended to use the DCMIPP, a higher performance pipe.

The previous DCMI is recommended only in two noticeable cases:

Note: The DCMIPP inputs pixels from one sensor via the CSI-2 interface, while the DCMI gets pixels from the second sensor via the parallel interface.

The DCMI path offers the following summarized maximum features:

The DCMIPP path offers the following summarized maximum features:

Some over-target use cases are possible, but with specific constraints:

37.2 Camera path main features

37.2.1 DCMI path main features

The DCMI path shows the following features:

See Section 38: Digital camera interface (DCMI) for more details.

37.2.2 DCMIPP path main features

The DCMIPP path groups the DCMIPP pixel pipeline block, its included parallel interface, and the abutted CSI-2-Host and PHY.

The parallel interface shows the following features:

See Section 39: Digital camera interface pixel pipeline (DCMIPP) for more details.

The CSI-2 serial interface shows the following features:

See Section 40: CSI-2 Host (CSI) for more details.

DCMIPP sizings and limitations

The DCMIPP provides three different pipes to process data:

DCMIPP ISP capabilities common to pipe1 and 2, are the following:

The DCMIPP application capabilities, replicated for pipe1 and 2, are listed below:

Output pixel format:

The full features are detailed in Section 38: Digital camera interface (DCMI) .

37.3 Camera subsystem implementation

37.3.1 Hardware settings

The camera subsystem is implemented with the following specific settings:

37.4 Camera subsystem functional description

37.4.1 Camera subsystem block diagram

Figure 386 shows the blocks in the camera subsystem.

Figure 386. Camera subsystem block diagram

Figure 386. Camera subsystem block diagram. The diagram shows the internal components of the camera subsystem and their connections to external sensors. The subsystem includes DCMI, PSSI, DCMIPP, CSI Host, and CSI2-DPHY. External sensors include LowRes // Sensor, LowPerf // Device, HighRes // Sensor, and HighRes CSI Sensor. Connections are labeled with bus widths like 14b//, 16b//, and AXI-M64b.

The block diagram illustrates the internal architecture of the camera subsystem and its interfaces to external sensors. The subsystem is enclosed in a dashed box labeled 'Camera subsystem' and contains the following components:

External sensors are connected as follows:The diagram is labeled with MSV67464V1 in the bottom right corner.

Figure 386. Camera subsystem block diagram. The diagram shows the internal components of the camera subsystem and their connections to external sensors. The subsystem includes DCMI, PSSI, DCMIPP, CSI Host, and CSI2-DPHY. External sensors include LowRes // Sensor, LowPerf // Device, HighRes // Sensor, and HighRes CSI Sensor. Connections are labeled with bus widths like 14b//, 16b//, and AXI-M64b.

Note: The PSSI is not used for camera purposes, but rather for general-purpose parallel input (not described in this section).

37.4.2 Camera subsystem pins and external signal interface

Details about the pins can be found in the respective block sections.

37.4.3 Camera subsystem reset and clocks

The clock diagram of the camera subsystem is shown in Figure 387 .

Figure 387. Camera subsystem clock diagram. This block diagram illustrates the clocking architecture for the camera subsystem. It features three main functional blocks: DCMI, DCMIPP, and CSI2-Host, all contained within a dashed boundary representing the subsystem. External clock inputs include ck_icn_p_cci, ck_ker_csi2_txesc, ck_icn_p_csi2, ck_icn_p_dcmipp, ck_icn_m_dcmipp, ck_ker_csi2, and ck_ker_csiphy. The DCMI block has an internal pixck_pdck_cci clock. The DCMIPP block has internal pclk, ack, and clk_proc clocks. The CSI2-Host block has internal clk_pix, clk_proc, and o_RxByteClkHS clocks. A CSI2-DPHY block is also shown, receiving clk_txesc, o_RxByteClkHS, and Clk_cfg signals, and providing io_Cn and io_Cp signals to external cn_rx_dphy and cp_rx_dphy components. The diagram shows the flow of various clock signals between these internal blocks and the external environment. A small note 'MSV67465V2' is present in the bottom right corner of the diagram area.

Figure 387. Camera subsystem clock diagram

Figure 387. Camera subsystem clock diagram. This block diagram illustrates the clocking architecture for the camera subsystem. It features three main functional blocks: DCMI, DCMIPP, and CSI2-Host, all contained within a dashed boundary representing the subsystem. External clock inputs include ck_icn_p_cci, ck_ker_csi2_txesc, ck_icn_p_csi2, ck_icn_p_dcmipp, ck_icn_m_dcmipp, ck_ker_csi2, and ck_ker_csiphy. The DCMI block has an internal pixck_pdck_cci clock. The DCMIPP block has internal pclk, ack, and clk_proc clocks. The CSI2-Host block has internal clk_pix, clk_proc, and o_RxByteClkHS clocks. A CSI2-DPHY block is also shown, receiving clk_txesc, o_RxByteClkHS, and Clk_cfg signals, and providing io_Cn and io_Cp signals to external cn_rx_dphy and cp_rx_dphy components. The diagram shows the flow of various clock signals between these internal blocks and the external environment. A small note 'MSV67465V2' is present in the bottom right corner of the diagram area.

The camera subsystem has been implemented with the following clock max frequencies:

37.4.4 Streaming from the camera subsystem to slave peripherals

The DCMIPP in the camera subsystem can stream pixels to a set of slave peripherals NPU, HP/GPDMA, and VENC. This avoids to write the full frame into a frame buffer (and allocate this frame buffer).

The streaming is done via a small intermediate buffer in the system memory, organized in double ping-pong buffer. The double buffer must be a power-of-two amount of lines, with a recommended size of 16 lines high each.

The processing is as follows:

  1. 1. The DCMIPP starts writing into the first buffer.
  2. 2. After step 1 completion, the DCMIPP emits an hardware trigger signal to the slave peripheral, and continues to write to the second buffer.
  3. 3. At hardware trigger reception, the slave peripheral reads from the first buffer. It is expected to read and process the buffer faster than the DCMIPP fills the other.
  1. 4. After completing writing the second buffer, the DCMIPP emits again a trigger, and wraps back its write address counter to restart writing into the first buffer.
  2. 5. After finishing reading the second buffer, the slave peripheral wraps back reading into the first buffer.

All DCMIPP pipes can be streamed, but not to all slave peripherals:

The VENC can only get streamed from Pipe1 (only pipe that can output a YUV420 pixel format, needed for its H264 or JPG encode).

A same pipe can be streamed simultaneously to several peripherals if the resolution and pixel format match: after receiving the same broadcasted trigger, the various slave peripherals simply read in parallel from the same shared intermediate buffer.

More information about the streaming can be found in the respective peripheral sections:

37.4.5 Camera subsystem security

The security of the camera subsystem is based on the protection of its register map, and of its master access to the system interconnect, based on default RISUP and RIMU respectively.

More information can be found in Section 3.5.3: RIF infrastructure .

The RISUP differentiates the access right of accesses performed toward the following RIF protected peripheral ID (indexes refer to Table 20: RISUP indexes ):

The RIMU (RIMU_AXI_DCMIPP) gives the access right to the DCMIPP AXI master: CID, S and P are set using the RIF RIMC_ATTR9 register (no inheritance from RISPU to RIMU).

By default, the camera subsystem is expected to work in a nonsecure mode:

For secure applications, the user can put the camera subsystem in S, P, or S-and-P mode, with the following impacts:

The output of the DCMIPP is differentiated from other masters: only the DCMIPP can write into its buffer. This buffer content is known to come from the camera sensor, which makes impossible to fill-in (by software) a fake face to be recognized.

The above recommended non-protected settings are shown in Table 315: Camera subsystem RIF peripheral ID .

The table below lists vertically all camera subsystem blocks, and horizontally the associated RIF peripheral ID.

Table color legend for the recommended settings : gray for a permanently non-protected setting, blue for a temporarily protected versus non-protected, and pink for a permanently protected state: by default, all the settings are non-protected.

Table 315. Camera subsystem RIF peripheral ID

SubsystemBlockAddress rangeStart addressRISUP - slave
94 and 959392
DCMI and PSSIDCMIPPCSI2H
Camera low resolutionDCMIConfiguration0x0000Y--
Pixel Rx0x0A00Y--
Configuration0x0B00Y--
Data in/out0x0C00Y--
Camera high resolutionDCMIPPCommon configuration0x0-Y-
P1 configuration0x0-Y-
P2 configuration0x0-Y-
P3 configuration0x0-Y-
CSIConfiguration control0x0000--Y
Configuration wrap0x1000--Y
CSI DPHY-Rx (no register)----

37.5 Camera subsystem programmable parameters

This section describes the configuration of the various main use cases supported by the camera subsystem. Its purpose is to provide the settings that are global to the camera subsystem, and not provided by the local relevant blocks.

The use cases supported by default are the following:

Some over-target use cases are possible, but with specific constraints:

The limiting factors are

To cope with these limitations, the decimation can be used, ahead of the pipes, to reduce the width and/or height of the frame, and/or deactivate the limiting operators, the demosaicing, downsize, YUV420.

37.5.1 Low-resolution parallel camera and DCMI

This use case is recommended only for backward compatibility, or to grab pictures simultaneously from two sensors (a low and a high resolution).

The steps to setup this path are the following:

37.5.2 High-resolution parallel camera and DCMIPP

This is the default use case to grab a picture from a parallel camera (up to the target 5 Mpixels @30fps). Any RGB or RawBayer picture can be grabbed.

The steps to setup this path are the following:

37.5.3 High-resolution CSI2 camera and DCMIPP

This is the default use-case to grab a picture from a CSI2 camera (up to the target 5 Mpixels @30 fps). Any RGB or RawBayer picture can be grabbed.

The steps to setup this path are the following:

37.5.4 Sensors over target resolution

The sensors over the target resolution (5 Mpixels) are supported with the following restrictions:

After taking into account these restrictions, the configuration itself is the same as for the default high-resolution CSI2 camera, described in previous section.

37.5.5 Sensors over target pixel rate

The sensors over the target pixel rate (150 Mpixel/s average, ~200 Mpixel/s peak) are supported with the following restrictions:

After taking into account these restrictions, the configuration itself is the same as for the default high-resolution CSI2 camera, described in Section 37.5.3 .

37.5.6 Double sensors

Two sensors are simultaneously and independently supported with the following restriction:

The configuration of the double sensors is the same as for the default low-resolution and default high-resolution sensors, described in previous sections.

37.6 Camera subsystem interrupts

Sections for each peripheral describe in details the provided interrupts.

37.7 Camera subsystem registers

All the registers are described in sections for each peripheral.