Camera Protocol

The camera protocol is used to configure camera payloads and request their status. It supports photo capture, and video capture and streaming. It also includes messages to query and configure the onboard camera storage.

The Dronecode Camera Manager provides an implementation of this protocol.

Camera Connection

Camera components are expected to follow the Heartbeat/Connection Protocol and sent a constant flow of heartbeats (nominally at 1Hz). Each camera must use a different pre-defined camera component ID: MAV_COMP_ID_CAMERA to MAV_COMP_ID_CAMERA6.

The first time a heartbeat is detected from a new camera, a GCS (or other receiving system) should start the Camera Identification process.

If a receiving system stops receiving heartbeats from the camera it is assumed to be disconnected, and should be removed from the list of available cameras. If heartbeats are again detected, the camera identification process below must be restarted from the beginning.

Basic Camera Operations

The CAMERA_INFORMATION.flags provides information about camera capabilities. It contains a bitmap of CAMERA_CAP_FLAGS values that tell the GCS if the camera supports still image capture, video capture, or video streaming, and if it needs to be in a certain mode for capture, etc.

Camera Identification

The camera identification operation determines what cameras are available/exist (this is carried out before all other operations).

The first time a heartbeat is received from a camera component the GCS will send the camera a MAV_CMD_REQUEST_CAMERA_INFORMATION message. The camera component will then respond with the a COMMAND_ACK message containing a result. On success (result is MAV_RESULT_ACCEPTED) the camera component must then send a CAMERA_INFORMATION message.

GCSCameraHEARTBEAT [cmp id: MAV_COMP_ID_CAMERA] (first)MAV_CMD_REQUEST_CAMERA_INFORMATIONStart timeoutCOMMAND_ACKIf MAV_RESULT_ACCEPTED sendinfo.CAMERA_INFORMATIONGCSCamera

The operation follows the normal Command Protocol rules for command/acknowledgment (if no COMMAND_ACK response is received for MAV_CMD_REQUEST_CAMERA_INFORMATION the command will be re-sent a number of times before failing). If CAMERA_INFORMATION is not received after receiving an ACK with MAV_RESULT_ACCEPTED, the protocol assumes the message was lost, and the cycle of sending MAV_CMD_REQUEST_CAMERA_INFORMATION is repeated. If CAMERA_INFORMATION is still not received after three cycle repeats, the GCS may assume that the camera is not supported.

The CAMERA_INFORMATION response contains the bare minimum information about the camera and what it can or cannot do. This is sufficient for basic image and/or video capture.

If a camera provides finer control over its settings CAMERA_INFORMATION.cam_definition_uri will include a URI to a Camera Definition File. If this URI exists, the GCS will request it (using a standard HTTP GET request), parse it and prepare the UI for the user to control the camera settings. The definition file can be hosted anywhere.

If the camera component provides an HTTP interface, the definition file can be hosted on the camera itself. Otherwise, it can be hosted by any regular, reachable server.

The CAMERA_INFORMATION.cam_definition_version field should provide a version for the definition file, allowing the GCS to cache it. Once downloaded, it would only be requested again if the version number changes.

If a vehicle has more than one camera, each camera will have a different component ID and send its own heartbeat. The GCS should create multiple instances of a camera controller based on the component ID of each camera. All commands are sent to a specific camera by addressing the command to a specific component ID.

Camera Modes

Some cameras must be in a certain mode for still and/or video capture.

The GCS can determine if it needs to make sure the camera is in the proper mode prior to sending a start capture (image or video) command by checking whether the CAMERA_CAP_FLAGS_HAS_MODES bit is set true in CAMERA_INFORMATION.flags.

In addition, some cameras can capture images in any mode but with different resolutions. For example, a 20 megapixel camera would take a full resolution image when set to CAMERA_MODE_IMAGE but only at the current video resolution if it is set to CAMERA_MODE_VIDEO.

To get the current mode, the GCS would send a MAV_CMD_REQUEST_CAMERA_SETTINGS command. The camera component will then respond with the a COMMAND_ACK message containing a result. On success (COMMAND_ACK.result is MAV_RESULT_ACCEPTED) the camera must then send a CAMERA_SETTINGS message. The current mode is the CAMERA_SETTINGS.mode_id field.

The sequence is shown below:

GCSCameraMAV_CMD_REQUEST_CAMERA_SETTINGSStart timeoutCOMMAND_ACKIf MAV_RESULT_ACCEPTED sendinfo.CAMERA_SETTINGSGCSCamera

Command acknowledgment and message resending is handled in the same way as for camera identification (if a successful ACK is received the camera will expect the CAMERA_SETTINGS message, and repeat the cycle - up to 3 times - until it is received).

To set the camera to a specific mode, the GCS would send the MAV_CMD_SET_CAMERA_MODE command with the appropriate mode.

The sequence is shown below:

GCSCameraMAV_CMD_SET_CAMERA_MODEStart timeoutCOMMAND_ACKIf MAV_RESULT_ACCEPTED, modewas changed.GCSCamera

The operation follows the normal Command Protocol rules for command/acknowledgment.

Storage Status

Before capturing images and/or videos, a GCS should query the storage status to determine if the camera has enough free space for these operations (and provide the user with feedback as to the current storage status). The GCS will send the MAV_CMD_REQUEST_STORAGE_INFORMATION command and it expects a STORAGE_INFORMATION response. For formatting (or erasing depending on your implementation), the GCS will send a MAV_CMD_STORAGE_FORMAT command.

Camera Capture Status

In addition to querying about storage status, the GCS will also request the current Camera Capture Status in order to provide the user with proper UI indicators. The GCS will send a MAV_CMD_REQUEST_CAMERA_CAPTURE_STATUS command and it expects a CAMERA_CAPTURE_STATUS response.

Still Image Capture

A camera supports still image capture if the CAMERA_CAP_FLAGS_CAPTURE_IMAGE bit is set in CAMERA_INFORMATION.flags.

To capture an image, the GCS uses the MAV_CMD_IMAGE_START_CAPTURE command. Each time an image is captured, a CAMERA_IMAGE_CAPTURED message is sent back to the GCS.

The CAMERA_IMAGE_CAPTURED message not only tells the GCS the image was captured, it is also intended for geo-tagging.

The capture command can be used to request one single image capture or a time lapse. If the command is set to take more than one single image, the GCS might use the MAV_CMD_IMAGE_STOP_CAPTURE command to stop it.

Video Capture

A camera supports video capture if the CAMERA_CAP_FLAGS_CAPTURE_VIDEO bit is set in CAMERA_INFORMATION.flags.

To start recording videos, the GCS uses the MAV_CMD_VIDEO_START_CAPTURE command. If requested, the CAMERA_CAPTURE_STATUS message is sent to the GCS at a set interval.

To stop recording, the GCS uses the MAV_CMD_VIDEO_STOP_CAPTURE command.

Video Streaming

A camera is capable of streaming video if it sets the CAMERA_CAP_FLAGS_HAS_VIDEO_STREAM bit set in CAMERA_INFORMATION.flags.

When the GCS receives the CAMERA_INFORMATION message and it detects the CAMERA_CAP_FLAGS_HAS_VIDEO_STREAM flag, it will then send the MAV_CMD_REQUEST_VIDEO_STREAM_INFORMATION message to the camera requesting the video streaming configuration. In response, the camera returns a VIDEO_STREAM_INFORMATION message for each stream it supports.

If your camera only provides video streaming and nothing else (no camera features), the CAMERA_CAP_FLAGS_HAS_VIDEO_STREAM flag is the only flag you need to set. The GCS will then provide video streaming support and skip camera control.

Message/Enum Summary

MessageDescription
MAV_CMD_REQUEST_CAMERA_INFORMATIONSend command to request CAMERA_INFORMATION.
CAMERA_INFORMATIONBasic information about camera including supported features and URI link to extended information (cam_definition_uri field).
MAV_CMD_REQUEST_CAMERA_SETTINGSSend command to request CAMERA_SETTINGS.
CAMERA_SETTINGSTimestamp and camera mode information.
MAV_CMD_SET_CAMERA_MODESend command to set CAMERA_MODE.
MAV_CMD_REQUEST_VIDEO_STREAM_INFORMATIONSend command to request VIDEO_STREAM_INFORMATION. This is sent once for each camera when a camera is detected and it has set the CAMERA_CAP_FLAGS_HAS_VIDEO_STREAM flag within the CAMERA_INFORMATION message flags field.
VIDEO_STREAM_INFORMATIONInformation defining a video stream configuration. If a camera has more than one video stream, it would send one of this for each video stream, with their specific configuration. Each stream must have its own, unique stream_id.
MAV_CMD_REQUEST_VIDEO_STREAM_STATUSSend command to request VIDEO_STREAM_STATUS. This is sent whenever there is a mode change (when MAV_CMD_SET_CAMERA_MODE is sent.) It allows the camera to update the stream configuration when a camera mode change occurs.
VIDEO_STREAM_STATUSInformation updating a video stream configuration.
MAV_CMD_REQUEST_STORAGE_INFORMATIONSend command to request STORAGE_INFORMATION.
STORAGE_INFORMATIONStorage information (e.g. number and type of storage devices, total/used/available capacity, read/write speeds).
MAV_CMD_STORAGE_FORMATSend command to format the specified storage device.
MAV_CMD_REQUEST_CAMERA_CAPTURE_STATUSSend command to request CAMERA_CAPTURE_STATUS.
CAMERA_CAPTURE_STATUSCamera capture status, including current capture type (if any), capture interval, available capacity.
MAV_CMD_IMAGE_START_CAPTURESend command to start image capture, specifying the duration between captures and total number of images to capture.
MAV_CMD_IMAGE_STOP_CAPTURESend command to stop image capture.
CAMERA_IMAGE_CAPTUREDInformation about image captured (returned to GPS every time an image is captured).
MAV_CMD_VIDEO_START_CAPTURESend command to start video capture, specifying the frequency that CAMERA_CAPTURE_STATUS messages should be sent while recording.
MAV_CMD_VIDEO_STOP_CAPTURESend command to stop video capture.
CAMERA_IMAGE_CAPTUREDInformation about image captured (returned to GPS every time an image is captured).
MAV_CMD_VIDEO_START_STREAMINGSend command to start video streaming for the given Stream ID (stream_id.) This is mostly for streaming protocols that push a stream. If your camera uses a connection based streaming configuration (RTSP, TCP, etc.), you may ignore it if you don't need it but note that you still must ACK the command, like all MAV_CMD_XXX commands. When using a connection based streaming configuration, the GCS will connect the stream from its side. When a camera offers more than one stream and the user switches from one stream to another, the GCS will send a MAV_CMD_VIDEO_STOP_STREAMING command targeting the current Stream ID followed by a MAV_CMD_VIDEO_START_STREAMING targeting the newly selected Stream ID.
MAV_CMD_VIDEO_STOP_STREAMINGSend command to stop video streaming for the given Stream ID (stream_id.) This is mostly for streaming protocols that push a stream. If your camera uses a connection based streaming configuration (RTSP, TCP, etc.), you may ignore it if you don't need it but note that you still must ACK the command, like all MAV_CMD_XXX commands. When using a connection based streaming configuration, the GCS will disconnect the stream from its side. When a camera offers more than one stream and the user switches from one stream to another, the GCS will send a MAV_CMD_VIDEO_STOP_STREAMING command targeting the current Stream ID followed by a MAV_CMD_VIDEO_START_STREAMING targeting the newly selected Stream ID.
EnumDescription
CAMERA_CAP_FLAGSCamera capability flags (Bitmap). For example: ability to capture images in video mode, support for survey mode etc. Received in CAMERA_INFORMATION.
CAMERA_MODECamera mode (image, video, survey etc.). Received in CAMERA_SETTINGS.
VIDEO_STREAM_TYPEType of stream - e.g. RTSP, MPEG. Received in VIDEO_STREAM_INFORMATION .
VIDEO_STREAM_STATUS_FLAGSBitmap of stream status flags - e.g. zoom, thermal imaging, etc. Received in VIDEO_STREAM_INFORMATION .

results matching ""

    No results matching ""