Filter results by

ARTIK Modules

ARTIK Cloud

Developer

Forums

IoT Marketplace

MIPI Camera

This guide describes how to capture still images or movies from a MIPI camera, as well as how to configure video streaming.

Environment:
– Image capture program: fswebcam
– Video recording and streaming program: ffmpeg
– Video streaming server program: ffserver
– Image preview program: gstreamer (gst-launch development tool)

Connect MIPI Camera Module

Acquire the proper MIPI camera module.
- ARTIK 520 or 1020 board: S5K5EA
- ARTIK 530 or 710 board: SIP-ASRNXS001
The procedure here shows the ARTIK 530/710 MIPI camera accessory board.
Connect it to the ARTIK board. With power disconnected and both boards placed upside down, attach
- the flat cable end marked "PLATFORM Board" to the MIPI Camera connector of the ARTIK 530/710 board
- the other end to the MIPI camera.
The connectors snap together precisely, so align them carefully before applying pressure.
Connect the ARTIK board to a TV or HDMI monitor as described here and power up the system.
Verify that you get a console login screen. (You will not be able to log in on that screen unless you have a USB keyboard attached to the ARTIK board.)
Check video node information. The video node will be /dev/videoXX – for example:

# ls -al /dev/video6
```
crw-rw---- 1 root video 81, 5 Jan  1  2014 /dev/video6
```
The node number will be different between board types. For example:
- ARTIK 520 MIPI is video100
- ARTIK 530 or 710 MIPI is video6
- ARTIK 1020 MIPI is video101
video6 is used in the examples in this section, but you will need to modify it for your system type.

You can use a sysfs feature to list possible video input nodes for your device. Go to /sys/class/video4linux and run ls -al
The device that shows v4l2 ("video for Linux" – the third character is an L) is the device you will be referencing in the commands below.
Read the Choose and Install Programs section to learn about capture programs available. We suggest you start by installing ffmpeg.
Follow the ffmpeg Display Preview topic. Simply copy and paste the specified line with no modifications.

You should immediately see a sharp, clear video image on the monitor.

Choose and Install Programs

Many programs are available for capturing and displaying video streams. Some of the more popular ones are included in the default system firmware image, or can be installed as noted below if necessary.

fswebcam

fswebcam is a simple utility program used to capture an image from a webcam. The captured image can be stored as one or more PNG or JPEG files.

Supported image resolution : VGA(640x480), HD(1280x720) and (5MP)2560x1920
Supported palette format of source image : YUYV

To install from the command line, enter:

Ubuntu:
apt install fswebcam
Fedora:
dnf install fswebcam

ffmpeg

ffmpeg is widely used to encode and decode video streams from a device or file. It also supports grabbing an input stream from V4L2 (Video4Linux2) devices.

Supported resolutions and frame rates

YUYV
– VGA(640x480): 15fps, 10fps and 5fps
– HD(1280x720): 7fps and 5fps
MJPEG
– Either resolution: 30fps, 24fps, 15fps, 10fps and 5fps

To install from the command line, enter:

Ubuntu:
apt install ffmpeg
Fedora:
dnf install ffmpeg

ffserver

You can stream a video with RTSP protocol over the network by using ffserver along with ffmpeg. ffserver is installed as part of the ffmpeg package.

gstreamer

The Gstreamer framework allows you to build video, audio, and other processing pipelines using Gstreamer "plug-ins".

Supported resolution and frame rates for preview:
– VGA(640x480) : 30fps, 24fps, 15fps, 10fps and 5fps

Additionally supported on ARTIK 7:
– HD(1280x720) : 15fps, 10fps and 5fps
– 5MP (2560x1920) : still

To install Gstreamer plug-ins from the command line, enter:

Ubuntu:
apt install gstreamer*
apt install gstreamer1.0-plugins-bad
Fedora:
dnf install gstreamer*

Display Preview on LCD

To clear the LCD screen before displaying a new preview, enter on the command line:

# cat /dev/zero > /dev/fb0

ffmpeg Display Preview

To display a preview screen on the LCD with ffmpeg, this example

ffmpeg -f v4l2 -s 640x480 -r 30 -i /dev/video6 -pix_fmt bgra -f fbdev /dev/fb0

specifies:

-f v4l2 – use V4L2 capture device
-s 640x480 – resolution
-r 30 – frame rate is 30 fps
-i /dev/video6 – input device for recording (MIPI camera)
-pix_fmt bgra – pixel format of LCD screen
-f fbdev /dev/fb0 – output device is frame buffer device.

gstreamer Display Preview

You can display a preview screen on the LCD by using gst-launch with plug-ins appended. You can see a detailed description for each plug-in by executing

gst-inspect [plug-in name]

For example,

gst-launch -e v4l2src queue-size=16 device=/dev/video6 ! video/x-raw-yuv,framerate=30/1,width=640,height=480 ! ffmpegcolorspace ! fbdevsink device=/dev/fb0

specifies:

-e v4l2src queue-size=16 device=/dev/video6 – use v4l2src plugin
video/x-raw-yuv,framerate=30/1,width=640,height=480 – format, framerate, and size of video, respectively
ffmpegcolorspace – do color space conversion between yuv video and fbdevsink
fbdevsink device=/dev/fb0 – output device is frame buffer device.

Rotate the preview screen on the LCD by adding the “videoflip” plug-in to the Gstreamer pipeline, such as in

gst-launch -e v4l2src queue-size=16 device=/dev/video6 ! video/x-raw-yuv,framerate=30/1,width=640,height=480 ! videoflip method=counterclockwise ! ffmpegcolorspace ! fbdevsink device=/dev/fb0

where:

videoflip method=counterclockwise`

specifies the rotation direction.

Capture Images or Movies

fswebcam Image Capture

Refer to online man page for detailed usage information. Typical command line usage:

fswebcam -d [device node] -r [Resolution] -p [palette format] [filename]

ARTIK 710 example

fswebcam -d /dev/video6 -r 2560x1920 -p YUYV -D 1 --no-banner image.jpg

ARTIK 530 example

fswebcam -d /dev/video6 -r 2560x1920 -p YUV420P --no-banner image.jpg

specifies:

-d /dev/video6 – name of device node (MIPI camera)
-r 2560x1920 – resolution
-p YUYV or p YUV420P – format of source image
-D 1 – pre-capture Delay time (seconds). Since a camera usually needs time to process details such as focus, exposure, and white balance, use at least a 1 second pre-capture delay in order to capture a clearer image.
--no-banner – to hide the banner on the JPEG image.

gstreamer Movie Recording (ARTIK 530/710)

The ARTIK 530 and 710 modules incorporate a hardware video codec. These examples take advantage of the hardware features to record .avi and .mp4 videos.

Example 1 – Record a video with no audio:

gst-launch-1.0 -e camerasrc camera-crop-width=1280 camera-crop-height=720 ! nxvideoenc bitrate=12000000 ! avimux ! filesink location=result.avi

Example 2 – Record a video with preview (using a 'tee' plugin to provide a 1-to-N pipe fitting):

gst-launch-1.0 -e camerasrc camera-crop-width=1280 camera-crop-height=720 ! tee name=t \
t. ! queue ! nxvideosink \
t. ! queue ! nxvideoenc bitrate=12000000 ! mp4mux ! filesink location=result.mp4

Example 3 – Record FHD movie by using scaler. Although maximum resolution of camera source is HD(1280x720), recording a FHD(1920x1080) movie using the hardware scaler is possible.

gst-launch-1.0 -e camerasrc camera-crop-width=1280 camera-crop-height=720 ! nxscaler scaler-crop-width=1280 scaler-crop-height=720 scaler-dst-width=1920 scaler-dst-height=1080 ! tee name=t \
t. ! queue ! nxvideosink \
t. ! queue ! nxvideoenc bitrate=17000000 ! avimux ! filesink location=result_fhd.avi

Example 4 – Record a movie by using camera source plugin with sound, with simultaneous display on LCD or HDMI screen (need to install additional gstreamer plugin for encoding sound)

Ubuntu:
apt install gstreamer1.0-plugins-ugly
Fedora:
dnf install gstreamer1-plugins-ugly

gst-launch-1.0 -e camerasrc camera-crop-width=1280 camera-crop-height=720 ! tee name=t \
t. ! queue ! nxvideoenc bitrate=12000000 ! queue ! mux. \
autoaudiosrc ! queue max-size-buffers=1000 ! liveadder start-time-selection=2 start-time=600000000 ! lamemp3enc ! queue ! mux. \
t. ! nxvideosink \
mp4mux name=mux ! filesink location=result_a.mp4

ffmpeg Movie Recording

Refer to online man page for detailed usage information.

Record without Sound

To record video with no audio, use this example as a template for ARTIK 520 or 1020.

ffmpeg -y -f v4l2 -s 640x480 -r 15 -i /dev/video6 -b:v 2048k -vcodec mpeg4 test.avi

Record with Sound

To record a movie with sound, additionally specify audio channels (1:Mono, 2:Stereo), and card number from arecord results (ac: audio channel, hw: card number, -t duration). Refer to the Audio Guide for more examples of ALSA parameters.

For an ARTIK 520/1020 example,

ffmpeg -y -thread_queue_size 2048 -f alsa -ac 1 -i hw:0 -f v4l2 -s 1280x720 -r 7 -i /dev/video6 -b:v 3072k -vcodec mpeg4 -t 20 test.avi

specifies:

-y – overwrite output files
-thread_queue_size 2048 – maximum number of queued packets. Raising this value can avoid audio packet loss on low-latency and high-rate live streams.
-f alsa – use ALSA for audio
-ac 1 – audio channel is mono
-i hw:0 – input audio card is 0
-f v4l2 – use V4L2 capture device
-s 640x480 – resolution
-r 15 – frame rate is 15 fps
-i /dev/video6 – input device for recording (MIPI camera)
-b:v 2048k – target bitrate of recorded video : 2Mbps
-vcodec mpeg4 – use mpeg4 as a video codec
-t 20 – duration of recording is 20 sec.
test.avi – output file name.

For an ARTIK 530/710 example,

ffmpeg -y -itsoffset 0.15 -thread_queue_size 2048 -f alsa -i hw:0 -f v4l2 -s 1280x720 -r 15 -i /dev/video6 -b:v 3072k -vcodec mpeg4 -t 20 test.avi

additionally specifies:

-itsoffset 0.15 – synchronize between audio and video by specified audio offset. For example:
– when audio is faster than video by 0.15 sec : -itsoffset 0.15
– when audio is slower than video by 0.5 sec : -itsoffset -0.5

Recommended itsoffset value to apply to .avi video is 0.15 (sec).

Record with 'vcodec copy' Option

When encoding video using a software codec, rate and resolution are limited by the CPU performance. Maximum frame rate of video encoding with the ARTIK 5 solution is approximately 7 fps for HD (1280x720), requiring use of the -r option to trade off frame rate for resolution.

In order to get full frame rate video with high resolution, it is necessary to bypass the software video codec as follows.

Change the output format of the camera to mjpeg.
Specify -vcodec copy to just copy the stream without encoding with the codec.

ffmpeg -y -thread_queue_size 2048 -f alsa -ac 1 -i hw:0 -f v4l2 -s 1280x720 -vcodec mjpeg -r 30 -i /dev/video6 -vcodec copy -t 20 -loglevel error test.avi

Stream Video

You can stream a video with RTSP protocol over the network by using ffmpeg and ffserver. ffserver receives FFM streams from the ffmpeg instance as input, then streams them over RTSP. ARTIK acts as the RTSP server.

Supported resolutions and frame rates for video streaming

VGA(640x480) : 15fps, 10fps and 5fps
HD(1280x720): 7fps, and 5fps

Configure ffserver with a server configuration file, such as this ffserver.conf file for VGA 15fps streaming.

Port 8090
BindAddress 0.0.0.0

RTSPPort 8091
RTSPBindAddress 0.0.0.0

MaxClients 1000
MaxBandwidth 10000
CustomLog -

<Feed feed1.ffm>
File /tmp/feed1.ffm
ACL Allow 127.0.0.1
</Feed>

<Stream test1.mp4>
Feed feed1.ffm
Format rtp
VideoFrameRate 15
VideoBitRate 5000
VideoSize 640x480
VideoQMin 3
VideoQMax 31
NoAudio
NoDefaults
</Stream>

            

Launch ffserver in the background.
# ffserver -f ffserver.conf &
Run a stream feed with ffmpeg on ARTIK device.
# ffmpeg -f v4l2 -s 640x480 -r 15 -i /dev/video6 http://localhost:8090/feed1.ffm

Receive and Play Streaming Content

On your PC, you can use media players like mplayer or VLC as the RTSP client to receive and play video streams.

mplayer rtsp://[IP Address]:8091/test1.mp4

Note that you run mplayer on the Host PC, not on the ARTIK board, to receive and play the video stream on the host (although you could set up a second ARTIK 530, 710, or 1020 board with an HDMI monitor to act as host).

Last updated on: Nov 22 2017, 11:43 PM