The P6 Model range with thermal sensors allows users to obtain thermal raw data from the camera for use in 3rd-party software.
This can be done with the event stream SDK which can be accessed by following the instructions here:
The SDK contains code samples for various use cases. There is a player function, audio upload, MxPeg to MjPeg converter and the two functions covered here: Live stream and thermal raw data stream.
The SDK is available for different operating systems.
Some tips for your decision of the OS to use:
The Windows version will require Microsofts Visual Basic 2015 or newer.
You will need to purchase this if you don’t have it yet.
Follow the intructions in the “ReadMe” file on how to compile the modules into VB-projects.
The best Linux distro to run the SDK on at the moment (June 2019) is Fedora.
Currently, the install package for Linux has some trouble with executing on other Distros and might fail.
I tested Debian 9 and Mint, and they refused to run the SDK. This topic will show up again later in this article.
MacOS installation will requre additional installation of “XCode” and the “Command Line Tools” through Apples Store or, for older MacOS versions from
https://developer.apple.com/download/more (Apple account required)
This article will show an example installation on Fedora Linux, which is pretty similar to the Mac OS install.
Getting Started:
Download the SDK archive file to any location on your computer. I just dropped them into “Downloads”.
Extract the archive. The result shold look similar to this:
Check the “doc” folder and open “ReadMe.txt”. This lists the content of the SDK and explains the fuction of each module.
For each one of this modules, there is an seperate example and the short manual on how to run it.
We will focus on live-stream and thermal-raw packages.
Let’s see some live images first:
Navigate to the “live-stream” folder in your systems Terminal or Console.
If your camera is configured to use any other admin password than the default “meinsm” one, you will need to change the credentials in “AuthorizationHandler.cpp”
Look for these lines, and replace “admin” and “meinsm” with the correct credentials.
A word advice here: Each time you need to change camera credentials, you will need to also save the changes in AuthorizationHandler.cpp and run the following “make” again.
Now comes the most tricky part in this process.
Typing “make” into the command line will compile the code within the /live-stream folder to finally run the module. It will create a “build” folder and include the code along with the AutorizationHandler data into the final script.
You might encounter error messages about missing commands or programs. Fedora’s packet manager was quite helpfull for me, i installed “gcc” and “g++” packets and went back to run the “make” again without any issues.
At this point, your Linux system might display a lot of errors referring to the “fPIC” option. Currently this is not available in the download package, and you might as well need to switch to Fedora.
UPDATE:
If you receive “pie”-related error messages during bulid, you can edit each makefile in the package to adjust to “non-pie” Systems:
Find the line “CXXFLAGS” and add the options “-no-pie -fPIC” ** as in this example:
CXXFLAGS += -I$(INCLUDE_DIR) -std=c++11 -Wall -ggdb -no-pie -fPIC
(tested on Linux Mint 19.3 Tricia 64-bit)
If you missed preparing your MAC with Xcode and the matching command line tools, your Mac will let you know now. Get the Xcode and tools packages from above mentioned URL and prepare for about 5 GB of download. Make sure to download the correct versions matching your OS, or you will end up downloading these packages multiple times.
In the end, a “good” make execution should produce an output like this:
--------------------------------------------
*##Building file: main.cpp*
*##Invoking: GCC C++ Compiler*
*g++ -I../..//include -std=c++11 -Wall -ggdb -c -MMD -MP -MF"build/./main.d" -MT"build/./main.d" -o "build/./main.o" "main.cpp"*
*##Finished building: main.cpp*
*##Building file: SinkVideo.cpp*
*##Invoking: GCC C++ Compiler*
*g++ -I../..//include -std=c++11 -Wall -ggdb -c -MMD -MP -MF"build/./SinkVideo.d" -MT"build/./SinkVideo.d" -o "build/./SinkVideo.o" "SinkVideo.cpp"*
*##Finished building: SinkVideo.cpp*
*##Building file: SinkAudio.cpp*
*##Invoking: GCC C++ Compiler*
*g++ -I../..//include -std=c++11 -Wall -ggdb -c -MMD -MP -MF"build/./SinkAudio.d" -MT"build/./SinkAudio.d" -o "build/./SinkAudio.o" "SinkAudio.cpp"*
*##Finished building: SinkAudio.cpp*
*##Building file: AuthorizationHandler.cpp*
*##Invoking: GCC C++ Compiler*
*g++ -I../..//include -std=c++11 -Wall -ggdb -c -MMD -MP -MF"build/./AuthorizationHandler.d" -MT"build/./AuthorizationHandler.d" -o "build/./AuthorizationHandler.o" "AuthorizationHandler.cpp"*
*##Finished building: AuthorizationHandler.cpp*
*##Building file: NotificationListener.cpp*
*##Invoking: GCC C++ Compiler*
*g++ -I../..//include -std=c++11 -Wall -ggdb -c -MMD -MP -MF"build/./NotificationListener.d" -MT"build/./NotificationListener.d" -o "build/./NotificationListener.o" "NotificationListener.cpp"*
*##Finished building: NotificationListener.cpp*
*##Building file: build/./main.o*
*g++ -I../..//include -std=c++11 -Wall -ggdb -o build/live-stream build/./main.o build/./SinkVideo.o build/./SinkAudio.o build/./AuthorizationHandler.o build/./NotificationListener.o -L../..//lib/Linux -leventstreamclient*
*##Finished building: build/./main.o*
Run ./build/live-Stream <CAMERA-IP>
The script should start and give messages like this:
EventStream client SDK live stream example
Creating SinkVideo
created file output for YUV video stream: video_stream.raw
Creating SinkAudio
created file output for raw PCM16 audio stream: audio_stream.raw
Creating AuthorizationHandler
Creating NotificationListener
Starting EventStream client SDK 1.1.0-3
Initializing streaming module
listener got message: 0: {"result":{"eventstream":{"version_major":1,"version_minor":65},"imageengine":{"prefix":"","maj":2,"min":4,"sub":2,"rev":1,"suffix":""},"camera":"MX-V5.2.3.30"},"error":null,"id":0}
listed
received video frame res: 1600x600 type: YUV ts: 1561468771:983824 size: 1440000 bytes
received video frame res: 1600x600 type: YUV ts: 1561468772:383825 size: 1440000 bytes
This will continue until you stop the process using 'CTRL+C` and will fill up your available disk space eventually.
The code is only demo-style to give developers an first glance, file handling will need to be programmed into the final application.
You can now use VLC player to view the stream:
On Linux, enter
vlc --demux rawvideo --rawvid-fps 12 --rawvid-width $WIDTH --rawvid-height $HEIGHT --rawvid-chroma I420 video_stream.raw
*(replace $WIDTH and $HEIGHT with actal values, in my example WIDTH=1600 *
and HEIGHT=600)
As result, you should see the first live images in VLC player.
OK, what about the thermal data?
If you were able to follow the process until now, there’s not many more steps.
If VLC and the live-stream script are still running, cancel them.
Navigate to the /samples/thermal-raw folder.
Remember to adjust the “AuthorizationHandler.cpp” to your cameras admin credentials and save.
Enable raw data in your x6-thermal camera:
Again, run the “make” command, and wait for it fo finish. Output will look similar to the results in /live-stream.
Run
./build/thermal-raw <CAMERA-IP>
Stream will begin and display something similar to this:
EventStream client SDK thermal raw data example
Creating SinkVideo using stream as basename for frames and thermal raw data
Creating dummy SinkAudio
Creating AuthorizationHandler
Creating NotificationListener
Starting EventStream client SDK 1.1.0-3
Initializing streaming module
listener got message: 0: {"result":{"eventstream":{"version_major":1,"version_minor":65},"imageengine":{"prefix":"","maj":2,"min":4,"sub":2,"rev":1,"suffix":""},"camera":"MX-V5.2.3.30"},"error":null,"id":0}
received video frame #1 type: BGRA ts: 1561469478478648
-> wrote raw rgb image: 1228800 bytes, res: 640x480
Got thermal raw data from right sensor
-> wrote thermal raw data of sensor 0 169344 bytes, res: 336x252
-> converted thermal raw data of sensor 0 to integer csv file
-> converted thermal raw data of sensor 0 to Celsius csv file
The directory /saples/thermal-raw will start to fill up with thes types of files:
- The decoded RGB frame of the video stream
- The thermal raw data stored as binary binary output as it is received from the camera (if thermal raw export is activated)
- Thermal data converted to numeric values. The data is stored as integer values in a csv file.
- Thermal data converted to degrees Celsius. The data is stored as float values in a csv file. (only for TR sensors)
The single .csv files can then be viewed in any table calculation software:
Thermal values for each pixel of the sensor are listed in this table.
Update on .raw file processing:
The data exported by the SDK is decoded raw data. Image decoder supports two output formats: YUV420 and RGB (bgra). Which one is used needs to be specified while stsarting up the SDK. The sample program “thermal-raw” by default specifies this to be RGB:
Please see main.cpp:
/*
* Create a new eventstream client SDK object
*/
MxPEG_SDK::shared_ptr_t client = MxPEG_SDK_Factory::create(sinkAudio, sinkVideo, MxPEG_ImageMode::im_RGB, authHandler,notificationListener);
The RGB image output is plain image data that does not include further metadata like resolution or pixel format. Such formats are usually internally used by image processing tools, but rarely stored in files. To open such a file you’ll need to manually provide the necessary metadata (resolution, pixel format, …) Because of this, not many image viewers support loading such files.
The RGB output created by the SDK is stored using ARGB8888 pixel format. (See https://en.wikipedia.org/wiki/RGBA_color_model). A simple image viewer that supports such plain formats is vooya (You can download the Linux version for free here: https://www.offminor.de/downloads.html). There is also an online viewer for such files at rawpixels.net (use RGB32, BGRA).
The .raw files contain the raw data as it is generated by the thermal sensor. I am not aware of any picture viewer that could simply visualize this data.
The data structure itself is described in the header file MxPEG_ThermalRawData.hpp of the SDK. For each pixel the sensor generates a 14 bit value numeric value that represents a temperature. How to interpret this numeric value depends on the operation mode of the sensor. To covert this numeric value to a temperature you’ll also need a sensor (model) specific formula.
A far simpler way to access the sensor data is to use the convertTo() function of the SDK. This already knows all the relevant operational parameters of the sensor and can convert the raw value of a pixel coordinate to Celsius or Fahrenheit.