Periodically send data to MATLAB from mexFile

compile mex file matlab
how to run mex file in matlab
matlab mex tutorial
call matlab function in mex file
matlab::mex c++
matlab mex -setup
matlab view mex file

I'm working right now on a Data Acquisition Tool completely written in MATLAB. It was the wish of my colleagues that i write this thing in MATLAB so that they can expand and modify it.

The Software needs to grab a picture from two connected USB cameras. The API for these cameras is written in C++ and is documented -> Here.

Here is the Problem: When i write a mex file which grabs a picture it includes the initialization and configuration-loading of the cameras which takes a long time. When i want to grab the pictures this way it takes MATLAB over 1 second to perform the task. The cameras are able, once initialized, to record and send 100 fps. The minimum frame rate i need is 10 fps. I need to be able to send every recorded picture back to MATLAB. Because the recording session for which the Acquisition Tool is needed takes approx 12 hours and we need a Live Screen with some slight PostProcessing.

Is it possible to generate a loop within the mex File which sends data to MATLAB, then waits for a return signal from MATLAB and continues ? This way i could initialize the cameras and send periodically the images to MATLAB.

I'am a Beginner in C++ and it is quite possible that i don't understand a fundamental concept why this is not possible.

Thank you for any advice or sources where i could look.

Please find below the Code which initializes the Cameras using the Pylon API provided by Basler.

// Based on the Grab_MultipleCameras.cpp Routine from Basler
This routine grabs one frame from 2 cameras connected
via two USB3 ports. It directs the Output to MATLAB.

// Include files to use the PYLON API.
#include <pylon/PylonIncludes.h>
#include <pylon/usb/PylonUsbIncludes.h>
#include <pylon/usb/BaslerUsbInstantCamera.h>
#include <pylon/PylonUtilityIncludes.h>
// Include Files for MEX Generation
#include <matrix.h>
#include <mex.h>   

// Namespace for using pylon objects.
using namespace Pylon;

// We are lazy and use Basler USB namespace
using namespace Basler_UsbCameraParams;

// Standard namespace
using namespace std;

// Define Variables Globally to be remembered between each call
// Filenames for CamConfig
const String_t filenames[] = { "NodeMapCam1.pfs","NodeMapCam2.pfs" };

// Limits the amount of cameras used for grabbing.
static const size_t camerasToUse = 2;

// Create an array of instant cameras for the found devices and 
// avoid exceeding a maximum number of devices.
CBaslerUsbInstantCameraArray cameras(camerasToUse);

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
  // Automagically call PylonInitialize and PylonTerminate to ensure the pylon runtime system.
  // is initialized during the lifetime of this object
  PylonAutoInitTerm autoInitTerm;

    // Get the transport layer factory
    CTlFactory& tlFactory = CTlFactory::GetInstance();

    // Get all attached devices and exit application if no device or USB Port is found.
    DeviceInfoList_t devices;
    ITransportLayer *pTL = dynamic_cast<ITransportLayer*>(tlFactory.CreateTl(BaslerUsbDeviceClass));
    if (pTL == NULL)
      throw RUNTIME_EXCEPTION("No USB transport layer available.");

    if (pTL->EnumerateDevices(devices) == 0)
      throw RUNTIME_EXCEPTION("No camera present.");

    // Create and attach all Pylon Devices. Load Configuration
    for (size_t i = 0; i < cameras.GetSize(); ++i)

    // Open all cameras.

    // Load Configuration and execute Trigger
    for (size_t i = 0; i < cameras.GetSize(); ++i)
      CFeaturePersistence::Load(filenames[i], &cameras[i].GetNodeMap());
    if (cameras[0].IsOpen() && cameras[1].IsOpen())
      mexPrintf("\nCameras are fired up and configuration is applied\n");
  catch (GenICam::GenericException &e)
    // Error handling
    mexPrintf("\nAn exception occured:\n");


You could loop and send images back to MATLAB periodically, but how do you want it to be in the workspace (multiple 2D images, a huge 3D/4D array, cell, etc.)? I think the solution you are looking for is a stateful MEX file, which can be launched with an 'init' or 'new' command, and then called again repeatedly with 'capture' commands for an already initialized camera.

There is an example of how to do this in my GitHub. Start with class_wrapper_template.cpp and modify it for your commands (new, capture, delete, etc.). Here is a rough and untested example of how the core of it might look (also mirrored on Gist.GitHub):

// pylon_mex_camera_interface.cpp
#include "mex.h"
#include <vector>
#include <map>
#include <algorithm>
#include <memory>
#include <string>
#include <sstream>

////////////////////////  BEGIN Step 1: Configuration  ////////////////////////
// Include your class declarations (and PYLON API).
#include <pylon/PylonIncludes.h>
#include <pylon/usb/PylonUsbIncludes.h>
#include <pylon/usb/BaslerUsbInstantCamera.h>
#include <pylon/PylonUtilityIncludes.h>

// Define class_type for your class
typedef CBaslerUsbInstantCameraArray class_type;

// List actions
enum class Action
    // create/destroy instance - REQUIRED
    // user-specified class functionality

// Map string (first input argument to mexFunction) to an Action
const std::map<std::string, Action> actionTypeMap =
    { "new",        Action::New },
    { "delete",     Action::Delete },
    { "capture",    Action::Capture }
}; // if no initializer list available, put declaration and inserts into mexFunction

using namespace Pylon;
using namespace Basler_UsbCameraParams;

const String_t filenames[] = { "NodeMapCam1.pfs","NodeMapCam2.pfs" };
static const size_t camerasToUse = 2;
/////////////////////////  END Step 1: Configuration  /////////////////////////

// boilerplate until Step 2 below
typedef unsigned int handle_type;
typedef std::pair<handle_type, std::shared_ptr<class_type>> indPtrPair_type; // or boost::shared_ptr
typedef std::map<indPtrPair_type::first_type, indPtrPair_type::second_type> instanceMap_type;
typedef indPtrPair_type::second_type instPtr_t;

// getHandle pulls the integer handle out of prhs[1]
handle_type getHandle(int nrhs, const mxArray *prhs[]);
// checkHandle gets the position in the instance table
instanceMap_type::const_iterator checkHandle(const instanceMap_type&, handle_type);

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {

    // static storage duration object for table mapping handles to instances
    static instanceMap_type instanceTab;

    if (nrhs < 1 || !mxIsChar(prhs[0]))
        mexErrMsgTxt("First input must be an action string ('new', 'delete', or a method name).");

    char *actionCstr = mxArrayToString(prhs[0]); // convert char16_t to char
    std::string actionStr(actionCstr); mxFree(actionCstr);

    for (auto & c : actionStr) c = ::tolower(c); // remove this for case sensitivity

    if (actionTypeMap.count(actionStr) == 0)
        mexErrMsgTxt(("Unrecognized action (not in actionTypeMap): " + actionStr).c_str());

    // If action is not 'new' or 'delete' try to locate an existing instance based on input handle
    instPtr_t instance;
    if ( != Action::New && != Action::Delete) {
        handle_type h = getHandle(nrhs, prhs);
        instanceMap_type::const_iterator instIt = checkHandle(instanceTab, h);
        instance = instIt->second;

    //////// Step 2: customize each action in the switch in mexFuction ////////
    switch (
    case Action::New:
        if (nrhs > 1 && mxGetNumberOfElements(prhs[1]) != 1)
            mexErrMsgTxt("Second argument (optional) must be a scalar, N.");

        handle_type newHandle = instanceTab.size() ? (instanceTab.rbegin())->first + 1 : 1;

        // Store a new CBaslerUsbInstantCameraArray in the instance map
        std::pair<instanceMap_type::iterator, bool> insResult = 
            instanceTab.insert(indPtrPair_type(newHandle, std::make_shared<class_type>(camerasToUse)));

        if (!insResult.second) // sanity check
            mexPrintf("Oh, bad news.  Tried to add an existing handle."); // shouldn't ever happen
            mexLock(); // add to the lock count

        // return the handle
        plhs[0] = mxCreateDoubleScalar(insResult.first->first); // == newHandle

        // Get all attached devices and exit application if no device or USB Port is found.
        CTlFactory& tlFactory = CTlFactory::GetInstance();
        // Check if cameras are attached
        ITransportLayer *pTL = dynamic_cast<ITransportLayer*>(tlFactory.CreateTl(BaslerUsbDeviceClass));
        // todo: some checking here... (pTL == NULL || pTL->EnumerateDevices(devices) == 0)

        // Create and attach all Pylon Devices. Load Configuration
        CBaslerUsbInstantCameraArray &cameras = *instance;
        DeviceInfoList_t devices;
        for (size_t i = 0; i < cameras.GetSize(); ++i) {

        // Open all cameras.

        // Load Configuration and execute Trigger
        for (size_t i = 0; i < cameras.GetSize(); ++i) {
          CFeaturePersistence::Load(filenames[i], &cameras[i].GetNodeMap());

        if (cameras[0].IsOpen() && cameras[1].IsOpen()) {
            mexPrintf("\nCameras are fired up and configuration is applied\n");

    case Action::Delete:
        instanceMap_type::const_iterator instIt = checkHandle(instanceTab, getHandle(nrhs, prhs));
        (instIt->second).close(); // may be unnecessary if d'tor does it
        plhs[0] = mxCreateLogicalScalar(instanceTab.empty()); // just info
    case Action::Capture:
        CBaslerUsbInstantCameraArray &cameras = *instance; // alias for the instance

        // TODO: create output array and capture a frame(s) into it
        plhs[0] = mxCreateNumericArray(...);
        pixel_type* data = (pixel_type*) mxGetData(plhs[0]);
        // also for cameras[1]?
        mexErrMsgTxt(("Unhandled action: " + actionStr).c_str());
    ////////////////////////////////  DONE!  ////////////////////////////////

// See github for getHandle and checkHandle

The idea is that you would call it once to init:

>> h = pylon_mex_camera_interface('new');

Then you would call it in a MATLAB loop to get frames:

>> newFrame{i} = pylon_mex_camera_interface('capture', h);

When you are done:

>> pylon_mex_camera_interface('delete', h)

You should wrap this with a MATLAB class. Derive from cppclass.m to do this easily. For a derived class example see pqheap.m.

How can I make memory persistent between calls to a MEX-file in , By default, in a MEX-file, mxCalloc generates non-persistent mxCalloc data. In other words, the memory management facility automatically de-allocates the memory as soon as the MEX-file ends. To run the MEX-file, pass in a single number. to the relevent documentation by entering the following at the MATLAB prompt:. I need to send data and receive data from a C++ application from MATLAB (and later use that code in Simulink). The C++ program use a API to control a drone to handle the WiFi connection. All the common are done by the API, seen command, read sensor etc. But it MUST use a multi-threaded architecture to work.

Instead of sending data to MATLAB you should make your mex file store camera related settings so that it does not initialize in each call. One way to do this is to use two modes of calls for your mex file. An 'init' call and a call to get data. Pseudo code in MATLAB would be

cameraDataPtr = myMex('init');
while ~done
   data = myMex('data', cameraDataPtr);

In your mex file, you should store the camera settings in a memory which is persistent across calls. One way to do this is using 'new' in c++. You should return this memory pointer as an int64 type to MATLAB which is shown as cameraDataPtr in the above code. When 'data' is asked for you should take cameraDataPtr as input and cast back to your camera settings. Say in C++, you have a CameraSettings object which stores all data related to camera then, a rough pseudo code in c++ would be

if prhs[0] == 'init' { // Use mxArray api to check this
  cameraDataPtr = new CameraSettings; // Initialize and setup camera
  plhs[0] = createMxArray(cameraDataPtr); // Use mxArray API to create int64 from pointer
} else {
   // Need data
   cameraDataPtr = getCameraDataPtr(plhs[1]);
   // Use cameraDataPtr after checking validity to get next frame

This works because mex files stay in memory once loaded until you clear them. You should use mexAtExit function to release camera resource when the mex file is unloaded from memory. You could also use 'static' to store your camera settings in c++ if this is the only place your mex file is going to be used. This will avoid writing some mxArray handling code for returning your c++ pointer.

If you wrap the call to this mex file inside a MATLAB object you can control the initialization and run-time process more easily and present a better API to your users.

Data from MEX-File to MATLAB - MATLAB Answers, Suppose I want to find all divisors of an input to a MEX-File and return them to the user If you pass in a single value of 10 you would end up creating outputs for  As of MATLAB R2018a, write your C++ MEX functions using these APIs, which support C++11 programming features. These APIs, based on the matlab::data::Array class, provide better type safety, array bounds checking, and support for modern C++ constructs to simplify coding.

I ran into the same problem and wanted to use a Basler camera with the mex API in Matlab. The contributions and hints here definitely helped me to come up with some ideas. However, there is a much simpler solution than the previously proposed one. It's not necessary to return the camera pointer to Matlab back, because objects will stay in memory across multiple mex calls. Here is a working code which I programmed with the new mex C++ API. Have fun with it.

Here is the C++ File which can be compiled with mex:

#include <opencv2/core/core.hpp>
#include <opencv2/opencv.hpp>
#include <pylon/PylonIncludes.h>
#include <pylon/usb/PylonUsbIncludes.h>
#include <pylon/usb/BaslerUsbInstantCamera.h>
#include <pylon/PylonUtilityIncludes.h>
#include "mex.hpp"
#include "mexAdapter.hpp"
#include <chrono>
#include <string>
using namespace matlab::data;
using namespace std;
using namespace Pylon;
using namespace Basler_UsbCameraParams;
using namespace GenApi;
using namespace cv;
using matlab::mex::ArgumentList;

class MexFunction : public matlab::mex::Function{
    matlab::data::ArrayFactory factory;
    double Number = 0;
    std::shared_ptr<matlab::engine::MATLABEngine> matlabPtr = getEngine();
    std::ostringstream stream;
    Pylon::CInstantCamera* camera;
    INodeMap* nodemap;
    double systemTime;
    double cameraTime;


    void operator()(ArgumentList outputs, ArgumentList inputs) {

        try {
        Number = Number + 1;


            matlab::data::CharArray InputKey = inputs[0];

            stream << "You called: " << InputKey.toAscii() << std::endl;

            // If "Init" is the input value
            if(InputKey.toUTF16() == factory.createCharArray("Init").toUTF16()){

                // Important: Has to be closed
                IPylonDevice* pDevice = CTlFactory::GetInstance().CreateFirstDevice();
                camera = new CInstantCamera(pDevice);

                nodemap = &camera->GetNodeMap();


                camera->RegisterConfiguration( new CSoftwareTriggerConfiguration, RegistrationMode_ReplaceAll, Cleanup_Delete);

                CharArray DeviceInfo = factory.createCharArray(camera -> GetDeviceInfo().GetModelName().c_str());

                stream << "Message: Used Camera is " << DeviceInfo.toAscii() << std::endl;

            // If "Grab" is  called
            if(InputKey.toUTF16() == factory.createCharArray("Grab").toUTF16()){
                    static const uint32_t c_countOfImagesToGrab = 1;
                    camera -> StartGrabbing(c_countOfImagesToGrab);
                    CGrabResultPtr ptrGrabResult;
                    Mat openCvImage;
                    CImageFormatConverter formatConverter;
                    CPylonImage pylonImage;
                    while (camera -> IsGrabbing()) {
                        camera -> RetrieveResult(5000, ptrGrabResult, TimeoutHandling_ThrowException);
                        if (ptrGrabResult->GrabSucceeded()) {
                            formatConverter.Convert(pylonImage, ptrGrabResult);

                            Mat openCvImage = cv::Mat(ptrGrabResult->GetHeight(), ptrGrabResult->GetWidth(), CV_8UC1,(uint8_t *)pylonImage.GetBuffer(), Mat::AUTO_STEP);

                            const size_t rows = openCvImage.rows;
                            const size_t cols = openCvImage.cols;

                            matlab::data::TypedArray<uint8_t> Yp = factory.createArray<uint8_t>({ rows, cols });

                            for(int i = 0 ;i < openCvImage.rows; ++i){

                                for(int j = 0; j < openCvImage.cols; ++j){

                                    Yp[i][j] =<uint8_t>(i,j);
                            outputs[0] =  Yp;

            // if "Delete"
            if(InputKey.toUTF16() == factory.createCharArray("Delete").toUTF16()){

                stream << "Camera instance removed" << std::endl;
                Number = 0;

        // ----------------------------------------------------------------
        stream << "Anzahl der Aufrufe bisher: " << Number << std::endl;
        // ----------------------------------------------------------------

        catch (const GenericException & ex) {
            matlabPtr->feval(u"disp", 0, std::vector<Array>({factory.createCharArray(ex.GetDescription()) }));

    void displayOnMATLAB(std::ostringstream& stream) {
        // Pass stream content to MATLAB fprintf function
        matlabPtr->feval(u"fprintf", 0,
                         std::vector<Array>({ factory.createScalar(stream.str()) }));
        // Clear stream buffer

This mex File can be called from Matlab with the following commands:

% Initializes the camera. The camera parameters can also be loaded here.

% Camera image is captured and sent back to Matlab
[Image] = NameOfMexFile('Grab');

% The camera connection has to be closed.

Optimization and improvements of this code are welcome. There are still problems with the efficiency of the code. An image acquisition takes about 0.6 seconds. This is mainly due to the cast from a cv::mat image to a TypedArray which is necessary to return it back to Matlab. See this line in the two loops: Yp[i][j] =<uint8_t>(i,j);

I have not figured out how to make this more efficient yet. Furthermore, the code cannot be used to return multiple images back to Matlab.

Maybe someone has an idea or hint to make the conversion from cv::mat to a Matlab array type faster. I already mentioned the problem in another post. See here: How to Return a Opencv image cv::mat to Matlab with the Mex C++ API

Build MEX function or engine application - MATLAB mex, MATLAB automatically selects a compiler, if installed, based on the language of the filenames arguments. Data Types: char. api — Release-specific API Hello, I have a quick question regarding converting between data types in a mex file. I am interfacing to some hardware which calls a WORD pointer in its measuring function, but I'm having trouble passing this quantity back to Matlab as the output appears as a bunch of zeros.

Upgrade MEX Files to Use Graphics Objects, MATLAB Version 8.4 (R2014b) changes the data type of handles to graphics Before Version 8.4, MEX files used the C/C++ and Fortran API Reference library to pass the handle to the mexGet and mexSet APIs, then MATLAB automatically  Using Intel® Data Analytics Acceleration Library on Matlab* By Ying H. , published on August 15, 2016 Intel ® Data Analytics Acceleration Library (Intel ® DAAL) is high performance library, which provides a rich set of algorithms, ranging from the most basic descriptive statistics for datasets to more advanced data mining and machine

How do I create a structure array from a C MEX-file and pass it to , structure array from a C Learn more about structure, mex, mxarray, pass, example MATLAB. //Create mxArray data structures to hold the data. //to be assigned for the will automatically take care. // of allocating required  % startdate - the market data will be requested from this data % enddate - the market data will be requested till this date % interval - the market data will be returned in this intervals % supported intervals are '1d', '5d', '1wk', '1mo', '3mo' % % OUTPUT: % data - is a retrieved dataset returned as a table

Entry point to C/C++ MEX function built with C Matrix API, To write MEX functions using modern C++ features and the MATLAB Data API, see When you invoke a MEX function, MATLAB automatically seeds nlhs , plhs , nrhs To experiment with passing input arguments, build the mexfunction.c  A version of this software for running 32-bit MATLAB on 64-bit Windows can be found here. In order to accomplish very fast port I/O using a NO COST add-on to MATLAB, we have developed a C++ extension (mex-file) that uses native methods to access low-level hardware. This mex-file is named io64.mexw64.

  • wow. thank you very much. great effort and great answer ! this helped me alot.
  • @JulianCarpenter No Problem. I'm sure there are mistakes and I'd love to integrate the actual syntax when you get it worked out.
  • can i ask a follow up question. i tried to implement your code and i am stumbling a bit ... in the line CBaslerUsbInstantCameraArray &cameras = instance; you are initializing a object reference with the smart pointer called instance .... well my compiler is complaining about it. i work with Visual Studio 2015 and get this Output: error C2440: 'initializing': cannot convert from 'instPtr_t' to 'class_type &' ... if you could point me - ;) - in the right direction for a solution it would be aweseome !
  • @JulianCarpenter My mistake, instance is of type shared_ptr<class_type> not just class_type. When using an instance, you have to dereference the shared_ptr. Try CBaslerUsbInstantCameraArray &cameras = *instance; instead. Wish I could compile your code... Of if that fails, try using it directly, like (*instance)[i].Attach().
  • thank you again ! you just saved my day.... by indexing the dereferenced pointer it finally worked. the cameras stayed open through multiple calls. when i finish the whole thing i'll upload it and give you the link ! thanks again
  • great answer, thank you very much. it helped me alot !