How do I pass an OpenCV Mat into a C++ Tensorflow graph?

opencv mat type
opencv mat to tensor
opencv mat example
opencv mat::step
opencv foreach
opencv mat python
opencv mat access element
tensorflow c++ api opencv

In Tensorflow C++ I can load an image file into the graph using

tensorflow::Node* file_reader =  tensorflow::ops::ReadFile(tensorflow::ops::Const(IMAGE_FILE_NAME, b.opts()),b.opts().WithName(input_name));
tensorflow::Node* image_reader = tensorflow::ops::DecodePng(file_reader, b.opts().WithAttr("channels", 3).WithName("png_reader"));
tensorflow::Node* float_caster = tensorflow::ops::Cast(image_reader, tensorflow::DT_FLOAT, b.opts().WithName("float_caster"));
tensorflow::Node* dims_expander = tensorflow::ops::ExpandDims(float_caster, tensorflow::ops::Const(0, b.opts()), b.opts());
tensorflow::Node* resized = tensorflow::ops::ResizeBilinear(dims_expander, tensorflow::ops::Const({input_height, input_width},b.opts().WithName("size")),b.opts());

For an embedded application I would like to instead pass an OpenCV Mat into this graph.

How would I convert the Mat to a tensor that could be used as input to tensorflow::ops::Cast or tensorflow::ops::ExpandDims?

It's not directly from a CvMat, but you can see an example of how to initialize a Tensor from an in-memory array in the TensorFlow Android example: https://github.com/tensorflow/tensorflow/blob/0.6.0/tensorflow/examples/android/jni/tensorflow_jni.cc#L173

You would start off by creating a new tensorflow::Tensor object, with something like this (all code untested):

tensorflow::Tensor input_tensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({1, height, width, depth}));

This creates a Tensor object with float values, with a batch size of 1, and a size of widthxheight, and with depth channels. For example a 128 wide by 64 high image with 3 channels would pass in a shape of {1, 64, 128, 3}. The batch size is just used when you need to pass in multiple images in a single call, and for simple uses you can leave it as 1.

Then you would get the underlying array behind the tensor using a line like this:

auto input_tensor_mapped = input_tensor.tensor<float, 4>();

The input_tensor_mapped object is an interface to the data in your newly-created tensor, and you can then copy your own data into it. Here I'm assuming you've set source_data as a pointer to your source data, for example:

const float* source_data = some_structure.imageData;

You can then loop through your data and copy it over:

for (int y = 0; y < height; ++y) {
    const float* source_row = source_data + (y * width * depth);
    for (int x = 0; x < width; ++x) {
        const float* source_pixel = source_row + (x * depth);
        for (int c = 0; c < depth; ++c) {
           const float* source_value = source_pixel + c;
           input_tensor_mapped(0, y, x, c) = *source_value;
        }
    }
}

There are obvious opportunities to optimize this naive approach, and I don't have sample code on hand to show how to deal with the OpenCV side of getting the source data, but hopefully this is helpful to get you started.

Mat, In those days the library was built around a C interface and to store the As a nice bonus if Duration: 0:33 Posted: Dec 31, 2019 So, the data layout in Mat is fully compatible with CvMat, IplImage, and CvMatND types from OpenCV 1.x. It is also compatible with the majority of dense array types from the standard toolkits and SDKs, such as Numpy (ndarray), Win32 (independent device bitmaps), and others, that is, with any array that uses steps (or strides ) to compute the

I had tried to run inception model on the opencv Mat file and following code worked for me https://gist.github.com/kyrs/9adf86366e9e4f04addb. Although there are some issue with integration of opencv and tensorflow. Code worked without any issue for .png files but failed to load .jpg and .jpeg. You can follow this for more info https://github.com/tensorflow/tensorflow/issues/1924

Mat - The Basic Image Container, In those days the library was built around a C interface and to store the image in the As a Duration: 0:33 Posted: Jul 18, 2011 opencv C++ create Mat object from android NV21 image data buffer Tag: android , c++ , opencv I have implemented an android application that starts the camera and send all the preview buffer down to native components using JNI interface.

Here is complete example to read and feed:

Mat image;
image = imread("flowers.jpg", CV_LOAD_IMAGE_COLOR);
cv::resize(image, image, cv::Size(input_height, input_width), 0, 0, CV_INTER_CUBIC);

int depth = 3;
tensorflow::Tensor input_tensor(tensorflow::DT_FLOAT,
                                tensorflow::TensorShape({1, image.rows, image.cols, depth}));

for (int y = 0; y < image.rows; y++) {
    for (int x = 0; x < image.cols; x++) {
        Vec3b pixel = image.at<Vec3b>(y, x);

        input_tensor_mapped(0, y, x, 0) = pixel.val[2]; //R
        input_tensor_mapped(0, y, x, 1) = pixel.val[1]; //G
        input_tensor_mapped(0, y, x, 2) = pixel.val[0]; //B
    }
}

auto result = Sub(root.WithOpName("subtract_mean"), input_tensor, {input_mean});
ClientSession session(root);
TF_CHECK_OK(session.Run({result}, out_tensors));

What is the best practise for passing cv::Mats around, Firstly, my image data is stored in a class (percepUnit) which has a number of cv:: Mat members: class percepUnit { public: cv::Mat image;� pass by reference a mat object OpenCv. Ask Question what does thread t1 do with the mat? copy and paste this URL into your RSS reader.

Tensor convertMatToTensor(Mat &input)
{
    int height = input.rows;
    int width = input.cols;
    int depth = input.channels();

    Tensor imgTensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({height, width, depth}));

    float* p = imgTensor.flat<float>().data();
    Mat outputImg(height, width, CV_32FC3, p);
    input.convertTo(outputImg, CV_32FC3);

    return imgTensor;
}

Operations with images, Since in OpenCV images are represented by the same structure as matrices, A conversion from Mat to C API data structures (C++ only): A call to waitKey() starts a message passing cycle that waits for a key stroke in the "image" window. I'm trying to encode some Mats into a gif file using giflib, but I can't get it to work. Here's how I convert the opencv mat's data into gif pixel: cv::resize(input_mat, input_mat, cv::Size (160,

How do I pass an OpenCV Mat into a C++ Tensorflow graph?, WithName("size")),b.opts()); For an embedded application I would like to instead pass an OpenCV Mat into this graph. How would I convert the Mat to a tensor� This is different from opencv's API. I'm not familiar with it.

How do I pass an OpenCV Mat into a C++ Tensorflow graph?, For an embedded application I would like to instead pass an OpenCV Mat into the C interface, as I only load and do forward inference on static graphs. cv::Mat� Therefore, the first thing you need to be familiar with is how OpenCV stores and handles images. Mat . OpenCV has been around since 2001. In those days the library was built around a C interface and to store the image in the memory they used a C structure called IplImage. This is the one you'll see in most of the older tutorials and educational

How to convert OpenCV Mat object to BufferedImage object using , If you try to read an image using the OpenCV imread() method it returns a Mat object. Creating BufferedImage object − Pass the Input Stream object the file String file = "C:/EXAMPLES/OpenCV/sample.jpg"; Mat image� Mat¶ OpenCV has been around since 2001. In those days the library was built around a C interface and to store the image in the memory they used a C structure called IplImage. This is the one you’ll see in most of the older tutorials and educational materials. The problem with this is that it brings to the table all the minuses of the C language.

Comments
  • what is the number of depth in tensorflow::TensorShape for a greyscale image?