Hot questions for Using Neural networks in matcaffe


I thought we might be able to compile a Caffeinated description of some methods of performing multiple category classification.

By multi category classification I mean: The input data containing representations of multiple model output categories and/or simply being classifiable under multiple model output categories.

E.g. An image containing a cat & dog would output (ideally) ~1 for both the cat & dog prediction categories and ~0 for all others.

  1. Based on this paper, this stale and closed PR and this open PR, it seems caffe is perfectly capable of accepting labels. Is this correct?

  2. Would the construction of such a network require the use of multiple neuron (inner product -> relu -> inner product) and softmax layers as in page 13 of this paper; or does Caffe's ip & softmax presently support multiple label dimensions?

  3. When I'm passing my labels to the network which example would illustrate the correct approach (if not both)?:

    E.g. Cat eating apple Note: Python syntax, but I use the c++ source.

    Column 0 - Class is in input; Column 1 - Class is not in input

    [[1,0],  # Apple
     [0,1],  # Baseball
     [1,0],  # Cat
     [0,1]]  # Dog


    Column 0 - Class is in input

    [[1],  # Apple
     [0],  # Baseball
     [1],  # Cat
     [0]]  # Dog

If anything lacks clarity please let me know and I will generate pictorial examples of the questions I'm trying to ask.


Nice question. I believe there is no single "canonical" answer here and you may find several different approaches to tackle this problem. I'll do my best to show one possible way. It is slightly different than the question you asked, so I'll re-state the problem and suggest a solution.

The problem: given an input image and a set of C classes, indicate for each class if it is depicted in the image or not.

Inputs: in training time, inputs are pairs of image and a C-dim binary vector indicating for each class of the C classes if it is present in the image or not.

Output: given an image, output a C-dim binary vector (same as the second form suggested in your question).

Making caffe do the job: In order to make this work we need to modify the top layers of the net using a different loss. But first, let's understand the usual way caffe is used and then look into the changes needed. The way things are now: image is fed into the net, goes through conv/pooling/... layers and finally goes through an "InnerProduct" layer with C outputs. These C predictions goes into a "Softmax" layer that inhibits all but the most dominant class. Once a single class is highlighted "SoftmaxWithLoss" layer checks that the highlighted predicted class matches the ground truth class.

What you need: the problem with the existing approach is the "Softmax" layer that basically selects a single class. I suggest you replace it with a "Sigmoid" layer that maps each of the C outputs into an indicator whether this specific class is present in the image. For training, you should use "SigmoidCrossEntropyLoss" instead of the "SoftmaxWithloss" layer.


I have a data set which form is matlab file. The data set contains 600,000 samples and every sample is a matrix of 7-by-256. My data is not image but signal. I want to use CNN of caffe to train the data. So how can I convert it to LMDB as my input of CNN.

I'm badly need the solution!


Converting data in matlab directly to lmdb might be a little tricky.

Why don't you try exporting your data to hdf5 binary files (supported both by matlab and caffe)?

Here is an answer describing how this can be done.


With caffe_.cpp being a private function, when I call functions like caffe.reset_all(), there is always an error telling me it cannot find caffe_.cpp.

So how to use that in MATLAB?


You can find 'caffe_.cpp' under matlab/+caffe/private/. Make sure you cloned caffe git properly and that you built the matlab interface:

~$ make matcaffe


My data set is a HDF5 file consists of data with shape [129028,1,12,1024] and label of shape [129028,1,1,1]. But when I run solver.prototxt, I get the error message:

I0413 08:54:34.689985 17769 hdf5.cpp:32] Datatype class: H5T_FLOAT
F0413 08:54:40.661201 17769 hdf5_data_layer.cpp:53] Check failed:   
hdf_blobs_[i] ->shape(0) == num (1 vs. 1024) 
*** Check failure stack trace: ***


It looks like you saved your hdf5 from matlab, rather than python (judging by your previous question). When saving data from Matlab one has to remember that Matlab stores multi-dimensions arrays in memory in colums-first fashion (fortran style), while python, caffe and many other applications expects multi-dimension arrays in row-first fashion (C style). Thus, you need to permute the data in matlab before saving it to hdf5 for caffe. See this answer for more details.

I suspect that if you run h5ls in shell on the hdf5 file you stored you'll notice that the shape of the stored arrays are actually

data   [1024, 12, 1, 129028]
label  [1, 1, 1, 129028]