Hot questions for Using Neural networks in opencv3.0

Question:

I need to use a neural network in my OpenCV (version 3.0) project. I've created and trained neural network and it works, but if I want to load neural network from YML file, it doesn't predict.

This is a code where I creat, train and save my neural network:

FileStorage fs("nn.yml", FileStorage::WRITE);

int input_neurons = 7;
int hidden_neurons = 100;
int output_neurons = 5;

Ptr<TrainData> train_data = TrainData::loadFromCSV("data.csv", 10, 7, 12);

Ptr<ANN_MLP> neural_network = ANN_MLP::create();
neural_network->setTrainMethod(ANN_MLP::BACKPROP);
neural_network->setBackpropMomentumScale(0.1);
neural_network->setBackpropWeightScale(0.05);
neural_network->setTermCriteria(TermCriteria(TermCriteria::MAX_ITER, (int)10000, 1e-6));

Mat layers = Mat(3, 1, CV_32SC1);
layers.row(0) = Scalar(input_neurons);
layers.row(1) = Scalar(hidden_neurons);
layers.row(2) = Scalar(output_neurons);

neural_network->setLayerSizes(layers);
neural_network->setActivationFunction(ANN_MLP::SIGMOID_SYM, 1, 1); 

neural_network->train(train_data);

if (neural_network->isTrained()) { 
    neural_network->write(fs);
    cout << "It's OK!" << endl; 
}

But next time, if I want to load it from YML file:

Ptr<ANN_MLP> neural_network = Algorithm::load<ANN_MLP>("nn.yml", "neural_network");

I get the output:

[-1.#IND, -1.#IND, -1.#IND, -1.#IND, -1.#IND]

[-1.#IND, 1.0263158, 1.0263158, 1.0263158, 1.0263158]

[1.0263158, 1.0263158, 1.0263158, 1.0263158, 1.0263158]

[-1.#IND, -1.#IND, -1.#IND, -1.#IND, -1.#IND]

Ptr<ANN_MLP> neural_network = Algorithm::load<ANN_MLP>("nn.yml");

This line cause that I get an error:

OpenCV Error: Unspecified error (The node is neither a map nor an empty collecti on) in cvGetFileNodeByName, file C:\builds\master_PackSlave-win64-vc12-shared\op encv\modules\core\src\persistence.cpp, line 739

What am I doing wrong? Where is the problem?


Answer:

You can use save and load, or write and read, but you shouldn't mix them.

So you either need to do:

// Save
neural_network->save("nn.yml");

// Load
Ptr<ANN_MLP> nn = Algorithm::load<ANN_MLP>("nn.yml");

or:

// Write
neural_network->write(fs);

// Read
FileStorage ffs("nn.yml", FileStorage::READ);
Ptr<ANN_MLP> nn = Algorithm::read<ANN_MLP>(ffs.root());

Question:

I wanted to know what is the correct way to save a tensorflow model that I have trained in python so that I can import it in OpenCV using the dnn module of opencv. This is my Tensorflow graph

X = tf.placeholder(tf.float32, [None,training_set.shape[1]],name = 'X')
Y = tf.placeholder(tf.float32,[None,training_labels.shape[1]], name = 'Y')

A1 = tf.contrib.layers.fully_connected(X, num_outputs = 50, activation_fn = tf.nn.relu)
A1 = tf.nn.dropout(A1, 0.8)
A2 = tf.contrib.layers.fully_connected(A1, num_outputs = 2, activation_fn = None)
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = A2, labels = Y))
global_step = tf.Variable(0, trainable=False)
start_learning_rate = 0.001
learning_rate = tf.train.exponential_decay(start_learning_rate, global_step, 100, 0.1, True )
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)

As you can see it doesn't contain any variables. So my question is how should this graph be saved in Tensorflow so that it can be loaded using cv::dnn::readNetFromTensorflow. Should I save the model as .pb or .pbtxt file. And will the .pb or .pbtxt contain the graph as well as the weights or just the graph ??. How to load both the graph and the weights in OpenCV ??.


Answer:

The code that belongs to OP posted link is posted here. URL may change, code renamed or vanished. Therefore I've posted the code to where it is referred by OP.

I guess a first question is how to save the graph at least to load it in TensorFlow again? Because you need to find a way to restore it. There is some way to do it:

-- Save

# Save a graph definition (once)
tf.train.write_graph(sess.graph.as_graph_def(), "", "graph.pb")

# Weights initialization
sess.run(tf.global_variables_initializer())

# Training
...

# Save a checkpoint (weights only, no graph definition)
saver = tf.train.Saver()
saver.save(sess, 'tmp.ckpt')

-- Freeze (merge graph definition with weights, remove training-only nodes)

python ~/tensorflow/tensorflow/python/tools/freeze_graph.py \
  --input_graph=graph.pb \
  --input_checkpoint=tmp.ckpt \
  --output_graph=frozen_graph.pb \
  --output_node_names="NameOfOutputNode"

Only after these steps you might load frozen_graph.pb contains both graph definition and weights using OpenCV.