Hot questions for Using Neural networks in protocol buffers

Question:

I can't find how to write comments in prototxt files.

Is there any way to have comments in a prototxt file, how?

Thanks


Answer:

You can comment by adding the # char: everything in the line after that is a comment:

layer {
  name: "aLayerWithComments" # I picked this cool name by myself
  type: "ReLU"
  bottom: "someData" # this is the output of the layer below
  top: "someData" # same name means this is an "in-place" layer
}
# and now you can comment the entire line...

Question:

I've got a huge data set in LMDB (40Gb) that I use for training a binary classifier with caffe.

Data layer in Caffe contains integer labels.

Are there any ready layers that could transform them into floats with adding some random jitter, so I could apply label smoothing technique, as described in 7.5.1 here

I have seen examples with HDF5, but they require regenerating data set, and I would like to avoid it.


Answer:

You can use DummyData layer to generate the random noise you wish to add to the labels. Once you have the noise, use Eltwise layer to sum them up:

layer {
  name: "noise"
  type: "DummyData"
  top: "noise"
  dummy_data_param {
    shape { dim: 10 dim: 1 dim: 1 dim: 1 } # assuming batch size = 10
    data_filler { type: "uniform" min: -0.1 max: 0.1 } # noise ~U(-0.1, 0.1) 
  }
}
layer {
  name: "label_noise"
  type: "Eltwise"
  bottom: "label"  # the input integer labels
  bottom: "noise"
  top: "label_noise"
  eltwise_param { operation: SUM }
}

Question:

I am trying to understand the caffe library. For that I run through step by step for feature_extraction.cpp and classification.cpp.

In those cpp files, I found out layers, prototxt file, caffemodel, net.cpp, caffe.pb.cc, caffe.pb.hfiles.

I know caffe is formed using different layers. So those layer files inside layer folder are used.

prototxt file is meant for the structure of a particular network such as googlenet, alexnet etc. Different net has different structure.

caffemodel is the trained model using caffe library for a specific type of net structure.

What do those net.cpp, caffe.pb.cc do? I mean how to understand their roles in forming this caffe deep learning network.


Answer:

You understand correctly that caffe implements deep learning by stacking "layers" one on top of the other to form a "net".

'net.cpp' Each layer works as a "functional block" and its behavior/implementation is defined in src/caffe/layers/<layer>.cpp, src/caffe/layers/<layer>.cu and include/caffe/layers/<layer>.hpp. The code that actually "stack" all the layers into a net can be found (mostly) in net.cpp.

'caffe.pb.h', 'caffe.pb.cc' In order to define the specific structure of a specific deep net architecture (e.g., AlexNet, GoogLeNet, ResNet etc.) caffe uses protocol-buffers library. The specific format of caffe protocol buffer is defined in src/caffe/proto/caffe.proto. The caffe.proto is "compiled" using google protobuffer compiler to produce 'caffe.pb.h' and 'caffe.pb.cc' c++ code for parsing and processing caffe prototxt and caffemodel files.

Question:

I'm working with some older branch of caffe. Now I need to modify the prototxt file by slicing the input layer.

I know that in the new syntax it looks like this:

layer {
  name: "slice"
  type: "Slice"
  bottom: "labelAndMask"
  ## Example of layer with a shape N x 5 x Height x Width
  top: "label"
  top: "mask"
  slice_param {
    axis: 1
    slice_point: 1
  }
}

What would be the equivalent in the old prototxt format? Also, where in the caffe sources could I look this up by myself?


Answer:

You should look at the bottom of $CAFFE_ROOT/src/caffe/proto/caffe.proto, you'll see the V1LayerParameter definition.

For old syntax slice layer:

layers {
  type: SLICE # this is NOT a string, but an enum
  name: "slice"
  bottom: "labelAndMask"
  ## Example of layer with a shape N x 5 x Height x Width
  top: "label"
  top: "mask"
  slice_param {
    axis: 1
    slice_point: 1
  }
}

Question:

when I run train_caffenet.sh, I get the following errors:

I0906 10:56:42.327703 21556 solver.cpp:91] Creating training net from net file: /home/pris/caffe-master/examples/myself/train_val.prototxt
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 26:12: Message type "caffe.ImageDataParameter" has no field named "backend".
F0906 10:56:42.327837 21556 upgrade_proto.cpp:79] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /home/pris/caffe-master/examples/myself/train_val.prototxt
*** Check failure stack trace: ***
    @     0x7f5013ca0daa  (unknown)
    @     0x7f5013ca0ce4  (unknown)
    @     0x7f5013ca06e6  (unknown)
    @     0x7f5013ca3687  (unknown)
    @     0x7f50142b019e  caffe::ReadNetParamsFromTextFileOrDie()
    @     0x7f501429e76b  caffe::Solver<>::InitTrainNet()
    @     0x7f501429f83c  caffe::Solver<>::Init()
    @     0x7f501429fb6a  caffe::Solver<>::Solver()
    @     0x7f50143de663  caffe::Creator_SGDSolver<>()
    @           0x40e9be  caffe::SolverRegistry<>::CreateSolver()
    @           0x407b62  train()
    @           0x4059ec  main
    @     0x7f5012faef45  (unknown)
    @           0x406121  (unknown)
    @              (nil)  (unknown)
Aborted (core dumped)

I've tried to solve it for a few days but still can't figure out how it comes wrong. here is my train_val.prototxt, mainly modified from the one in $CAFFE_TOOT/models/bvlc_reference_caffenet

name: "CaffeNet"
layer {
  name: "data"
  type: "ImageData"
  top: "data"
  top: "label"
  include {
    phase: TRAIN
  }
  transform_param {
    mirror: true
    crop_size: 227
    mean_file: "/home/pris/caffe-master/data/myself/myimagenet_mean.binaryproto"
  }
# mean pixel / channel-wise mean instead of mean image
#  transform_param {
#    crop_size: 227
#    mean_value: 104
#    mean_value: 117
#    mean_value: 123
#    mirror: true
#  }
  image_data_param {
    source: "/home/pris/caffe-master/examples/myself/imagenet_train_leveldb"
    batch_size: 256
    backend: LEVELDB
  }
}
layer 
{
  name: "data"
  type: "ImageData"
  top: "data"
  top: "label"
  include { phase: TEST }
  transform_param 
  {
    mirror: false
    crop_size: 227
    mean_file: "/home/pris/caffe-master/data/myself/myimagenet_mean.binaryproto"
  }
# mean pixel / channel-wise mean instead of mean image
#  transform_param {
#    crop_size: 227
#    mean_value: 104
#    mean_value: 117
#    mean_value: 123
#    mirror: false
#  }
  image_data_param 
  {
    source: "/home/pris/caffe-master/examples/myself/imagenet_val_leveldb"
    batch_size: 50
    backend: LEVELDB
  }
}
layer {
  name: "conv1"
  type: "Convolution"
  bottom: "data"
  top: "conv1"
  param 
  {
    lr_mult: 1
    decay_mult: 1
  }
  param 
  {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param 
  {
    num_output: 96
    kernel_size: 11
    stride: 4
    weight_filler 
    {
      type: "gaussian"
      std: 0.01
    }
    bias_filler 
    {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "relu1"
  type: "ReLU"
  bottom: "conv1"
  top: "conv1"
}
layer {
  name: "pool1"
  type: "Pooling"
  bottom: "conv1"
  top: "pool1"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "norm1"
  type: "LRN"
  bottom: "pool1"
  top: "norm1"
  lrn_param {
    local_size: 5
    alpha: 0.0001
    beta: 0.75
  }
}
layer {
  name: "conv2"
  type: "Convolution"
  bottom: "norm1"
  top: "conv2"
  param 
  {
    lr_mult: 1
    decay_mult: 1
  }
  param 
  {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param 
  {
    num_output: 256
    pad: 2
    kernel_size: 5
    group: 2
    weight_filler 
    {
      type: "gaussian"
      std: 0.01
    }
    bias_filler 
    {
      type: "constant"
      value: 1
    }
  }
}
layer {
  name: "relu2"
  type: "ReLU"
  bottom: "conv2"
  top: "conv2"
}
layer {
  name: "pool2"
  type: "Pooling"
  bottom: "conv2"
  top: "pool2"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "norm2"
  type: "LRN"
  bottom: "pool2"
  top: "norm2"
  lrn_param {
    local_size: 5
    alpha: 0.0001
    beta: 0.75
  }
}
layer {
  name: "conv3"
  type: "Convolution"
  bottom: "norm2"
  top: "conv3"
  param 
  {
    lr_mult: 1
    decay_mult: 1
  }
  param 
  {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param 
  {
    num_output: 384
    pad: 1
    kernel_size: 3
    weight_filler 
    {
      type: "gaussian"
      std: 0.01
    }
    bias_filler 
    {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "relu3"
  type: "ReLU"
  bottom: "conv3"
  top: "conv3"
}
layer {
  name: "conv4"
  type: "Convolution"
  bottom: "conv3"
  top: "conv4"
  param 
  {
    lr_mult: 1
    decay_mult: 1
  }
  param 
  {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param 
  {
    num_output: 384
    pad: 1
    kernel_size: 3
    group: 2
    weight_filler 
    {
      type: "gaussian"
      std: 0.01
    }
    bias_filler 
    {
      type: "constant"
      value: 1
    }
  }
}
layer {
  name: "relu4"
  type: "ReLU"
  bottom: "conv4"
  top: "conv4"
}
layer {
  name: "conv5"
  type: "Convolution"
  bottom: "conv4"
  top: "conv5"
  param 
  {
    lr_mult: 1
    decay_mult: 1
  }
  param 
  {
    lr_mult: 2
    decay_mult: 0
  }
  convolution_param {
    num_output: 256
    pad: 1
    kernel_size: 3
    group: 2
    weight_filler {
      type: "gaussian"
      std: 0.01
    }
    bias_filler {
      type: "constant"
      value: 1
    }
  }
}
layer {
  name: "relu5"
  type: "ReLU"
  bottom: "conv5"
  top: "conv5"
}
layer {
  name: "pool5"
  type: "Pooling"
  bottom: "conv5"
  top: "pool5"
  pooling_param {
    pool: MAX
    kernel_size: 3
    stride: 2
  }
}
layer {
  name: "fc6"
  type: "InnerProduct"
  bottom: "pool5"
  top: "fc6"
  param 
  {
    lr_mult: 1
    decay_mult: 1
  }
  param 
  {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 4096
    weight_filler {
      type: "gaussian"
      std: 0.005
    }
    bias_filler {
      type: "constant"
      value: 1
    }
  }
}
layer {
  name: "relu6"
  type: "ReLU"
  bottom: "fc6"
  top: "fc6"
}
layer {
  name: "drop6"
  type: "Dropout"
  bottom: "fc6"
  top: "fc6"
  dropout_param {
    dropout_ratio: 0.5
  }
}
layer {
  name: "fc7"
  type: "InnerProduct"
  bottom: "fc6"
  top: "fc7"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 4096
    weight_filler {
      type: "gaussian"
      std: 0.005
    }
    bias_filler {
      type: "constant"
      value: 1
    }
  }
}
layer {
  name: "relu7"
  type: "ReLU"
  bottom: "fc7"
  top: "fc7"
}
layer {
  name: "drop7"
  type: "Dropout"
  bottom: "fc7"
  top: "fc7"
  dropout_param {
    dropout_ratio: 0.5
  }
}
layer {
  name: "fc8"
  type: "InnerProduct"
  bottom: "fc7"
  top: "fc8"
  param {
    lr_mult: 1
    decay_mult: 1
  }
  param {
    lr_mult: 2
    decay_mult: 0
  }
  inner_product_param {
    num_output: 1000
    weight_filler {
      type: "gaussian"
      std: 0.01
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "accuracy"
  type: "Accuracy"
  bottom: "fc8"
  bottom: "label"
  top: "accuracy"
  include {
    phase: TEST
  }
}
layer {
  name: "loss"
  type: "SoftmaxWithLoss"
  bottom: "fc8"
  bottom: "label"
  top: "loss"
}

and the sovler.prototxt

net: "/home/pris/caffe-master/examples/myself/train_val.prototxt"
test_iter: 10
test_interval: 500
base_lr: 0.001
lr_policy: "step"
gamma: 0.1
stepsize: 100000
display: 20
max_iter: 450000
momentum: 0.9
weight_decay: 0.0005
snapshot: 2000
snapshot_prefix:"/home/pris/caffe-master/examples/myself/result"
solver_mode: GPU

train_caffenet.sh:

#!/usr/bin/env sh

/home/pris/caffe-master/build/tools/caffe train \
    --solver=/home/pris/caffe-master/examples/myself/solver.prototxt

I will really appreciate if someone could help me fixed it.


Answer:

You are reading training data from leveldb database, you should use input layer of type "Data" and not "ImageData".