caffe源码学习（六） 自定义层

经过前面对google protocol buffer、Blob、SyncedMemory 与 shared_ptr、layer、data layer的初步学习，已经能够仿照caffe中已有的层来写自定层了。所以接下来就在caffe深度学习框架下写自定义层。首先应该注意，不同版本的caffe可能会有一些区别，所以要根据官网指南来写自定义层。为了方便，把目前版本的指南（20160605）贴出来：

Developing new layers

1. Add a class declaration for your layer to include/caffe/layers/your_layer.hpp. 
(1) Include an inline implementation of type overriding the method virtual inline const char* type() const { return "YourLayerName"; } replacing YourLayerName with your layer’s name. 
(2) Implement the {*}Blobs() methods to specify blob number requirements; see /caffe/include/caffe/layers.hpp to enforce strict top and bottom Blob counts using the inline {*}Blobs() methods. 
(3) Omit the *_gpu declarations if you’ll only be implementing CPU code. 
2. Implement your layer in src/caffe/layers/your_layer.cpp. 
(1) (optional) LayerSetUp for one-time initialization: reading parameters, fixed-size allocations, etc. 
(2) Reshape for computing the sizes of top blobs, allocating buffers, and any other work that depends on the shapes of bottom blobs 
(3) Forward_cpu for the function your layer computes 
(4) Backward_cpu for its gradient (Optional – a layer can be forward-only) 
3. (Optional) Implement the GPU versions Forward_gpu and Backward_gpu in layers/your_layer.cu. 
4. If needed, declare parameters in proto/caffe.proto, using (and then incrementing) the “next available layer-specific ID” declared in a comment above message LayerParameter. 
5. Instantiate and register your layer in your cpp file with the macro provided in layer_factory.hpp. Assuming that you have a new layer MyAwesomeLayer, you can achieve it with the following command:

INSTANTIATE_CLASS(MyAwesomeLayer);REGISTER_LAYER_CLASS(MyAwesome);
1
2
1
2

6. Note that you should put the registration code in your own cpp file, so your implementation of a layer is self-contained. 
7. Optionally, you can also register a Creator if your layer has multiple engines. For an example on how to define a creator function and register it, see GetConvolutionLayer in caffe/layer_factory.cpp. 
8. Write tests in test/test_your_layer.cpp. Use test/test_gradient_check_util.hpp to check that your Forward and Backward implementations are in numerical agreement.

Forward-Only Layers 
If you want to write a layer that you will only ever include in a test net, you do not have to code the backward pass. For example, you might want a layer that measures performance metrics at test time that haven’t already been implemented. Doing this is very simple. You can write an inline implementation of Backward_cpu (or Backward_gpu) together with the definition of your layer in include/caffe/your_layer.hpp that looks like:

virtual void Backward_cpu(const vector<Blob<Dtype>*>& top, const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {  NOT_IMPLEMENTED;}
1
2
3
1
2
3

The NOT_IMPLEMENTED macro (defined in common.hpp) throws an error log saying “Not implemented yet”. For examples, look at the accuracy layer (accuracy_layer.hpp) and threshold layer (threshold_layer.hpp) definitions.

官网指南中已经说的比较详细，下面自定义层主要用到1,2,3,4,5,6中提到的，简单的实现了y = x^n点对点的简单计算，其中n从prototxt文件中读取。7和8目前还没有实现，后面有需要的话再更新。然后通过matlab接口进行验证该层的计算是否正确。

还是按照老习惯，先给出源码，再做总结。

1.源码

mysquare.hpp(该代码是仿照caffe中已经实现的absval_layer和dropout_layer来写的，所以保留原有的一些内容。最开始要是实现y = x^2，接下来为了实现能够从prototxt中读取数据，实现y = x^n，n从prototxt中读取，所以命名为mysquare。)

#ifndef MYSQUARE_LAYER_HPP#define MYSQUARE_LAYER_HPP#include <vector>#include "caffe/blob.hpp"#include "caffe/layer.hpp"#include "caffe/proto/caffe.pb.h"#include "caffe/layers/neuron_layer.hpp"namespace caffe {/** * @brief Computes @f$ y = x^2 @f$ * * @param bottom input Blob vector (length 1) *   -# @f$ (N \times C \times H \times W) @f$ *      the inputs @f$ x @f$ * @param top output Blob vector (length 1) *   -# @f$ (N \times C \times H \times W) @f$ *      the computed outputs @f$ y = x^2 @f$ */template <typename Dtype>class MySquareLayer : public NeuronLayer<Dtype> { public:  explicit MySquareLayer(const LayerParameter& param)      : NeuronLayer<Dtype>(param) {}  virtual void LayerSetUp(const vector<Blob<Dtype>*>& bottom,      const vector<Blob<Dtype>*>& top);  virtual inline const char* type() const { return "MySquare"; }  virtual inline int ExactNumBottomBlobs() const { return 1; }  virtual inline int ExactNumTopBlobs() const { return 1; } protected:  /// @copydoc MySquareLayer  virtual void Forward_cpu(const vector<Blob<Dtype>*>& bottom,      const vector<Blob<Dtype>*>& top);  virtual void Forward_gpu(const vector<Blob<Dtype>*>& bottom,      const vector<Blob<Dtype>*>& top);  /**   * @brief Computes the error gradient w.r.t. the absolute value inputs.   *   * @param top output Blob vector (length 1), providing the error gradient with   *      respect to the outputs   *   -# @f$ (N \times C \times H \times W) @f$   *      containing error gradients @f$ \frac{\partial E}{\partial y} @f$   *      with respect to computed outputs @f$ y @f$   * @param propagate_down see Layer::Backward.   * @param bottom input Blob vector (length 2)   *   -# @f$ (N \times C \times H \times W) @f$   *      the inputs @f$ x @f$; Backward fills their diff with   *      gradients @f$   *        \frac{\partial E}{\partial x} =   *            \mathrm{sign}(x) \frac{\partial E}{\partial y}   *      @f$ if propagate_down[0]   */ virtual void Backward_cpu(const vector<Blob<Dtype>*>& top,      const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom); virtual void Backward_gpu(const vector<Blob<Dtype>*>& top,      const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom); float power_;};}  // namespace caffe#endif // MYSQUARE_LAYER_HPP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

mysquare_layer.cpp

#include <vector>#include "caffe/layers/mysquare_layer.hpp"#include "caffe/util/math_functions.hpp"namespace caffe {template <typename Dtype>void MySquareLayer<Dtype>::LayerSetUp(const vector<Blob<Dtype>*>& bottom,      const vector<Blob<Dtype>*>& top) {  NeuronLayer<Dtype>::LayerSetUp(bottom, top);  CHECK_NE(top[0], bottom[0]) << this->type() << " Layer does not "    "allow in-place computation.";  power_ = this->layer_param_.mysquare_param().power();}template <typename Dtype>void MySquareLayer<Dtype>::Forward_cpu(    const vector<Blob<Dtype>*>& bottom, const vector<Blob<Dtype>*>& top) {  const int count = top[0]->count();  Dtype* top_data = top[0]->mutable_cpu_data();  caffe_powx(count, bottom[0]->cpu_data(), Dtype(power_), top_data);}template <typename Dtype>void MySquareLayer<Dtype>::Backward_cpu(const vector<Blob<Dtype>*>& top,    const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {  const int count = top[0]->count();  const Dtype* top_diff = top[0]->cpu_diff();  if (propagate_down[0]) {    const Dtype* bottom_data = bottom[0]->cpu_data();    Dtype* bottom_diff = bottom[0]->mutable_cpu_diff();    caffe_powx(count, bottom_data, Dtype(power_ - 1), bottom_diff);    caffe_scal(count, Dtype(power_), bottom_diff);    caffe_mul(count, bottom_diff, top_diff, bottom_diff);  }}#ifdef CPU_ONLYSTUB_GPU(MySquareLayer);#endifINSTANTIATE_CLASS(MySquareLayer);REGISTER_LAYER_CLASS(MySquare);}  // namespace caffe
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

mysquare_layer.cu

#include <vector>#include "caffe/layers/mysquare_layer.hpp"#include "caffe/util/math_functions.hpp"#include <iostream>using namespace std;namespace caffe {template <typename Dtype>void MySquareLayer<Dtype>::Forward_gpu(    const vector<Blob<Dtype>*>& bottom, const vector<Blob<Dtype>*>& top) {  const int count = top[0]->count();  Dtype* top_data = top[0]->mutable_gpu_data();  caffe_gpu_powx(count, bottom[0]->gpu_data(), Dtype(power_), top_data);}template <typename Dtype>void MySquareLayer<Dtype>::Backward_gpu(const vector<Blob<Dtype>*>& top,    const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {  const int count = top[0]->count();  const Dtype* top_diff = top[0]->gpu_diff();  if (propagate_down[0]) {    const Dtype* bottom_data = bottom[0]->gpu_data();    Dtype* bottom_diff = bottom[0]->mutable_gpu_diff();    const Dtype* bottom_data_w = bottom[0]->cpu_data();    const Dtype* bottom_diff_w = bottom[0]->cpu_diff();    cout << "bottom_data[0]: " << bottom_data_w[0] << endl;    cout << "bottom_diff[0]: " << bottom_diff_w[0] << endl;    caffe_gpu_powx(count, bottom_data, Dtype(power_ - 1), bottom_diff);    bottom_diff = bottom[0]->mutable_gpu_diff();    bottom_data_w = bottom[0]->cpu_data();    bottom_diff_w = bottom[0]->cpu_diff();    cout << "bottom_data[0]: " << bottom_data_w[0] << endl;    cout << "bottom_diff[0]: " << bottom_diff_w[0] << endl;    caffe_gpu_scal(count, Dtype(power_), bottom_diff);    bottom_diff = bottom[0]->mutable_gpu_diff();    bottom_data_w = bottom[0]->cpu_data();    bottom_diff_w = bottom[0]->cpu_diff();    cout << "bottom_data[0]: " << bottom_data_w[0] << endl;    cout << "bottom_diff[0]: " << bottom_diff_w[0] << endl;    caffe_gpu_mul(count, bottom_diff, top_diff, bottom_diff);    bottom_diff = bottom[0]->mutable_gpu_diff();    bottom_data_w = bottom[0]->cpu_data();    bottom_diff_w = bottom[0]->cpu_diff();    cout << "bottom_data[0]: " << bottom_data_w[0] << endl;    cout << "bottom_diff[0]: " << bottom_diff_w[0] << endl;  }}INSTANTIATE_LAYER_GPU_FUNCS(MySquareLayer);}  // namespace caffe
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

caffe.proto

message LayerParameter{...++ optional MySquareParameter mysquare_param = 150;...}...++ message MySquareParameter {++  optional float power = 1 [default = 2];++ }...message V1LayerParameter{...++ MYSQUARE = 40;...}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

deploy.prototxt

name: "CaffeNet"layer {  name: "data"  type: "Input"  top: "data"  input_param { shape: { dim: 10 dim: 3 dim: 227 dim: 227 } }}layer {  name: "mysquare"  type: "MySquare"  bottom: "data"  top: "mysquare_out"  mysquare_param {     power: 3  }}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

test.m

% 在caffe_root/matlab下im = imread('../examples/images/cat.jpg');input_data = {prepare_image(im)};input_data = {pp(im)};m = 'test/deploy.prototxt'caffe.set_mode_gpu();caffe.set_device(1);net = caffe.Net(m,'test')out = net.forward(input_data);ii = input_data{1,1};oo = out{1,1};% 简单观察一下计算是否正确ii(1:10,1:10,1,1)oo(1:10,1:10,1,1)(-60.4030)^3
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

pp.m

% 在caffe_root/matlab下% 下面是classification_demo.m中的一个函数。function crops_data = pp(im)% ------------------------------------------------------------------------% caffe/matlab/+caffe/imagenet/ilsvrc_2012_mean.mat contains mean_data that% is already in W x H x C with BGR channelsd = load('../+caffe/imagenet/ilsvrc_2012_mean.mat');mean_data = d.mean_data;IMAGE_DIM = 256;CROPPED_DIM = 227;% Convert an image returned by Matlab's imread to im_data in caffe's data% format: W x H x C with BGR channelsim_data = im(:, :, [3, 2, 1]);  % permute channels from RGB to BGRim_data = permute(im_data, [2, 1, 3]);  % flip width and heightim_data = single(im_data);  % convert from uint8 to singleim_data = imresize(im_data, [IMAGE_DIM IMAGE_DIM], 'bilinear');  % resize im_dataim_data = im_data - mean_data;  % subtract mean_data (already in W x H x C, BGR)% oversample (4 corners, center, and their x-axis flips)crops_data = zeros(CROPPED_DIM, CROPPED_DIM, 3, 10, 'single');indices = [0 IMAGE_DIM-CROPPED_DIM] + 1;n = 1;for i = indices  for j = indices    crops_data(:, :, :, n) = im_data(i:i+CROPPED_DIM-1, j:j+CROPPED_DIM-1, :);    crops_data(:, :, :, n+5) = crops_data(end:-1:1, :, :, n);    n = n + 1;  endendcenter = floor(indices(2) / 2) + 1;crops_data(:,:,:,5) = ...  im_data(center:center+CROPPED_DIM-1,center:center+CROPPED_DIM-1,:);crops_data(:,:,:,10) = crops_data(end:-1:1, :, :, 5);
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

2.总结

郁闷啊，马上总结完了，不小心弄没了，以后有时间再更新吧。。。