Merge pull request #21 from cloudygoose/master

add doc for layers

4 files changed, 65 insertions, 19 deletions

diff --git a/Makefile b/Makefile
index 0468f57..cb694a2 100644
--- a/Makefile
+++ b/Makefile
@@ -12,8 +12,8 @@ LUA_LIBS := matrix/init.lua io/init.lua nerv.lua \
 			nn/init.lua nn/layer_repo.lua nn/param_repo.lua nn/layer_dag.lua \
 			io/sgd_buffer.lua
 INCLUDE := -I build/luajit-2.0/include/luajit-2.0/ -DLUA_USE_APICHECK
-#CUDA_BASE := /usr/local/cuda-6.5
-CUDA_BASE := /usr/local/cuda-5.0
+CUDA_BASE := /usr/local/cuda-6.5
+#CUDA_BASE := /usr/local/cuda-5.0
 CUDA_INCLUDE := -I $(CUDA_BASE)/include/
 INCLUDE += $(CUDA_INCLUDE)
 LDFLAGS := -L$(CUDA_BASE)/lib64/  -Wl,-rpath=$(CUDA_BASE)/lib64/ -lcudart -lcublas
diff --git a/README.md b/README.md
index c37494c..f825e57 100644
--- a/README.md
+++ b/README.md
@@ -28,12 +28,14 @@ Nerv uses [luaT]\(a [Torch] library\) to define lua class in C.
 Enables object-oriented programming in Nerv.
 * __[The Nerv utility functions](doc/nerv.md)__  
 Inlcudes some utility functions from luaT to implement __Nerv.Class__.
-* __[The Nerv IO Package](doc/nerv_io.md)__  
-The IO package is used to read and write parameters to file.
 * __[The Nerv Matrix Package](doc/nerv_matrix.md)__  
 The matrix package is a basic package in __Nerv__ that is used to store and manipulate matrices.
+* __[The Nerv IO Package](doc/nerv_io.md)__  
+The IO package is used to read and write parameters to file.
 * __[The Nerv Parameter Package](doc/nerv_param.md)__  
 The parameter package is used to store, read model parameters from file.
+* __[The Nerv Layer Package](doc/nerv_layer.md)__  
+The layer package is used to define propagation and backpropagation of different type of layers.
 
 [luaT]:https://github.com/torch/torch7/tree/master/lib/luaT
 [Torch]:https://github.com/torch
diff --git a/doc/nerv_layer.md b/doc/nerv_layer.md
new file mode 100644
index 0000000..dd991df
--- /dev/null
+++ b/doc/nerv_layer.md
@@ -0,0 +1,43 @@
+#The Nerv Layer Package#
+Part of the [Nerv](../README.md) toolkit.
+
+##Description##
+__nerv.Layer__ is the base class and most of its methods are abstract.  
+###Class hierarchy and their members###
+* __nerv.Layer__.  
+	* `table dim_in` It specifies the dimensions of the inputs.  
+	* `table dim_out` It specifies the dimensions of the outputs.  
+	* `string id` ID of this layer.
+	* `table gconf` Stores the `global_conf`.
+* __nerv.AffineLayer__ inherits __nerv.Layer__, both `#dim_in` and `#dim_out` are 1.
+	* `MatrixParam ltp` The liner transform parameter.
+	* `BiasParam bp` The bias parameter.
+* __nerv.BiasLayer__ inherits __nerv.Layer__, both `#dim_in` nad `#dim_out` are 1.
+	* `BiasParam bias` The bias parameter.
+* __nerv.SigmoidLayer__ inherits __nerv.Layer__, both `#dim_in` and `#dim_out` are 1.
+* __nerv.SoftmaxCELayer__ inherits __nerv.Layer__, `#dim_in` is 2 and `#dim_out` is 1.
+	* `float total_ce` 
+	* `int total_frams` Records how many frames have passed.
+##Methods##
+* __void Layer.\_\_init(Layer self, string id, table global_conf, table layer_conf)__  
+Abstract method.  
+The constructing method should assign `id` to `self.id` and `global_conf` to `self.gconf`, `layer_conf.dim_in` to `self.dim_in`, `layer_conf.dim_out` to `self.dim_out`. `dim_in` and `dim_out` are a list specifies the dimensions of the inputs and outputs. Also, `layer_conf` will include the parameters, which should also be properly saved.
+* __void Layer.init(Layer self)__  
+Abstract method.  
+Initialization method, in this method the layer should do some self-checking and allocate space for intermediate results.
+* __void Layer.update(Layer self, table bp_err, table input, table output)__  
+Abstract method.  
+`bp_err[i]` should be the error on `output[i]`. In this method the parameters of `self` is updated.
+* __void Layer.propagate(Layer self, table input, table output)__  
+Abstract method.  
+Given `input` and the current parameters, propagate and store the result in `output`.
+* __void Layer.back_propagate(Layer self, Matrix next_bp_err, Matrix bp_err, Matrix input, Matrix output)__  
+Abstract method.  
+Calculate the error on the inputs and store them in `next_bp_err`.
+
+* __void Layer.check_dim_len(int len_in, int len_out)__  
+Check whether `#self.dim_in == len_in` and `#self.dim_out == len_out`, if violated, an error will be posted.
+* __void Layer.get_params(Layer self)__  
+Abstract method.  
+The layer should return a list containing its parameters.
+
diff --git a/doc/nerv_param.md b/doc/nerv_param.md
index 87ea1af..167cb11 100644
--- a/doc/nerv_param.md
+++ b/doc/nerv_param.md
@@ -3,24 +3,25 @@ Part of the [Nerv](../README.md) toolkit.
 
 ##Description##
 ###Class hierarchy###
-There is a base class __Nerv.Param__ defined in `layer/init.lua`.  
-__Nerv.MatrixParam__ inherits __Nerv.Param__.  
-__Nerv.LinearTransParam__, __Nerv.BiasParam__ inherits __Nerv.MatrixParam__.  
-###Class member###
-* __Nerv.MatrixParam__
-	* __nerv.CuMatrix__ trans
-	Stores the parameter matrix.
+There is a base class __Nerv.Param__ defined in `layer/init.lua`.
+
+###Class hierarchy and their members###
+* __nerv.MatrixParam__ inherits __nerv.Param__  
+	* `Matrix trans` stores the parameter matrix.
+* __nerv.LinearTransParam__ inherits __Nerv.MatrixParam__.  
+* __Nerv.BiasParam__ inherits __Nerv.MatrixParam__.  
 
 ##Methods##
 * __void Param.\_\_init(Param self, string id, table global_conf)__  
 Constructor of a __Param__, it will set `self.id` to be `id` and `self.gconf` to be `global_conf`.
-* __void Param.set_info(table info)__  
+* __void Param.set_info(Param self, table info)__  
 Set `self.info` to be `info`.
-* __table Param.get_info()__  
+* __table Param.get_info(Param self)__  
 Returns `self.info`.
-* __void Param.read(ChunkData pcdata)__  
-This is not implemented in `nerv.Param`.
-* __void MatrixParam.read(MatrixParam self, ChunkData pcdata)__  
-Read `self.trans` from `pcdata`.
-* __void MatrixParam.write(MatrixParam self, ChunkFileHandle pfhandle)__  
-Write `self.trans` to `pfhandle`.
\ No newline at end of file
+* __void Param.read(Param self, ChunkData pcdata)__  
+Abstract method.  
+In this method, `self` should in turn calls its members to load from `pcdata`.
+* __void Param.write(Param self, ChunkFileHandle pfhandle)__  
+Abstract method.  
+Save parameters to file. In this method, `self` should in turn calls its members to save to `pfhandle`.
+