diff options
| -rw-r--r-- | Makefile | 4 | ||||
| -rw-r--r-- | README.md | 6 | ||||
| -rw-r--r-- | doc/nerv_layer.md | 43 | ||||
| -rw-r--r-- | doc/nerv_param.md | 31 |
4 files changed, 65 insertions, 19 deletions
@@ -12,8 +12,8 @@ LUA_LIBS := matrix/init.lua io/init.lua nerv.lua \ nn/init.lua nn/layer_repo.lua nn/param_repo.lua nn/layer_dag.lua \ io/sgd_buffer.lua INCLUDE := -I build/luajit-2.0/include/luajit-2.0/ -DLUA_USE_APICHECK -#CUDA_BASE := /usr/local/cuda-6.5 -CUDA_BASE := /usr/local/cuda-5.0 +CUDA_BASE := /usr/local/cuda-6.5 +#CUDA_BASE := /usr/local/cuda-5.0 CUDA_INCLUDE := -I $(CUDA_BASE)/include/ INCLUDE += $(CUDA_INCLUDE) LDFLAGS := -L$(CUDA_BASE)/lib64/ -Wl,-rpath=$(CUDA_BASE)/lib64/ -lcudart -lcublas @@ -28,12 +28,14 @@ Nerv uses [luaT]\(a [Torch] library\) to define lua class in C. Enables object-oriented programming in Nerv. * __[The Nerv utility functions](doc/nerv.md)__ Inlcudes some utility functions from luaT to implement __Nerv.Class__. -* __[The Nerv IO Package](doc/nerv_io.md)__ -The IO package is used to read and write parameters to file. * __[The Nerv Matrix Package](doc/nerv_matrix.md)__ The matrix package is a basic package in __Nerv__ that is used to store and manipulate matrices. +* __[The Nerv IO Package](doc/nerv_io.md)__ +The IO package is used to read and write parameters to file. * __[The Nerv Parameter Package](doc/nerv_param.md)__ The parameter package is used to store, read model parameters from file. +* __[The Nerv Layer Package](doc/nerv_layer.md)__ +The layer package is used to define propagation and backpropagation of different type of layers. [luaT]:https://github.com/torch/torch7/tree/master/lib/luaT [Torch]:https://github.com/torch diff --git a/doc/nerv_layer.md b/doc/nerv_layer.md new file mode 100644 index 0000000..dd991df --- /dev/null +++ b/doc/nerv_layer.md @@ -0,0 +1,43 @@ +#The Nerv Layer Package# +Part of the [Nerv](../README.md) toolkit. + +##Description## +__nerv.Layer__ is the base class and most of its methods are abstract. +###Class hierarchy and their members### +* __nerv.Layer__. + * `table dim_in` It specifies the dimensions of the inputs. + * `table dim_out` It specifies the dimensions of the outputs. + * `string id` ID of this layer. + * `table gconf` Stores the `global_conf`. +* __nerv.AffineLayer__ inherits __nerv.Layer__, both `#dim_in` and `#dim_out` are 1. + * `MatrixParam ltp` The liner transform parameter. + * `BiasParam bp` The bias parameter. +* __nerv.BiasLayer__ inherits __nerv.Layer__, both `#dim_in` nad `#dim_out` are 1. + * `BiasParam bias` The bias parameter. +* __nerv.SigmoidLayer__ inherits __nerv.Layer__, both `#dim_in` and `#dim_out` are 1. +* __nerv.SoftmaxCELayer__ inherits __nerv.Layer__, `#dim_in` is 2 and `#dim_out` is 1. + * `float total_ce` + * `int total_frams` Records how many frames have passed. +##Methods## +* __void Layer.\_\_init(Layer self, string id, table global_conf, table layer_conf)__ +Abstract method. +The constructing method should assign `id` to `self.id` and `global_conf` to `self.gconf`, `layer_conf.dim_in` to `self.dim_in`, `layer_conf.dim_out` to `self.dim_out`. `dim_in` and `dim_out` are a list specifies the dimensions of the inputs and outputs. Also, `layer_conf` will include the parameters, which should also be properly saved. +* __void Layer.init(Layer self)__ +Abstract method. +Initialization method, in this method the layer should do some self-checking and allocate space for intermediate results. +* __void Layer.update(Layer self, table bp_err, table input, table output)__ +Abstract method. +`bp_err[i]` should be the error on `output[i]`. In this method the parameters of `self` is updated. +* __void Layer.propagate(Layer self, table input, table output)__ +Abstract method. +Given `input` and the current parameters, propagate and store the result in `output`. +* __void Layer.back_propagate(Layer self, Matrix next_bp_err, Matrix bp_err, Matrix input, Matrix output)__ +Abstract method. +Calculate the error on the inputs and store them in `next_bp_err`. + +* __void Layer.check_dim_len(int len_in, int len_out)__ +Check whether `#self.dim_in == len_in` and `#self.dim_out == len_out`, if violated, an error will be posted. +* __void Layer.get_params(Layer self)__ +Abstract method. +The layer should return a list containing its parameters. + diff --git a/doc/nerv_param.md b/doc/nerv_param.md index 87ea1af..167cb11 100644 --- a/doc/nerv_param.md +++ b/doc/nerv_param.md @@ -3,24 +3,25 @@ Part of the [Nerv](../README.md) toolkit. ##Description## ###Class hierarchy### -There is a base class __Nerv.Param__ defined in `layer/init.lua`. -__Nerv.MatrixParam__ inherits __Nerv.Param__. -__Nerv.LinearTransParam__, __Nerv.BiasParam__ inherits __Nerv.MatrixParam__. -###Class member### -* __Nerv.MatrixParam__ - * __nerv.CuMatrix__ trans - Stores the parameter matrix. +There is a base class __Nerv.Param__ defined in `layer/init.lua`. + +###Class hierarchy and their members### +* __nerv.MatrixParam__ inherits __nerv.Param__ + * `Matrix trans` stores the parameter matrix. +* __nerv.LinearTransParam__ inherits __Nerv.MatrixParam__. +* __Nerv.BiasParam__ inherits __Nerv.MatrixParam__. ##Methods## * __void Param.\_\_init(Param self, string id, table global_conf)__ Constructor of a __Param__, it will set `self.id` to be `id` and `self.gconf` to be `global_conf`. -* __void Param.set_info(table info)__ +* __void Param.set_info(Param self, table info)__ Set `self.info` to be `info`. -* __table Param.get_info()__ +* __table Param.get_info(Param self)__ Returns `self.info`. -* __void Param.read(ChunkData pcdata)__ -This is not implemented in `nerv.Param`. -* __void MatrixParam.read(MatrixParam self, ChunkData pcdata)__ -Read `self.trans` from `pcdata`. -* __void MatrixParam.write(MatrixParam self, ChunkFileHandle pfhandle)__ -Write `self.trans` to `pfhandle`.
\ No newline at end of file +* __void Param.read(Param self, ChunkData pcdata)__ +Abstract method. +In this method, `self` should in turn calls its members to load from `pcdata`. +* __void Param.write(Param self, ChunkFileHandle pfhandle)__ +Abstract method. +Save parameters to file. In this method, `self` should in turn calls its members to save to `pfhandle`. + |