From 6984519cbb659aac0b0b323de93d5a90aa2049b7 Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Wed, 3 Jun 2015 10:29:38 +0800 Subject: ... --- doc/nerv_matrix.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index 00356c3..9f4df5a 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -68,7 +68,7 @@ It sets the content of __Matrix__ `self` to be `alpha * ma + beta * mb`.__Matrix * __void Matrix.mul(Matrix self, Matrix ma, Matrix mb, Element_type alpha, Element_type beta, [string ta, string tb])__ It sets the content of __Matrix__ `self` to be `beta * self + alpha * ma * mb`. `ta` and `tb` is optional, if `ta` is 'T', then `ma` will be transposed, also if `tb` is 'T', `mb` will be transposed. * __void Matrix.add_row(Matrix self, Matrix va, Element_type beta)__ -It sets the content of __Matrix__ `self`(which should be row vector) to be `self + beta * va`. +Add `beta * va` to every row of __Matrix__ `self`. * __void Matrix.fill(Matrix self, Element_type value)__ Fill the content of __Matrix__ `self` to be `value`. * __void Matrix.sigmoid(Matrix self, Matrix ma)__ -- cgit v1.2.3-70-g09d2 From cb17b5715683a56459ba8097446fa11b0fc325ca Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Wed, 3 Jun 2015 12:51:30 +0800 Subject: ... --- doc/nerv_matrix.md | 8 +++++++- speech | 2 +- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index 9f4df5a..ef5118e 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -23,7 +23,7 @@ Also note that all assigning operation in __Nerv__ is reference copy, you can us ###Class hierarchy### The class hierarchy of the matrix classes can be clearly observed in `matrix/init.c`. First there is a abstract base class __Nerv.Matrix__, which is inherited by __Nerv.CuMatrix__ and __Nerv.MMatrix__(also abstract). -Finally, there is __Nerv.CuMatrixFloat__, __Nerv.CuMatrixDouble__, inheriting __Nerv.CuMatrix__, and __Nerv.MMatrixFloat__, __Nerv.MMatrixDouble__, inheriting __Nerv.MMatrix__. +Finally, there is __Nerv.CuMatrixFloat__, __Nerv.CuMatrixDouble__, inheriting __Nerv.CuMatrix__, and __Nerv.MMatrixFloat__, __Nerv.MMatrixDouble__, __Nerv.MMatrixInt__ , inheriting __Nerv.MMatrix__. ##Methods## Mind that usually a matrix object can only do calculation with matrix of its own type(a __Nerv.CuMatrixFloat__ matrix can only do add operation with a __Nerv.CuMatrixFloat__). @@ -77,3 +77,9 @@ Set the element of __Matrix__ `self` to be elementwise-sigmoid of `ma`. Set the element of __Matrix__ `self`, to be `self[i][j]=err[i][j]*output[i][j]*(1-output[i][j])`. This function is used to propagate sigmoid layer error. * __void Matrix.softmax(Matrix self, Matrix a)__ Calculate a row-by-row softmax of __Matrix__ `a` and save the result in `self`. +* __void Matrix.mul_elem(Matrix self, Matrix ma, Matrix mb)__ +Calculate element-wise multiplication of __Matrix__ `ma` and `mb`, store the result in `self`. +* __void Matrix.log_elem(Matrix self, Matrix ma)__ +Calculate element-wise log of __Matrix__ `ma`, store the result in `self`. +* __void Matrix.copy_rows_fromh_by_idx(Matrix self, MMatrix ma, MMatrixInt idx)__ +`idx` should be a row vector. This function copy the rows of `ma` to `self` according to `idx`, in other words, it assigns `ma[idx[i]]` to `self[i]`. \ No newline at end of file diff --git a/speech b/speech index 0c6ca6a..d8ea67e 160000 --- a/speech +++ b/speech @@ -1 +1 @@ -Subproject commit 0c6ca6a17f06821cd5d612f489ca6cb68c2c4d5b +Subproject commit d8ea67ee420c2fc73085da04de86df023acd98d7 -- cgit v1.2.3-70-g09d2 From 5c018e6065d415183b23ed1d68bb4092e0bd89ea Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Wed, 3 Jun 2015 18:30:38 +0800 Subject: doc change for matrix --- doc/nerv_matrix.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index ef5118e..2edf691 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -82,4 +82,10 @@ Calculate element-wise multiplication of __Matrix__ `ma` and `mb`, store the res * __void Matrix.log_elem(Matrix self, Matrix ma)__ Calculate element-wise log of __Matrix__ `ma`, store the result in `self`. * __void Matrix.copy_rows_fromh_by_idx(Matrix self, MMatrix ma, MMatrixInt idx)__ -`idx` should be a row vector. This function copy the rows of `ma` to `self` according to `idx`, in other words, it assigns `ma[idx[i]]` to `self[i]`. \ No newline at end of file +`idx` should be a row vector. This function copy the rows of `ma` to `self` according to `idx`, in other words, it assigns `ma[idx[i]]` to `self[i]`. +* __void Matrix.expand_frm(Matrix self, Matrix a, int context)__ +Treating each row of `a` as speech feature, and do a feature expansion. The `self` should of size `(a.nrow, a.ncol * (context * 2 + 1))`. `self[i]` will be `(a[i-context] a[i-context+1] ... a[i] a[i+1] a[i+context])`. `a[0]` and `a[nrow]` will be copied to extend the index range. +* __void Matrix.rearrange_frm(Matrix self, Matrix a, int step)__ +Rearrange `a` according to its feature dimension. The `step` is the length of context. So, `self[i][j]` will be assigned `a[i][j / step + (j % step) * (a.ncol / step)]`. `a` and `self` should be of the same size and `step` should be divisible by `a.ncol`. +* __void Matrix.scale_row(Matrix self, Matrix scale)__ +Scale each column of `self` according to a vector `scale`. `scale` should be of size `1 * self.ncol`. \ No newline at end of file -- cgit v1.2.3-70-g09d2 From 98902f7fd28b53c34e71aad2a60c153c68b4d4b7 Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Wed, 3 Jun 2015 18:39:16 +0800 Subject: doc change for matrix --- doc/nerv_matrix.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index 2edf691..2cef099 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -88,4 +88,14 @@ Treating each row of `a` as speech feature, and do a feature expansion. The `sel * __void Matrix.rearrange_frm(Matrix self, Matrix a, int step)__ Rearrange `a` according to its feature dimension. The `step` is the length of context. So, `self[i][j]` will be assigned `a[i][j / step + (j % step) * (a.ncol / step)]`. `a` and `self` should be of the same size and `step` should be divisible by `a.ncol`. * __void Matrix.scale_row(Matrix self, Matrix scale)__ -Scale each column of `self` according to a vector `scale`. `scale` should be of size `1 * self.ncol`. \ No newline at end of file +Scale each column of `self` according to a vector `scale`. `scale` should be of size `1 * self.ncol`. +* __Matrix Matrix.\_\_add\_\_(Matrix ma, Matrix mb)__ +Returns a new __Matrix__ which stores the result of `ma+mb`. +* __Matrix Matrix.\_\_sub\_\_(Matrix ma, Matrix mb)__ +Returns a new __Matrix__ which stores the result of `ma-mb`. +* __Matrix Matrix.\_\_mul\_\_(Matrix ma, Matrix mb)__ +Returns a new __Matrix__ which stores the result of `ma*mb`. +* __CuMatrix CuMatrix.new_from_host(MMatrix m)__ +Return a new __CuMatrix__ which is a copy of `m`. +* __MMatrix CuMatrix.new_to_host(CuMatrix self)__ +Return a new __MMatrix__ which is a copy of `self`. \ No newline at end of file -- cgit v1.2.3-70-g09d2 From 88a2c29f347df2ef75b9891235bc176676e5dafd Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Thu, 4 Jun 2015 12:47:15 +0800 Subject: ... --- speech | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/speech b/speech index d8ea67e..a753eca 160000 --- a/speech +++ b/speech @@ -1 +1 @@ -Subproject commit d8ea67ee420c2fc73085da04de86df023acd98d7 +Subproject commit a753eca0121ac3ec81ed76bd719d3f1cb9522680 -- cgit v1.2.3-70-g09d2 From b53f40144715ef80d26e9a894ddc153e9ebfcc34 Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Thu, 4 Jun 2015 13:04:27 +0800 Subject: ... --- doc/nerv_matrix.md | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-) diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index 2cef099..1a2778b 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -98,4 +98,62 @@ Returns a new __Matrix__ which stores the result of `ma*mb`. * __CuMatrix CuMatrix.new_from_host(MMatrix m)__ Return a new __CuMatrix__ which is a copy of `m`. * __MMatrix CuMatrix.new_to_host(CuMatrix self)__ -Return a new __MMatrix__ which is a copy of `self`. \ No newline at end of file +Return a new __MMatrix__ which is a copy of `self`. +* __string Matrix.\_\_tostring\_\_(Matrix self)__ +Returns a string containing values of __Matrix__ `self`. +--- + +##Examples## +* Use `get_dataref_value` to test __Nerv__'s matrix space allocation. +``` +m = 10 +n = 10 +fm = nerv.MMatrixFloat(m, n) +dm = nerv.MMatrixDouble(m, n) +for i = 0, m - 1 do + for j = 0, n - 1 do + t = i / (j + 1) + fm[i][j] = t + dm[i][j] = t + end +end +print("test fm:get_dataref_value:", fm:get_dataref_value()) +print("forced a garbade collect") +collectgarbage("collect") +print("test fm:get_dataref_value:", fm:get_dataref_value()) +print(fm) +print(dm) +``` +* Test some __Matrix__ calculations. +``` +m = 4 +n = 4 +fm = nerv.CuMatrixFloat(m, n) +dm = nerv.CuMatrixDouble(m, n) +for i = 0, m - 1 do + for j = 0, n - 1 do + -- local t = math.random(10) + t = i / (j + 1) + fm[i][j] = t + dm[i][j] = t + end +end +print(fm) +fs = fm:create() +fs:softmax(fm) +-- print(fs) +print(dm) +ds = dm:create() +ds:softmax(dm) +-- print(ds) +print(fs) +print(fs + fs) +print(ds + ds) +print(fs - fs) +print(ds - ds) +a = fs:create() +a:mul_elem(fs, fs) +print(a) +a:log_elem(fs) +print(a) +``` \ No newline at end of file -- cgit v1.2.3-70-g09d2 From f9e116ef514b27680434eb92d737135edd316875 Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Thu, 4 Jun 2015 17:03:41 +0800 Subject: doc change --- README.md | 6 +++++- doc/nerv_io.md | 23 +++++++++++++++++++++++ doc/nerv_matrix.md | 6 ++++++ doc/nerv_param.md | 26 ++++++++++++++++++++++++++ 4 files changed, 60 insertions(+), 1 deletion(-) create mode 100644 doc/nerv_io.md create mode 100644 doc/nerv_param.md diff --git a/README.md b/README.md index d55672b..c37494c 100644 --- a/README.md +++ b/README.md @@ -28,8 +28,12 @@ Nerv uses [luaT]\(a [Torch] library\) to define lua class in C. Enables object-oriented programming in Nerv. * __[The Nerv utility functions](doc/nerv.md)__ Inlcudes some utility functions from luaT to implement __Nerv.Class__. +* __[The Nerv IO Package](doc/nerv_io.md)__ +The IO package is used to read and write parameters to file. * __[The Nerv Matrix Package](doc/nerv_matrix.md)__ -The Matrix package is a basic package in __Nerv__ that is used to store and manipulate matrices. +The matrix package is a basic package in __Nerv__ that is used to store and manipulate matrices. +* __[The Nerv Parameter Package](doc/nerv_param.md)__ +The parameter package is used to store, read model parameters from file. [luaT]:https://github.com/torch/torch7/tree/master/lib/luaT [Torch]:https://github.com/torch diff --git a/doc/nerv_io.md b/doc/nerv_io.md new file mode 100644 index 0000000..7066939 --- /dev/null +++ b/doc/nerv_io.md @@ -0,0 +1,23 @@ +#The Nerv IO Package# +Part of the [Nerv](../README.md) toolkit. + +##Description## +There are four classes in to deal with chunk data, which are __nerv.ChunkFile__, __nerv.ChunkFileHandle__, __nerv.ChunkInfo__, __nerv.ChunkData__. Below is the underlying C structs. +``` +typedef struct ChunkFileHandle { + FILE *fp; +} ChunkFileHandle; + +typedef struct ChunkInfo { + off_t offset, length; +} ChunkInfo; + +typedef struct ChunkData { + FILE *fp; + char *data; +} ChunkData; +``` + +##Methods## +* __ChunkFileHandle ChunkFile.\_\_init(string mode, string fn)__ +`mode` can be `r` or `w`, for reading or writing a file. diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index 1a2778b..22971d2 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -102,6 +102,12 @@ Return a new __MMatrix__ which is a copy of `self`. * __string Matrix.\_\_tostring\_\_(Matrix self)__ Returns a string containing values of __Matrix__ `self`. --- +* __MMatrix MMatrix.load(ChunkData chunk)__ +Return a new __MMatrix__ loaded from the file position in `chunk`. +* __void MMatrix.save(MMatrix self, ChunkFileHandle chunk)__ +Write `self` to the file position in `chunk`. +* __void MMatrix.copy_from(MMatrix ma, MMatrix mb,[int b_bgein, int b_end, int a_begin])__ +Copy a part of `mb`(rows of index `[b_begin..b_end)`) to `ma` beginning at row index `a_begin`. If not specified, `b_begin` will be `0`, `b_end` will be `b.nrow`, `a_begin` will be `0`. ##Examples## * Use `get_dataref_value` to test __Nerv__'s matrix space allocation. diff --git a/doc/nerv_param.md b/doc/nerv_param.md new file mode 100644 index 0000000..87ea1af --- /dev/null +++ b/doc/nerv_param.md @@ -0,0 +1,26 @@ +#The Nerv Parameter Package# +Part of the [Nerv](../README.md) toolkit. + +##Description## +###Class hierarchy### +There is a base class __Nerv.Param__ defined in `layer/init.lua`. +__Nerv.MatrixParam__ inherits __Nerv.Param__. +__Nerv.LinearTransParam__, __Nerv.BiasParam__ inherits __Nerv.MatrixParam__. +###Class member### +* __Nerv.MatrixParam__ + * __nerv.CuMatrix__ trans + Stores the parameter matrix. + +##Methods## +* __void Param.\_\_init(Param self, string id, table global_conf)__ +Constructor of a __Param__, it will set `self.id` to be `id` and `self.gconf` to be `global_conf`. +* __void Param.set_info(table info)__ +Set `self.info` to be `info`. +* __table Param.get_info()__ +Returns `self.info`. +* __void Param.read(ChunkData pcdata)__ +This is not implemented in `nerv.Param`. +* __void MatrixParam.read(MatrixParam self, ChunkData pcdata)__ +Read `self.trans` from `pcdata`. +* __void MatrixParam.write(MatrixParam self, ChunkFileHandle pfhandle)__ +Write `self.trans` to `pfhandle`. \ No newline at end of file -- cgit v1.2.3-70-g09d2 From 92cc3cffec87dbebd1e79b51ead2d04b52fbd0d4 Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Thu, 4 Jun 2015 22:28:45 +0800 Subject: ... --- doc/nerv_io.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/doc/nerv_io.md b/doc/nerv_io.md index 7066939..d452d5d 100644 --- a/doc/nerv_io.md +++ b/doc/nerv_io.md @@ -19,5 +19,8 @@ typedef struct ChunkData { ``` ##Methods## -* __ChunkFileHandle ChunkFile.\_\_init(string mode, string fn)__ +* __ChunkFile ChunkFile.\_\_init(string mode, string fn)__ `mode` can be `r` or `w`, for reading or writing a file. + +##Developer Notes## +In __Nerv.io__, a returned(by `ChunkFile.__init`) __nerv.ChunkFile__ will have a member `handle`, which is a __nerv.ChunkFileHandle__. \ No newline at end of file -- cgit v1.2.3-70-g09d2