From 3d71571c45a37fb322d6acb9feecb5a185ce510a Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Mon, 1 Jun 2015 23:17:11 +0800 Subject: doc change --- doc/nerv_matrix.md | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index c710ac8..097a2ed 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -3,12 +3,12 @@ Part of the [Nerv](../README.md) toolkit. ##Description## ###Underlying structure### -In the begining is could be useful to know something about the underlying structure of a __Nerv__ matrix. +In the begining is could be useful to know something about the underlying structure of a __Nerv__ matrix. Please keep in mind that matrice in __Nerv__ is row-major. Every matrix object is a encapsulation of a C struct that describes the attributes of this matrix. ``` typedef struct Matrix { size_t stride; /* size of a row */ - long ncol, nrow, nmax; /* dimension of the matrix */ + long ncol, nrow, nmax; /* dimension of the matrix, nmax is simply nrow * ncol */ union { float *f; double *d; @@ -18,7 +18,7 @@ typedef struct Matrix { } Matrix; ``` It is worth mentioning that that `data_ref` is a counter which counts the number of references to its memory space, mind that it will also be increased when a row of the matrix is referenced(`col = m[2]`). A __Nerv__ matrix will deallocate its space when this counter is decreased to zero. -Also note that all assigning operation in __Nerv__ is reference copy, you can use `copy_tod` or `copy_toh` method to copy value. +Also note that all assigning operation in __Nerv__ is reference copy, you can use `copy_tod` or `copy_toh` method to copy value. Also, row assigning operations like `m1[2]=m2[3]` is forbidden in __Nerv__. ###Class hierarchy### The class hierarchy of the matrix classes can be clearly observed in `matrix/init.c`. @@ -27,6 +27,19 @@ Finally, there is __Nerv.CuMatrixFloat__, __Nerv.CuMatrixDouble__, inheriting __ ##Methods## Mind that usually a matrix object can only do calculation with matrix of its own type(a __Nerv.CuMatrixFloat__ matrix can only do add operation with a __Nerv.CuMatrixFloat__). -In the methods description below, __Matrix__ could be __Nerv.CuMatrixFloat__, __Nerv.CuMatrixDouble__, __Nerv.MMatrixFloat__ or __Nerv.MMatrixDouble__. +In the methods description below, __Matrix__ could be __Nerv.CuMatrixFloat__, __Nerv.CuMatrixDouble__, __Nerv.MMatrixFloat__ or __Nerv.MMatrixDouble__. __Element_type__ could be `float` or `double`, respectively. * __Matrix = Matrix(int nrow, int ncol)__ -Returns a __Matrix__ object of `nrow` rows and `ncol` columns. \ No newline at end of file +Returns a __Matrix__ object of `nrow` rows and `ncol` columns. +* __Element_type = Matrix.get_elem(Matrix self, int index)__ +Returns the element value at the specific index(treating the matrix as a vector). The index should be less than `nmax` of the matrix. +* __void Matrix.set_elem(Matrix self, int index, Element_type v)__ +Set the value at `index` to be `v`. +* __int Matrix.ncol(Matrix self)__ +Get `ncol`, the number of columns. +* __int Matrix.nrow(Matrix self)__ +Get `nrow`, the number of rows. +* __int Matrix.get_dataref_value(Matrix self)__ +Returns the value(not a pointer) of space the `data_ref` pointer pointed to. This function is mainly for debugging. +* __Matrix/Element\_type, boolean Matrix.\_\_index\_\_(Matrix self, int index)__ +If the matrix has more than one row, will return the row at `index` as a __Matrix__ . Otherwise it will return the value at `index`. +* __boolean Matrix.\_\_newindex\_\_(Matrix self, int )__ \ No newline at end of file -- cgit v1.2.3-70-g09d2 From 5e7fcdf4e5be450927764254d492d87349e4114e Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Tue, 2 Jun 2015 11:14:14 +0800 Subject: ... --- doc/nerv_matrix.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index 097a2ed..0585ee6 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -32,8 +32,8 @@ In the methods description below, __Matrix__ could be __Nerv.CuMatrixFloat__, __ Returns a __Matrix__ object of `nrow` rows and `ncol` columns. * __Element_type = Matrix.get_elem(Matrix self, int index)__ Returns the element value at the specific index(treating the matrix as a vector). The index should be less than `nmax` of the matrix. -* __void Matrix.set_elem(Matrix self, int index, Element_type v)__ -Set the value at `index` to be `v`. +* __void Matrix.set_elem(Matrix self, int index, Element_type value)__ +Set the value at `index` to be `value`. * __int Matrix.ncol(Matrix self)__ Get `ncol`, the number of columns. * __int Matrix.nrow(Matrix self)__ @@ -42,4 +42,6 @@ Get `nrow`, the number of rows. Returns the value(not a pointer) of space the `data_ref` pointer pointed to. This function is mainly for debugging. * __Matrix/Element\_type, boolean Matrix.\_\_index\_\_(Matrix self, int index)__ If the matrix has more than one row, will return the row at `index` as a __Matrix__ . Otherwise it will return the value at `index`. -* __boolean Matrix.\_\_newindex\_\_(Matrix self, int )__ \ No newline at end of file +* __void Matrix.\_\_newindex\_\_(Matrix self, int index, Element_type value)__ +Set the element at `index` to be `value`. +--- -- cgit v1.2.3-70-g09d2 From 6e58756ea75fb912c27436b5b3f556fb03e5e647 Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Tue, 2 Jun 2015 13:59:41 +0800 Subject: doc change about matrix --- doc/nerv_matrix.md | 28 +++++++++++++++++++++++++++- matrix/generic/cumatrix.c | 2 +- 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index 0585ee6..aedefbe 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -42,6 +42,32 @@ Get `nrow`, the number of rows. Returns the value(not a pointer) of space the `data_ref` pointer pointed to. This function is mainly for debugging. * __Matrix/Element\_type, boolean Matrix.\_\_index\_\_(Matrix self, int index)__ If the matrix has more than one row, will return the row at `index` as a __Matrix__ . Otherwise it will return the value at `index`. -* __void Matrix.\_\_newindex\_\_(Matrix self, int index, Element_type value)__ +* __void Matrix.\_\_newindex\_\_(Matrix self, int index, Element_type value)__ Set the element at `index` to be `value`. --- +* __Matrix Matrix.create(Matrix a)__ +Return a new __Matrix__ of `a`'s size(of the same number of rows and columns). +* __Matrix Matrix.colsum(Matrix self)__ +Return a new __Matrix__ of size (1,`self.ncol`), which stores the sum of all columns of __Matrix__ `self`. +* __Matrix Matrix.rowsum(Matrix self)__ +Return a new __Matrix__ of size (`self.nrow`,1), which stores the sum of all rows of __Matrix__ `self`. +* __Matrix Matrix.rowmax(Matrix self)__ +Return a new __Matrix__ of size (`self.nrow`,1), which stores the max value of all rows of __Matrix__ `self`. +* __Matrix Matrix.trans(Matrix self)__ +Return a new __Matrix__ of size (`self.ncol`,`self.nrow`), which stores the transpose of __Matrix__ `self`. +* __void Matrix.copy_fromh(Matrix self, MMatrix a)__ +Copy the content of a __MMatrix__ `a` to __Matrix__ `self`, they should be of the same size. +* __void Matrix.copy_fromd(Matrix self, CuMatrix a)__ +Copy the content of a __CuMatrix__ `a` to __Matrix__ `self`, they should be of the same size. +* __void Matrix.copy_toh(Matrix self, MMatrix a)__ +Copy the content of the __Matrix__ `self` to a __MMatrix__ `a`. +* __void Matrix.copy_tod(Matrix self, CuMatrix a)__ +Copy the content of the __Matrix__ `self` to a __CuMatrix__ `a`. +* __void Matrix.add(Matrix self, Matrix ma, Matrix mb, Element_type alpha, Element_type beta)__ +It sets the content of __Matrix__ `self` to be `alpha * ma + beta * mb`.__Matrix__ `ma,mb,self` should be of the same size. +* __void Matrix.mul(Matrix self, Matrix ma, Matrix mb, Element_type alpha, Element_type beta, [string ta, string tb])__ +It sets the content of __Matrix__ `self` to be `beta * self + alpha * ma * mb`. `ta` and `tb` is optional, if `ta` is 'T', then `ma` will be transposed, also if `tb` is 'T', `mb` will be transposed. +* __void Matrix.add_row(Matrix self, Matrix va, Element_type beta)__ +It sets the content of __Matrix__ `self`(which should be row vector) to be `self + beta * va`. +* __void Matrix.fill(Matrix self, Element_type value)__ +Fill the content of __Matrix__ to be `value`. \ No newline at end of file diff --git a/matrix/generic/cumatrix.c b/matrix/generic/cumatrix.c index 58f3679..0df1bd7 100644 --- a/matrix/generic/cumatrix.c +++ b/matrix/generic/cumatrix.c @@ -74,7 +74,7 @@ static int nerv_matrix_(mul)(lua_State *L) { if (an != bm) nerv_error(L, "Wrong dimension of multipliers"); /* MATRIX_ELEM alpha = 1.0f, beta = 0.0f; */ - CUBLAS_SAFE_CALL( + CUBLAS_SAFE_CALL( //Because matrix in Nerv is row-major, here b comes first NERV_CUBLAS_(gemm)(cublas_handle, tb, ta, bn, am, bm, &alpha, -- cgit v1.2.3-70-g09d2 From 788ddace103237d046bc1d87edd281366e2fdcca Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Tue, 2 Jun 2015 14:56:36 +0800 Subject: doc change about matrix --- doc/nerv_matrix.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index aedefbe..f1986a5 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -70,4 +70,9 @@ It sets the content of __Matrix__ `self` to be `beta * self + alpha * ma * mb`. * __void Matrix.add_row(Matrix self, Matrix va, Element_type beta)__ It sets the content of __Matrix__ `self`(which should be row vector) to be `self + beta * va`. * __void Matrix.fill(Matrix self, Element_type value)__ -Fill the content of __Matrix__ to be `value`. \ No newline at end of file +Fill the content of __Matrix__ `self` to be `value`. +* __void Matrix.sigmoid(Matrix self, Matrix ma)__ +Set the element of __Matrix__ `self` to be elementwise-sigmoid of `ma`. +* __void Matrix.sigmoid_grad(Matrix self, Matrix err, Matrix output)__ +Set the element of __Matrix__ `self`, to be `self[i][j]=err[i][j]*output[i][j]*(1-output[i][j])`. This function is used to propagate sigmoid layer error. + -- cgit v1.2.3-70-g09d2 From 64ba1ff100f8f417f92b47a57cbffca4343b4150 Mon Sep 17 00:00:00 2001 From: cloudygoose Date: Tue, 2 Jun 2015 21:21:16 +0800 Subject: doc change --- doc/nerv_matrix.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/doc/nerv_matrix.md b/doc/nerv_matrix.md index f1986a5..00356c3 100644 --- a/doc/nerv_matrix.md +++ b/doc/nerv_matrix.md @@ -17,17 +17,17 @@ typedef struct Matrix { long *data_ref; } Matrix; ``` -It is worth mentioning that that `data_ref` is a counter which counts the number of references to its memory space, mind that it will also be increased when a row of the matrix is referenced(`col = m[2]`). A __Nerv__ matrix will deallocate its space when this counter is decreased to zero. +It is worth mentioning that that `data_ref` is a counter which counts the number of references to its memory space, mind that it will also be increased when a row of the matrix is referenced(`col = m[2]`). A __Nerv__ matrix will deallocate its space when this counter is decreased to zero. Also note that all assigning operation in __Nerv__ is reference copy, you can use `copy_tod` or `copy_toh` method to copy value. Also, row assigning operations like `m1[2]=m2[3]` is forbidden in __Nerv__. ###Class hierarchy### -The class hierarchy of the matrix classes can be clearly observed in `matrix/init.c`. +The class hierarchy of the matrix classes can be clearly observed in `matrix/init.c`. First there is a abstract base class __Nerv.Matrix__, which is inherited by __Nerv.CuMatrix__ and __Nerv.MMatrix__(also abstract). Finally, there is __Nerv.CuMatrixFloat__, __Nerv.CuMatrixDouble__, inheriting __Nerv.CuMatrix__, and __Nerv.MMatrixFloat__, __Nerv.MMatrixDouble__, inheriting __Nerv.MMatrix__. ##Methods## Mind that usually a matrix object can only do calculation with matrix of its own type(a __Nerv.CuMatrixFloat__ matrix can only do add operation with a __Nerv.CuMatrixFloat__). -In the methods description below, __Matrix__ could be __Nerv.CuMatrixFloat__, __Nerv.CuMatrixDouble__, __Nerv.MMatrixFloat__ or __Nerv.MMatrixDouble__. __Element_type__ could be `float` or `double`, respectively. +In the methods description below, __Matrix__ could be __Nerv.CuMatrixFloat__, __Nerv.CuMatrixDouble__, __Nerv.MMatrixFloat__ or __Nerv.MMatrixDouble__. __Element_type__ could be `float` or `double`, respectively. * __Matrix = Matrix(int nrow, int ncol)__ Returns a __Matrix__ object of `nrow` rows and `ncol` columns. * __Element_type = Matrix.get_elem(Matrix self, int index)__ @@ -75,4 +75,5 @@ Fill the content of __Matrix__ `self` to be `value`. Set the element of __Matrix__ `self` to be elementwise-sigmoid of `ma`. * __void Matrix.sigmoid_grad(Matrix self, Matrix err, Matrix output)__ Set the element of __Matrix__ `self`, to be `self[i][j]=err[i][j]*output[i][j]*(1-output[i][j])`. This function is used to propagate sigmoid layer error. - +* __void Matrix.softmax(Matrix self, Matrix a)__ +Calculate a row-by-row softmax of __Matrix__ `a` and save the result in `self`. -- cgit v1.2.3-70-g09d2