get good PPL for h300, see m-tests/LOG-tnn-h300

2 files changed, 3599 insertions, 4 deletions

diff --git a/nerv/examples/lmptb/m-tests/LOG-tnn-h300 b/nerv/examples/lmptb/m-tests/LOG-tnn-h300
new file mode 100644
index 0000000..4fcffed
--- /dev/null
+++ b/nerv/examples/lmptb/m-tests/LOG-tnn-h300
@@ -0,0 +1,3595 @@
+Greetings
+[SCHEDULER]: creating work_dir...
+[LOG]LMVocab: Vocab building on file /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.train.txt.adds...
+[LOG]LMVocab: Building finished, vocab size now is 10000.
+[SCHEDULER]: preparing parameters...
+reading chunk 0 from 0
+metadata: return {type="nerv.LinearTransParam",id="ltp_ih"}
+
+reading chunk 1 from 34509841
+metadata: return {type="nerv.LinearTransParam",id="ltp_hh"}
+
+reading chunk 2 from 35545246
+metadata: return {type="nerv.LinearTransParam",id="ltp_ho"}
+
+reading chunk 3 from 70044639
+metadata: return {type="nerv.BiasParam",id="bp_h"}
+
+reading chunk 4 from 70048135
+metadata: return {type="nerv.BiasParam",id="bp_o"}
+
+reading chunk 5 from 70163127
+[SCHEDULER]: preparing parameters end.
+===INITIAL VALIDATION===
+[SCHEDULER]: preparing parameters...
+reading chunk 0 from 0
+metadata: return {type="nerv.LinearTransParam",id="ltp_ih"}
+
+reading chunk 1 from 34509841
+metadata: return {type="nerv.LinearTransParam",id="ltp_hh"}
+
+reading chunk 2 from 35545246
+metadata: return {type="nerv.LinearTransParam",id="ltp_ho"}
+
+reading chunk 3 from 70044639
+metadata: return {type="nerv.BiasParam",id="bp_h"}
+
+reading chunk 4 from 70048135
+metadata: return {type="nerv.BiasParam",id="bp_o"}
+
+reading chunk 5 from 70163127
+[SCHEDULER]: preparing parameters end.
+[SCHEDULER]: preparing layers...
+(23:21:04 2015-11-11)[nerv] info: create layer: recurrentL1
+(23:21:04 2015-11-11)[nerv] info: create layer: sigmoidL1
+(23:21:04 2015-11-11)[nerv] info: create layer: combinerL1
+(23:21:04 2015-11-11)[nerv] info: create layer: outputL
+(23:21:04 2015-11-11)[nerv] info: create layer: softmaxL
+(23:21:04 2015-11-11)[nerv] info: create layer: selectL1
+[SCHEDULER]: preparing layers end.
+[SCHEDULER]: Generate and initing TNN ...
+<input>	selectL1	0
+selectL1	recurrentL1	0
+recurrentL1	sigmoidL1	0
+sigmoidL1	combinerL1	0
+combinerL1	recurrentL1	1
+combinerL1	outputL	0
+outputL	softmaxL	0
+<input>	softmaxL	0
+softmaxL	<output>	0
+recurrentL1	#dim_in:	2	#dim_out:	1	#i_conns_p:	2	#o_conns_p	1
+outputL	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+combinerL1	#dim_in:	1	#dim_out:	2	#i_conns_p:	1	#o_conns_p	2
+sigmoidL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+selectL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	0	#o_conns_p	1
+softmaxL	#dim_in:	2	#dim_out:	1	#i_conns_p:	1	#o_conns_p	0
+TNN initing storage	selectL1	->	recurrentL1
+TNN initing storage	recurrentL1	->	sigmoidL1
+TNN initing storage	sigmoidL1	->	combinerL1
+TNN initing storage	combinerL1	->	recurrentL1
+TNN initing storage	combinerL1	->	outputL
+TNN initing storage	outputL	->	softmaxL
+[SCHEDULER]: Initing TNN end.
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds...
+batch_size:	10	chunk_size	15
+[SCHEDULER]: 40095 words processed Wed Nov 11 23:21:10 2015.
+	[SCHEDULER]: log prob per sample :-4.052709.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	0
+gconf.fz2	3370
+[SCHEDULER]: Displaying result:
+[SCHEDULER]: LMResult status of rnn: <SEN_CN 3370> <W_CN 70390> <PPL_NET 11363.085973621> <PPL_OOV 11232.52378889> <LOGP -298763.20911767>
+[SCHEDULER]: Doing on /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds end.
+
+[SCHEDULER]: preparing parameters...
+reading chunk 0 from 0
+metadata: return {type="nerv.LinearTransParam",id="ltp_ih"}
+
+reading chunk 1 from 34509841
+metadata: return {type="nerv.LinearTransParam",id="ltp_hh"}
+
+reading chunk 2 from 35545246
+metadata: return {type="nerv.LinearTransParam",id="ltp_ho"}
+
+reading chunk 3 from 70044639
+metadata: return {type="nerv.BiasParam",id="bp_h"}
+
+reading chunk 4 from 70048135
+metadata: return {type="nerv.BiasParam",id="bp_o"}
+
+reading chunk 5 from 70163127
+[SCHEDULER]: preparing parameters end.
+[SCHEDULER]: preparing layers...
+(23:21:20 2015-11-11)[nerv] info: create layer: recurrentL1
+(23:21:20 2015-11-11)[nerv] info: create layer: sigmoidL1
+(23:21:20 2015-11-11)[nerv] info: create layer: combinerL1
+(23:21:20 2015-11-11)[nerv] info: create layer: outputL
+(23:21:20 2015-11-11)[nerv] info: create layer: softmaxL
+(23:21:20 2015-11-11)[nerv] info: create layer: selectL1
+[SCHEDULER]: preparing layers end.
+[SCHEDULER]: Generate and initing TNN ...
+<input>	selectL1	0
+selectL1	recurrentL1	0
+recurrentL1	sigmoidL1	0
+sigmoidL1	combinerL1	0
+combinerL1	recurrentL1	1
+combinerL1	outputL	0
+outputL	softmaxL	0
+<input>	softmaxL	0
+softmaxL	<output>	0
+recurrentL1	#dim_in:	2	#dim_out:	1	#i_conns_p:	2	#o_conns_p	1
+outputL	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+combinerL1	#dim_in:	1	#dim_out:	2	#i_conns_p:	1	#o_conns_p	2
+sigmoidL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+selectL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	0	#o_conns_p	1
+softmaxL	#dim_in:	2	#dim_out:	1	#i_conns_p:	1	#o_conns_p	0
+TNN initing storage	selectL1	->	recurrentL1
+TNN initing storage	recurrentL1	->	sigmoidL1
+TNN initing storage	sigmoidL1	->	combinerL1
+TNN initing storage	combinerL1	->	recurrentL1
+TNN initing storage	combinerL1	->	outputL
+TNN initing storage	outputL	->	softmaxL
+[SCHEDULER]: Initing TNN end.
+===ITERATION 1 LR 1.000000===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER1 LR1]: 40099 words processed Wed Nov 11 23:21:34 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-3.248245.
+[SCHEDULER ITER1 LR1]: 80063 words processed Wed Nov 11 23:21:49 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-3.066372.
+[SCHEDULER ITER1 LR1]: 120068 words processed Wed Nov 11 23:22:04 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.998897.
+[SCHEDULER ITER1 LR1]: 160017 words processed Wed Nov 11 23:22:19 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.933869.
+[SCHEDULER ITER1 LR1]: 200138 words processed Wed Nov 11 23:22:34 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.895356.
+[SCHEDULER ITER1 LR1]: 240007 words processed Wed Nov 11 23:22:49 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.860453.
+[SCHEDULER ITER1 LR1]: 280135 words processed Wed Nov 11 23:23:04 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.830565.
+[SCHEDULER ITER1 LR1]: 320080 words processed Wed Nov 11 23:23:19 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.806896.
+[SCHEDULER ITER1 LR1]: 360059 words processed Wed Nov 11 23:23:34 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.781789.
+[SCHEDULER ITER1 LR1]: 400021 words processed Wed Nov 11 23:23:49 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.765120.
+[SCHEDULER ITER1 LR1]: 440102 words processed Wed Nov 11 23:24:04 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.746893.
+[SCHEDULER ITER1 LR1]: 480051 words processed Wed Nov 11 23:24:19 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.729376.
+[SCHEDULER ITER1 LR1]: 520093 words processed Wed Nov 11 23:24:34 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.714592.
+[SCHEDULER ITER1 LR1]: 560039 words processed Wed Nov 11 23:24:49 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.697929.
+[SCHEDULER ITER1 LR1]: 600112 words processed Wed Nov 11 23:25:04 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.688283.
+[SCHEDULER ITER1 LR1]: 640076 words processed Wed Nov 11 23:25:19 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.677096.
+[SCHEDULER ITER1 LR1]: 680026 words processed Wed Nov 11 23:25:34 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.668041.
+[SCHEDULER ITER1 LR1]: 720133 words processed Wed Nov 11 23:25:49 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.659308.
+[SCHEDULER ITER1 LR1]: 760048 words processed Wed Nov 11 23:26:04 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.648075.
+[SCHEDULER ITER1 LR1]: 800117 words processed Wed Nov 11 23:26:19 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.640555.
+[SCHEDULER ITER1 LR1]: 840116 words processed Wed Nov 11 23:26:34 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.630580.
+[SCHEDULER ITER1 LR1]: 880037 words processed Wed Nov 11 23:26:49 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.623065.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	42068
+gconf.fz2	42068
+[SCHEDULER ITER1 LR1]: Displaying result:
+[SCHEDULER ITER1 LR1]: LMResult status of rnn: <SEN_CN 42068> <W_CN 887521> <PPL_NET 479.30006237909> <PPL_OOV 418.29117140708> <LOGP -2436897.7616863>
+[SCHEDULER ITER1 LR1]: Doing on /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf end.
+===VALIDATION 1===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER1 LR1]: 40095 words processed Wed Nov 11 23:27:01 2015.
+	[SCHEDULER ITER1 LR1]: log prob per sample :-2.523185.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	0
+gconf.fz2	3370
+[SCHEDULER ITER1 LR1]: Displaying result:
+[SCHEDULER ITER1 LR1]: LMResult status of rnn: <SEN_CN 3370> <W_CN 70390> <PPL_NET 348.88659837747> <PPL_OOV 313.72060811114> <LOGP -184145.01523291>
+[SCHEDULER ITER1 LR1]: Doing on /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds end.
+[SCHEDULER ITER1 LR1]: saving net to file /home/slhome/txh18/workspace/nerv/play/dagL_test/params...
+
+[SCHEDULER ITER1 LR1]: preparing parameters...
+reading chunk 0 from 0
+metadata: return {type="nerv.LinearTransParam",id="ltp_hh"}
+
+reading chunk 1 from 1076965
+metadata: return {type="nerv.BiasParam",id="bp_h"}
+
+reading chunk 2 from 1080621
+metadata: return {type="nerv.LinearTransParam",id="ltp_ho"}
+
+reading chunk 3 from 35617574
+metadata: return {type="nerv.BiasParam",id="bp_o"}
+
+reading chunk 4 from 35733895
+metadata: return {type="nerv.LinearTransParam",id="ltp_ih"}
+
+reading chunk 5 from 70251684
+[SCHEDULER ITER1 LR1]: preparing parameters end.
+[SCHEDULER ITER1 LR1]: preparing layers...
+(23:27:11 2015-11-11)[nerv] info: create layer: recurrentL1
+(23:27:11 2015-11-11)[nerv] info: create layer: sigmoidL1
+(23:27:11 2015-11-11)[nerv] info: create layer: combinerL1
+(23:27:11 2015-11-11)[nerv] info: create layer: outputL
+(23:27:11 2015-11-11)[nerv] info: create layer: softmaxL
+(23:27:11 2015-11-11)[nerv] info: create layer: selectL1
+[SCHEDULER ITER1 LR1]: preparing layers end.
+[SCHEDULER ITER1 LR1]: Generate and initing TNN ...
+<input>	selectL1	0
+selectL1	recurrentL1	0
+recurrentL1	sigmoidL1	0
+sigmoidL1	combinerL1	0
+combinerL1	recurrentL1	1
+combinerL1	outputL	0
+outputL	softmaxL	0
+<input>	softmaxL	0
+softmaxL	<output>	0
+recurrentL1	#dim_in:	2	#dim_out:	1	#i_conns_p:	2	#o_conns_p	1
+outputL	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+combinerL1	#dim_in:	1	#dim_out:	2	#i_conns_p:	1	#o_conns_p	2
+sigmoidL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+selectL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	0	#o_conns_p	1
+softmaxL	#dim_in:	2	#dim_out:	1	#i_conns_p:	1	#o_conns_p	0
+TNN initing storage	selectL1	->	recurrentL1
+TNN initing storage	recurrentL1	->	sigmoidL1
+TNN initing storage	sigmoidL1	->	combinerL1
+TNN initing storage	combinerL1	->	recurrentL1
+TNN initing storage	combinerL1	->	outputL
+TNN initing storage	outputL	->	softmaxL
+[SCHEDULER ITER1 LR1]: Initing TNN end.
+===ITERATION 2 LR 1.000000===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER2 LR1]: 40092 words processed Wed Nov 11 23:27:25 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.599442.
+[SCHEDULER ITER2 LR1]: 80099 words processed Wed Nov 11 23:27:40 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.548803.
+[SCHEDULER ITER2 LR1]: 120004 words processed Wed Nov 11 23:27:55 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.523943.
+[SCHEDULER ITER2 LR1]: 160114 words processed Wed Nov 11 23:28:10 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.512345.
+[SCHEDULER ITER2 LR1]: 200066 words processed Wed Nov 11 23:28:25 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.505397.
+[SCHEDULER ITER2 LR1]: 240045 words processed Wed Nov 11 23:28:40 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.499117.
+[SCHEDULER ITER2 LR1]: 280057 words processed Wed Nov 11 23:28:55 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.492474.
+[SCHEDULER ITER2 LR1]: 320106 words processed Wed Nov 11 23:29:10 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.484600.
+[SCHEDULER ITER2 LR1]: 360024 words processed Wed Nov 11 23:29:25 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.479672.
+[SCHEDULER ITER2 LR1]: 400089 words processed Wed Nov 11 23:29:40 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.475321.
+[SCHEDULER ITER2 LR1]: 440067 words processed Wed Nov 11 23:29:55 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.471777.
+[SCHEDULER ITER2 LR1]: 480051 words processed Wed Nov 11 23:30:10 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.468552.
+[SCHEDULER ITER2 LR1]: 520140 words processed Wed Nov 11 23:30:25 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.465361.
+[SCHEDULER ITER2 LR1]: 560132 words processed Wed Nov 11 23:30:40 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.462341.
+[SCHEDULER ITER2 LR1]: 600118 words processed Wed Nov 11 23:30:55 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.460604.
+[SCHEDULER ITER2 LR1]: 640090 words processed Wed Nov 11 23:31:10 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.458146.
+[SCHEDULER ITER2 LR1]: 680075 words processed Wed Nov 11 23:31:25 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.458095.
+[SCHEDULER ITER2 LR1]: 720043 words processed Wed Nov 11 23:31:40 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.457528.
+[SCHEDULER ITER2 LR1]: 760012 words processed Wed Nov 11 23:31:55 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.455970.
+[SCHEDULER ITER2 LR1]: 800113 words processed Wed Nov 11 23:32:10 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.454040.
+[SCHEDULER ITER2 LR1]: 840089 words processed Wed Nov 11 23:32:25 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.451647.
+[SCHEDULER ITER2 LR1]: 880052 words processed Wed Nov 11 23:32:40 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.449628.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	42068
+gconf.fz2	42068
+[SCHEDULER ITER2 LR1]: Displaying result:
+[SCHEDULER ITER2 LR1]: LMResult status of rnn: <SEN_CN 42068> <W_CN 887521> <PPL_NET 322.88493854907> <PPL_OOV 281.43208335099> <LOGP -2276910.7606898>
+[SCHEDULER ITER2 LR1]: Doing on /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf end.
+===VALIDATION 2===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER2 LR1]: 40095 words processed Wed Nov 11 23:32:50 2015.
+	[SCHEDULER ITER2 LR1]: log prob per sample :-2.408668.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	0
+gconf.fz2	3370
+[SCHEDULER ITER2 LR1]: Displaying result:
+[SCHEDULER ITER2 LR1]: LMResult status of rnn: <SEN_CN 3370> <W_CN 70390> <PPL_NET 274.09438409291> <PPL_OOV 242.65292128392> <LOGP -175916.53225967>
+[SCHEDULER ITER2 LR1]: Doing on /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds end.
+[SCHEDULER ITER2 LR1]: saving net to file /home/slhome/txh18/workspace/nerv/play/dagL_test/params...
+
+[SCHEDULER ITER2 LR1]: preparing parameters...
+reading chunk 0 from 0
+metadata: return {type="nerv.LinearTransParam",id="ltp_ih"}
+
+reading chunk 1 from 34524481
+metadata: return {type="nerv.BiasParam",id="bp_h"}
+
+reading chunk 2 from 34528138
+metadata: return {type="nerv.LinearTransParam",id="ltp_ho"}
+
+reading chunk 3 from 69086256
+metadata: return {type="nerv.BiasParam",id="bp_o"}
+
+reading chunk 4 from 69202439
+metadata: return {type="nerv.LinearTransParam",id="ltp_hh"}
+
+reading chunk 5 from 70278561
+[SCHEDULER ITER2 LR1]: preparing parameters end.
+[SCHEDULER ITER2 LR1]: preparing layers...
+(23:33:00 2015-11-11)[nerv] info: create layer: recurrentL1
+(23:33:00 2015-11-11)[nerv] info: create layer: sigmoidL1
+(23:33:00 2015-11-11)[nerv] info: create layer: combinerL1
+(23:33:00 2015-11-11)[nerv] info: create layer: outputL
+(23:33:00 2015-11-11)[nerv] info: create layer: softmaxL
+(23:33:00 2015-11-11)[nerv] info: create layer: selectL1
+[SCHEDULER ITER2 LR1]: preparing layers end.
+[SCHEDULER ITER2 LR1]: Generate and initing TNN ...
+<input>	selectL1	0
+selectL1	recurrentL1	0
+recurrentL1	sigmoidL1	0
+sigmoidL1	combinerL1	0
+combinerL1	recurrentL1	1
+combinerL1	outputL	0
+outputL	softmaxL	0
+<input>	softmaxL	0
+softmaxL	<output>	0
+recurrentL1	#dim_in:	2	#dim_out:	1	#i_conns_p:	2	#o_conns_p	1
+outputL	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+combinerL1	#dim_in:	1	#dim_out:	2	#i_conns_p:	1	#o_conns_p	2
+sigmoidL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+selectL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	0	#o_conns_p	1
+softmaxL	#dim_in:	2	#dim_out:	1	#i_conns_p:	1	#o_conns_p	0
+TNN initing storage	selectL1	->	recurrentL1
+TNN initing storage	recurrentL1	->	sigmoidL1
+TNN initing storage	sigmoidL1	->	combinerL1
+TNN initing storage	combinerL1	->	recurrentL1
+TNN initing storage	combinerL1	->	outputL
+TNN initing storage	outputL	->	softmaxL
+[SCHEDULER ITER2 LR1]: Initing TNN end.
+===ITERATION 3 LR 1.000000===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER3 LR1]: 40092 words processed Wed Nov 11 23:33:14 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.569663.
+[SCHEDULER ITER3 LR1]: 80099 words processed Wed Nov 11 23:33:29 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.489703.
+[SCHEDULER ITER3 LR1]: 120004 words processed Wed Nov 11 23:33:44 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.456065.
+[SCHEDULER ITER3 LR1]: 160114 words processed Wed Nov 11 23:33:59 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.441796.
+[SCHEDULER ITER3 LR1]: 200066 words processed Wed Nov 11 23:34:14 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.433088.
+[SCHEDULER ITER3 LR1]: 240045 words processed Wed Nov 11 23:34:29 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.424178.
+[SCHEDULER ITER3 LR1]: 280057 words processed Wed Nov 11 23:34:44 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.416746.
+[SCHEDULER ITER3 LR1]: 320106 words processed Wed Nov 11 23:34:59 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.408769.
+[SCHEDULER ITER3 LR1]: 360024 words processed Wed Nov 11 23:35:14 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.403516.
+[SCHEDULER ITER3 LR1]: 400089 words processed Wed Nov 11 23:35:29 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.401051.
+[SCHEDULER ITER3 LR1]: 440067 words processed Wed Nov 11 23:35:44 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.397870.
+[SCHEDULER ITER3 LR1]: 480051 words processed Wed Nov 11 23:35:59 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.394888.
+[SCHEDULER ITER3 LR1]: 520140 words processed Wed Nov 11 23:36:14 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.392029.
+[SCHEDULER ITER3 LR1]: 560132 words processed Wed Nov 11 23:36:29 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.389343.
+[SCHEDULER ITER3 LR1]: 600118 words processed Wed Nov 11 23:36:44 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.387280.
+[SCHEDULER ITER3 LR1]: 640090 words processed Wed Nov 11 23:36:59 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.384322.
+[SCHEDULER ITER3 LR1]: 680075 words processed Wed Nov 11 23:37:14 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.382924.
+[SCHEDULER ITER3 LR1]: 720043 words processed Wed Nov 11 23:37:29 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.381426.
+[SCHEDULER ITER3 LR1]: 760012 words processed Wed Nov 11 23:37:44 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.379613.
+[SCHEDULER ITER3 LR1]: 800113 words processed Wed Nov 11 23:37:59 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.377966.
+[SCHEDULER ITER3 LR1]: 840089 words processed Wed Nov 11 23:38:14 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.375823.
+[SCHEDULER ITER3 LR1]: 880052 words processed Wed Nov 11 23:38:29 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.374291.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	42068
+gconf.fz2	42068
+[SCHEDULER ITER3 LR1]: Displaying result:
+[SCHEDULER ITER3 LR1]: LMResult status of rnn: <SEN_CN 42068> <W_CN 887521> <PPL_NET 268.9224164231> <PPL_OOV 236.66635714317> <LOGP -2206971.199076>
+[SCHEDULER ITER3 LR1]: Doing on /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf end.
+===VALIDATION 3===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER3 LR1]: 40095 words processed Wed Nov 11 23:38:39 2015.
+	[SCHEDULER ITER3 LR1]: log prob per sample :-2.367482.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	0
+gconf.fz2	3370
+[SCHEDULER ITER3 LR1]: Displaying result:
+[SCHEDULER ITER3 LR1]: LMResult status of rnn: <SEN_CN 3370> <W_CN 70390> <PPL_NET 246.139092058> <PPL_OOV 218.53948958668> <LOGP -172563.72722511>
+[SCHEDULER ITER3 LR1]: Doing on /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds end.
+[SCHEDULER ITER3 LR1]: saving net to file /home/slhome/txh18/workspace/nerv/play/dagL_test/params...
+
+[SCHEDULER ITER3 LR1]: preparing parameters...
+reading chunk 0 from 0
+metadata: return {type="nerv.LinearTransParam",id="ltp_hh"}
+
+reading chunk 1 from 1075691
+metadata: return {type="nerv.BiasParam",id="bp_h"}
+
+reading chunk 2 from 1079348
+metadata: return {type="nerv.LinearTransParam",id="ltp_ho"}
+
+reading chunk 3 from 35651987
+metadata: return {type="nerv.BiasParam",id="bp_o"}
+
+reading chunk 4 from 35768166
+metadata: return {type="nerv.LinearTransParam",id="ltp_ih"}
+
+reading chunk 5 from 70297054
+[SCHEDULER ITER3 LR1]: preparing parameters end.
+[SCHEDULER ITER3 LR1]: preparing layers...
+(23:38:49 2015-11-11)[nerv] info: create layer: recurrentL1
+(23:38:49 2015-11-11)[nerv] info: create layer: sigmoidL1
+(23:38:49 2015-11-11)[nerv] info: create layer: combinerL1
+(23:38:49 2015-11-11)[nerv] info: create layer: outputL
+(23:38:49 2015-11-11)[nerv] info: create layer: softmaxL
+(23:38:49 2015-11-11)[nerv] info: create layer: selectL1
+[SCHEDULER ITER3 LR1]: preparing layers end.
+[SCHEDULER ITER3 LR1]: Generate and initing TNN ...
+<input>	selectL1	0
+selectL1	recurrentL1	0
+recurrentL1	sigmoidL1	0
+sigmoidL1	combinerL1	0
+combinerL1	recurrentL1	1
+combinerL1	outputL	0
+outputL	softmaxL	0
+<input>	softmaxL	0
+softmaxL	<output>	0
+recurrentL1	#dim_in:	2	#dim_out:	1	#i_conns_p:	2	#o_conns_p	1
+outputL	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+combinerL1	#dim_in:	1	#dim_out:	2	#i_conns_p:	1	#o_conns_p	2
+sigmoidL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+selectL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	0	#o_conns_p	1
+softmaxL	#dim_in:	2	#dim_out:	1	#i_conns_p:	1	#o_conns_p	0
+TNN initing storage	selectL1	->	recurrentL1
+TNN initing storage	recurrentL1	->	sigmoidL1
+TNN initing storage	sigmoidL1	->	combinerL1
+TNN initing storage	combinerL1	->	recurrentL1
+TNN initing storage	combinerL1	->	outputL
+TNN initing storage	outputL	->	softmaxL
+[SCHEDULER ITER3 LR1]: Initing TNN end.
+===ITERATION 4 LR 1.000000===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER4 LR1]: 40092 words processed Wed Nov 11 23:39:03 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.542208.
+[SCHEDULER ITER4 LR1]: 80099 words processed Wed Nov 11 23:39:18 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.484538.
+[SCHEDULER ITER4 LR1]: 120004 words processed Wed Nov 11 23:39:33 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.457613.
+[SCHEDULER ITER4 LR1]: 160114 words processed Wed Nov 11 23:39:48 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.445247.
+[SCHEDULER ITER4 LR1]: 200066 words processed Wed Nov 11 23:40:03 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.437367.
+[SCHEDULER ITER4 LR1]: 240045 words processed Wed Nov 11 23:40:18 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.422811.
+[SCHEDULER ITER4 LR1]: 280057 words processed Wed Nov 11 23:40:33 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.408160.
+[SCHEDULER ITER4 LR1]: 320106 words processed Wed Nov 11 23:40:48 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.395672.
+[SCHEDULER ITER4 LR1]: 360024 words processed Wed Nov 11 23:41:03 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.386943.
+[SCHEDULER ITER4 LR1]: 400089 words processed Wed Nov 11 23:41:18 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.380113.
+[SCHEDULER ITER4 LR1]: 440067 words processed Wed Nov 11 23:41:33 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.374574.
+[SCHEDULER ITER4 LR1]: 480051 words processed Wed Nov 11 23:41:48 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.369999.
+[SCHEDULER ITER4 LR1]: 520140 words processed Wed Nov 11 23:42:03 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.365840.
+[SCHEDULER ITER4 LR1]: 560132 words processed Wed Nov 11 23:42:18 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.361944.
+[SCHEDULER ITER4 LR1]: 600118 words processed Wed Nov 11 23:42:33 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.357189.
+[SCHEDULER ITER4 LR1]: 640090 words processed Wed Nov 11 23:42:48 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.353203.
+[SCHEDULER ITER4 LR1]: 680075 words processed Wed Nov 11 23:43:03 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.350827.
+[SCHEDULER ITER4 LR1]: 720043 words processed Wed Nov 11 23:43:18 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.348638.
+[SCHEDULER ITER4 LR1]: 760012 words processed Wed Nov 11 23:43:33 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.346314.
+[SCHEDULER ITER4 LR1]: 800113 words processed Wed Nov 11 23:43:48 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.344168.
+[SCHEDULER ITER4 LR1]: 840089 words processed Wed Nov 11 23:44:03 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.341504.
+[SCHEDULER ITER4 LR1]: 880052 words processed Wed Nov 11 23:44:18 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.339703.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	42068
+gconf.fz2	42068
+[SCHEDULER ITER4 LR1]: Displaying result:
+[SCHEDULER ITER4 LR1]: LMResult status of rnn: <SEN_CN 42068> <W_CN 887521> <PPL_NET 246.79967809481> <PPL_OOV 218.53089509171> <LOGP -2174785.4056976>
+[SCHEDULER ITER4 LR1]: Doing on /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf end.
+===VALIDATION 4===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER4 LR1]: 40095 words processed Wed Nov 11 23:44:28 2015.
+	[SCHEDULER ITER4 LR1]: log prob per sample :-2.344507.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	0
+gconf.fz2	3370
+[SCHEDULER ITER4 LR1]: Displaying result:
+[SCHEDULER ITER4 LR1]: LMResult status of rnn: <SEN_CN 3370> <W_CN 70390> <PPL_NET 231.94898519805> <PPL_OOV 206.071146743> <LOGP -170681.90777664>
+[SCHEDULER ITER4 LR1]: Doing on /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds end.
+[SCHEDULER ITER4 LR1]: saving net to file /home/slhome/txh18/workspace/nerv/play/dagL_test/params...
+
+[SCHEDULER ITER4 LR1]: preparing parameters...
+reading chunk 0 from 0
+metadata: return {type="nerv.LinearTransParam",id="ltp_ih"}
+
+reading chunk 1 from 34532576
+metadata: return {type="nerv.BiasParam",id="bp_h"}
+
+reading chunk 2 from 34536232
+metadata: return {type="nerv.LinearTransParam",id="ltp_ho"}
+
+reading chunk 3 from 69123169
+metadata: return {type="nerv.BiasParam",id="bp_o"}
+
+reading chunk 4 from 69239437
+metadata: return {type="nerv.LinearTransParam",id="ltp_hh"}
+
+reading chunk 5 from 70314686
+[SCHEDULER ITER4 LR1]: preparing parameters end.
+[SCHEDULER ITER4 LR1]: preparing layers...
+(23:44:38 2015-11-11)[nerv] info: create layer: recurrentL1
+(23:44:38 2015-11-11)[nerv] info: create layer: sigmoidL1
+(23:44:38 2015-11-11)[nerv] info: create layer: combinerL1
+(23:44:38 2015-11-11)[nerv] info: create layer: outputL
+(23:44:38 2015-11-11)[nerv] info: create layer: softmaxL
+(23:44:38 2015-11-11)[nerv] info: create layer: selectL1
+[SCHEDULER ITER4 LR1]: preparing layers end.
+[SCHEDULER ITER4 LR1]: Generate and initing TNN ...
+<input>	selectL1	0
+selectL1	recurrentL1	0
+recurrentL1	sigmoidL1	0
+sigmoidL1	combinerL1	0
+combinerL1	recurrentL1	1
+combinerL1	outputL	0
+outputL	softmaxL	0
+<input>	softmaxL	0
+softmaxL	<output>	0
+recurrentL1	#dim_in:	2	#dim_out:	1	#i_conns_p:	2	#o_conns_p	1
+outputL	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+combinerL1	#dim_in:	1	#dim_out:	2	#i_conns_p:	1	#o_conns_p	2
+sigmoidL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+selectL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	0	#o_conns_p	1
+softmaxL	#dim_in:	2	#dim_out:	1	#i_conns_p:	1	#o_conns_p	0
+TNN initing storage	selectL1	->	recurrentL1
+TNN initing storage	recurrentL1	->	sigmoidL1
+TNN initing storage	sigmoidL1	->	combinerL1
+TNN initing storage	combinerL1	->	recurrentL1
+TNN initing storage	combinerL1	->	outputL
+TNN initing storage	outputL	->	softmaxL
+[SCHEDULER ITER4 LR1]: Initing TNN end.
+===ITERATION 5 LR 1.000000===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER5 LR1]: 40092 words processed Wed Nov 11 23:44:52 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.300797.
+[SCHEDULER ITER5 LR1]: 80099 words processed Wed Nov 11 23:45:07 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.296818.
+[SCHEDULER ITER5 LR1]: 120004 words processed Wed Nov 11 23:45:22 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.290547.
+[SCHEDULER ITER5 LR1]: 160114 words processed Wed Nov 11 23:45:37 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.291868.
+[SCHEDULER ITER5 LR1]: 200066 words processed Wed Nov 11 23:45:52 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.291909.
+[SCHEDULER ITER5 LR1]: 240045 words processed Wed Nov 11 23:46:07 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.287914.
+[SCHEDULER ITER5 LR1]: 280057 words processed Wed Nov 11 23:46:22 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.284605.
+[SCHEDULER ITER5 LR1]: 320106 words processed Wed Nov 11 23:46:37 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.281239.
+[SCHEDULER ITER5 LR1]: 360024 words processed Wed Nov 11 23:46:52 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.279270.
+[SCHEDULER ITER5 LR1]: 400089 words processed Wed Nov 11 23:47:07 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.280374.
+[SCHEDULER ITER5 LR1]: 440067 words processed Wed Nov 11 23:47:22 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.279921.
+[SCHEDULER ITER5 LR1]: 480051 words processed Wed Nov 11 23:47:37 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.279388.
+[SCHEDULER ITER5 LR1]: 520140 words processed Wed Nov 11 23:47:52 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.279796.
+[SCHEDULER ITER5 LR1]: 560132 words processed Wed Nov 11 23:48:07 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.278872.
+[SCHEDULER ITER5 LR1]: 600118 words processed Wed Nov 11 23:48:22 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.276549.
+[SCHEDULER ITER5 LR1]: 640090 words processed Wed Nov 11 23:48:37 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.274624.
+[SCHEDULER ITER5 LR1]: 680075 words processed Wed Nov 11 23:48:52 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.273749.
+[SCHEDULER ITER5 LR1]: 720043 words processed Wed Nov 11 23:49:07 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.273133.
+[SCHEDULER ITER5 LR1]: 760012 words processed Wed Nov 11 23:49:22 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.272442.
+[SCHEDULER ITER5 LR1]: 800113 words processed Wed Nov 11 23:49:37 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.271901.
+[SCHEDULER ITER5 LR1]: 840089 words processed Wed Nov 11 23:49:52 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.270503.
+[SCHEDULER ITER5 LR1]: 880052 words processed Wed Nov 11 23:50:07 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.269999.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	42068
+gconf.fz2	42068
+[SCHEDULER ITER5 LR1]: Displaying result:
+[SCHEDULER ITER5 LR1]: LMResult status of rnn: <SEN_CN 42068> <W_CN 887521> <PPL_NET 210.35665534337> <PPL_OOV 186.22135221397> <LOGP -2110194.4304516>
+[SCHEDULER ITER5 LR1]: Doing on /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf end.
+===VALIDATION 5===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER5 LR1]: 40095 words processed Wed Nov 11 23:50:17 2015.
+	[SCHEDULER ITER5 LR1]: log prob per sample :-2.320095.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	0
+gconf.fz2	3370
+[SCHEDULER ITER5 LR1]: Displaying result:
+[SCHEDULER ITER5 LR1]: LMResult status of rnn: <SEN_CN 3370> <W_CN 70390> <PPL_NET 217.2449530512> <PPL_OOV 193.72332502292> <LOGP -168702.53801867>
+[SCHEDULER ITER5 LR1]: Doing on /home/slhome/txh18/workspace/nerv/nerv/nerv/examples/lmptb/PTBdata/ptb.valid.txt.adds end.
+[SCHEDULER ITER5 LR1]: saving net to file /home/slhome/txh18/workspace/nerv/play/dagL_test/params...
+
+[SCHEDULER ITER5 LR1]: preparing parameters...
+reading chunk 0 from 0
+metadata: return {type="nerv.LinearTransParam",id="ltp_hh"}
+
+reading chunk 1 from 1074581
+metadata: return {type="nerv.BiasParam",id="bp_h"}
+
+reading chunk 2 from 1078237
+metadata: return {type="nerv.LinearTransParam",id="ltp_ho"}
+
+reading chunk 3 from 35676725
+metadata: return {type="nerv.BiasParam",id="bp_o"}
+
+reading chunk 4 from 35793028
+metadata: return {type="nerv.LinearTransParam",id="ltp_ih"}
+
+reading chunk 5 from 70329168
+[SCHEDULER ITER5 LR1]: preparing parameters end.
+[SCHEDULER ITER5 LR1]: preparing layers...
+(23:50:27 2015-11-11)[nerv] info: create layer: recurrentL1
+(23:50:27 2015-11-11)[nerv] info: create layer: sigmoidL1
+(23:50:27 2015-11-11)[nerv] info: create layer: combinerL1
+(23:50:27 2015-11-11)[nerv] info: create layer: outputL
+(23:50:27 2015-11-11)[nerv] info: create layer: softmaxL
+(23:50:27 2015-11-11)[nerv] info: create layer: selectL1
+[SCHEDULER ITER5 LR1]: preparing layers end.
+[SCHEDULER ITER5 LR1]: Generate and initing TNN ...
+<input>	selectL1	0
+selectL1	recurrentL1	0
+recurrentL1	sigmoidL1	0
+sigmoidL1	combinerL1	0
+combinerL1	recurrentL1	1
+combinerL1	outputL	0
+outputL	softmaxL	0
+<input>	softmaxL	0
+softmaxL	<output>	0
+recurrentL1	#dim_in:	2	#dim_out:	1	#i_conns_p:	2	#o_conns_p	1
+outputL	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+combinerL1	#dim_in:	1	#dim_out:	2	#i_conns_p:	1	#o_conns_p	2
+sigmoidL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	1	#o_conns_p	1
+selectL1	#dim_in:	1	#dim_out:	1	#i_conns_p:	0	#o_conns_p	1
+softmaxL	#dim_in:	2	#dim_out:	1	#i_conns_p:	1	#o_conns_p	0
+TNN initing storage	selectL1	->	recurrentL1
+TNN initing storage	recurrentL1	->	sigmoidL1
+TNN initing storage	sigmoidL1	->	combinerL1
+TNN initing storage	combinerL1	->	recurrentL1
+TNN initing storage	combinerL1	->	outputL
+TNN initing storage	outputL	->	softmaxL
+[SCHEDULER ITER5 LR1]: Initing TNN end.
+===ITERATION 6 LR 1.000000===
+[LOG]LMSeqReader: opening file /home/slhome/txh18/workspace/nerv/play/dagL_test/train_fn_shuf...
+batch_size:	10	chunk_size	15
+[SCHEDULER ITER6 LR1]: 40092 words processed Wed Nov 11 23:50:41 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.255734.
+[SCHEDULER ITER6 LR1]: 80099 words processed Wed Nov 11 23:50:56 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.253719.
+[SCHEDULER ITER6 LR1]: 120004 words processed Wed Nov 11 23:51:11 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.247966.
+[SCHEDULER ITER6 LR1]: 160114 words processed Wed Nov 11 23:51:26 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.249946.
+[SCHEDULER ITER6 LR1]: 200066 words processed Wed Nov 11 23:51:41 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.250355.
+[SCHEDULER ITER6 LR1]: 240045 words processed Wed Nov 11 23:51:56 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.246729.
+[SCHEDULER ITER6 LR1]: 280057 words processed Wed Nov 11 23:52:11 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.243410.
+[SCHEDULER ITER6 LR1]: 320106 words processed Wed Nov 11 23:52:26 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.240495.
+[SCHEDULER ITER6 LR1]: 360024 words processed Wed Nov 11 23:52:41 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.239077.
+[SCHEDULER ITER6 LR1]: 400089 words processed Wed Nov 11 23:52:56 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.238622.
+[SCHEDULER ITER6 LR1]: 440067 words processed Wed Nov 11 23:53:11 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.238504.
+[SCHEDULER ITER6 LR1]: 480051 words processed Wed Nov 11 23:53:26 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.238241.
+[SCHEDULER ITER6 LR1]: 520140 words processed Wed Nov 11 23:53:41 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.239162.
+[SCHEDULER ITER6 LR1]: 560132 words processed Wed Nov 11 23:53:56 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.239312.
+[SCHEDULER ITER6 LR1]: 600118 words processed Wed Nov 11 23:54:11 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.237412.
+[SCHEDULER ITER6 LR1]: 640090 words processed Wed Nov 11 23:54:26 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.235684.
+[SCHEDULER ITER6 LR1]: 680075 words processed Wed Nov 11 23:54:41 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.235523.
+[SCHEDULER ITER6 LR1]: 720043 words processed Wed Nov 11 23:54:56 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.235372.
+[SCHEDULER ITER6 LR1]: 760012 words processed Wed Nov 11 23:55:11 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.234774.
+[SCHEDULER ITER6 LR1]: 800113 words processed Wed Nov 11 23:55:26 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.234615.
+[SCHEDULER ITER6 LR1]: 840089 words processed Wed Nov 11 23:55:41 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.233365.
+[SCHEDULER ITER6 LR1]: 880052 words processed Wed Nov 11 23:55:56 2015.
+	[SCHEDULER ITER6 LR1]: log prob per sample :-2.233021.
+[LOG]LMSeqReader: file expires, closing.
+gconf.fz	42068
+gconf.fz2	42068
+[SCHEDULER ITER6 LR1]: Displaying result:
+[SCHEDULER ITER6 LR1]: LMResult status of rnn: <SEN_CN 42068> <W_CN 887521> <PPL_NET 192.50276395295> <PPL_OOV 171.03816037907> <LOGP -2075858.7043799>
+[SCHEDULE