Skip to content
This repository has been archived by the owner on May 24, 2018. It is now read-only.

unrolled LSTM layer with batch BPTT #159

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

unrolled LSTM layer with batch BPTT #159

wants to merge 6 commits into from

Conversation

xiw9
Copy link

@xiw9 xiw9 commented May 21, 2015

An fast unrolled LSTM layer similar to BVLC/caffe#1873
Need 2 inputs,
The input sequence nodes_in[0] and the corresponding sequence label nodes_in[1].
nodes_in[0] size: [batch_size][1][1][input_width]
nodes_in[1] size: [batch_size][1][1][1]
Example config file:

data = train
iter = csv
  filename = "...."
  has_header = 0
iter = attachtxt
  txtfilename = "...."
iter = threadbuffer
  buffer_size = 4
iter = end

eval = val
iter = csv
  filename = "...."
  has_header = 0
iter = attachtxt
  txtfilename = "...."
iter = threadbuffer
  buffer_size = 4
iter = end

extra_data_shape[0] = 1,1,1
extra_data_num = 1

netconfig=start
layer[in,in_1->2] = lstm:lstm1
  nhidden = 1024
  parallel_size = 8
layer[2,in_1->3] = lstm:lstm2
  nhidden = 512
  parallel_size = 8
layer[3->4] = fullc:fc1
  nhidden = 51
layer[4->4] = softmax:softmax1
netconfig=end

# evaluation metric
metric = error

max_round = 40
num_round = 40

# input shape not including batch
input_shape = 1,1,4096

batch_size = 512

@@ -22,7 +22,7 @@ class AttachTxtIterator : public IIterator<DataBatch> {
}
virtual void SetParam(const char *name, const char *val) {
base_->SetParam(name, val);
if (!strcmp(name, "filename")) filename_ = val;
if (!strcmp(name, "txtfilename")) filename_ = val;
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conflict between txt iter and csv iter

data = train
iter = csv
  filename = "............"
  has_header = 0
iter = attachtxt
  txtfilename = "............."
iter = threadbuffer
  buffer_size = 4
iter = end

@xiw9 xiw9 changed the title unrolled LSTM layer with batched BPTT unrolled LSTM layer with batch BPTT May 21, 2015
@antinucleon
Copy link
Contributor

Thanks for your PR. I am still working on general case of LSTM, which is able to run CNN + LSTM together. In that branch we have special text IO for sequence. However I still have some bug to fix. Do you have interest to work together on that branch?

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants