theano update error when running DT_RNN_Tut tutorial

[moore269]

data length is 11
data length is 11
data length is 11
Traceback (most recent call last):
File "DT_RNN_Tut.py", line 416, in
jobman(state, None)
File "DT_RNN_Tut.py", line 225, in jobman
name='valid_fn', updates=valid_updates)
File "/Library/Python/2.7/site-packages/theano/compile/function.py", line 309, in function
output_keys=output_keys)
File "/Library/Python/2.7/site-packages/theano/compile/pfunc.py", line 487, in pfunc
no_default_updates=no_default_updates)
File "/Library/Python/2.7/site-packages/theano/compile/pfunc.py", line 214, in rebuild_collect_shared
raise TypeError(err_msg, err_sug)
TypeError: ('An update must have the same type as the original shared variable (shared_var=<TensorType(float32, vector)>, shared_var.type=TensorType(float32, vector), update_val=Subtensor{int64}.0, update_val.type=TensorType(float64, vector)).', 'If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')

I just modified this part in the code:
state['path']= "data_words.npz"
state['dictionary']= "data_char_words.npz"
state['chunks'] = 'words'

I also installed the latest version of theano, tables, hdf5.
Not sure how I should proceed to get the tutorial working. Any help is appreciated, thanks!

[nouiz]

Hi,

This is the important part of the error:

TypeError: ('An update must have the same type as the original shared
variable (shared_var=, shared_var.type=TensorType(float32, vector),
update_val=Subtensor{int64}.0, update_val.type=TensorType(float64,
vector)).', 'If the difference is related to the broadcast pattern, you can
call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to
remove broadcastable dimensions.')

The shared variable is of dtype float32, but its new value is of dtype
float64.

Something caused the code to upcast the data to float64. Use the theano
flag warn_float64 to help you find where this happen:

http://deeplearning.net/software/theano/library/config.html#config.warn_float64

Maybe the files you changed don't have the data in the right dtype in it.

On Sun, Sep 13, 2015 at 6:42 PM, moore269 [email protected] wrote:

data length is 11
data length is 11
data length is 11
Traceback (most recent call last):
File "DT_RNN_Tut.py", line 416, in
jobman(state, None)
File "DT_RNN_Tut.py", line 225, in jobman
name='valid_fn', updates=valid_updates)
File "/Library/Python/2.7/site-packages/theano/compile/function.py", line
309, in function
output_keys=output_keys)
File "/Library/Python/2.7/site-packages/theano/compile/pfunc.py", line
487, in pfunc
no_default_updates=no_default_updates)
File "/Library/Python/2.7/site-packages/theano/compile/pfunc.py", line
214, in rebuild_collect_shared
raise TypeError(err_msg, err_sug)
TypeError: ('An update must have the same type as the original shared
variable (shared_var=, shared_var.type=TensorType(float32, vector),
update_val=Subtensor{int64}.0, update_val.type=TensorType(float64,
vector)).', 'If the difference is related to the broadcast pattern, you can
call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to
remove broadcastable dimensions.')

I just modified this part in the code:
state['path']= "data_words.npz"
state['dictionary']= "data_char_words.npz"
state['chunks'] = 'words'

I also installed the latest version of theano, tables, hdf5.
Not sure how I should proceed to get the tutorial working. Any help is
appreciated, thanks!

—
Reply to this email directly or view it on GitHub
#36.

[moore269]

Thanks for your help! Here, I produced the following dump when I flag for float64. The first time I see the creation of float64 is line 140. I’m not sure if it is supposed to produce float32 instead. Any ideas?

data length is 11
data length is 11
data length is 11
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/ff_layers.py:255: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
W_ems = [(x+y) for x, y in zip(self.W_ems, self.nW_ems)]
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/utils/utils.py:177: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
return matrix[inp]
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/ff_layers.py:277: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
emb_val += b_ems[0]
DT_RNN_Tut.py:140: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
reset = TT.scalar('reset')
DT_RNN_Tut.py:142: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
init_state=h0_reset,
/Library/Python/2.7/site-packages/theano/scan_module/scan.py:475: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
actual_slice = seq['input'][k - mintap]
/Library/Python/2.7/site-packages/theano/scan_module/scan.py:477: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
_seq_val_slice = _seq_val[k - mintap]
/Library/Python/2.7/site-packages/theano/scan_module/scan.py:478: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
nw_slice = _seq_val_slice.type()
/Library/Python/2.7/site-packages/theano/scan_module/scan.py:512: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
nw_seq = seq['input'][start:]
/Library/Python/2.7/site-packages/theano/scan_module/scan.py:557: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
scan_seqs = [seq[:actual_n_steps] for seq in scan_seqs]
/Library/Python/2.7/site-packages/theano/scan_module/scan_utils.py:95: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
nw_x = x.type()
/Library/Python/2.7/site-packages/theano/scan_module/scan.py:632: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
tensor.shape_padleft(actual_arg), 0),
/Library/Python/2.7/site-packages/theano/compile/ops.py:689: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
return gof.Apply(self, [x], [t()])
/Library/Python/2.7/site-packages/theano/scan_module/scan_utils.py:414: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
dtype=tensor_var.dtype)
/Library/Python/2.7/site-packages/theano/scan_module/scan_utils.py:416: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
return tensor.set_subtensor(empty[:shapes[0]], tensor_var)
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/rec_layers.py:780: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
W_hhs = [(x+y) for x, y in zip(self.W_hhs,self.nW_hhs)]
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/rec_layers.py:781: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
W_shp = [(x+y) for x, y in zip(self.W_shortp,self.nW_shortp)]
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/rec_layers.py:796: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
h = self.activation[0](TT.dot%28state_before, W_hhs[0]%29 + slice_state_below%280%29)
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/rec_layers.py:791: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
return sb[st:ed]
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/rec_layers.py:815: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
TT.dot(state_before, W_shp[dx-1]) +
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/rec_layers.py:816: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
slice_state_below(dx))
/Library/Python/2.7/site-packages/theano/compile/pfunc.py:131: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
strict=rebuild_strict)
/Library/Python/2.7/site-packages/theano/compile/function_module.py:165: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
fgraph = gof.fg.FunctionGraph(orig_inputs, orig_outputs)
/Library/Python/2.7/site-packages/theano/scan_module/scan_utils.py:238: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
tmp_replace = [(x, x.type()) for x, y in items]
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:627: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
[t() for t in self.output_types])
/Library/Python/2.7/site-packages/theano/scan_module/scan.py:1066: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
out_ls.append(out[offsets[idx]:])
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/cost_layers.py:1006: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
emb_val = TT.dot(emb_val, W_em + nW_em)
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/cost_layers.py:1023: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
emb_val = temp * (emb_val + self.b_em)
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/utils/utils.py:80: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
e = TT.exp(x)
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/utils/utils.py:81: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
return e / TT.sum(e, axis=1).dimshuffle(0, 'x')
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/cost_layers.py:1093: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
return class_probs.flatten()[new_targ]
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/cost_layers.py:1132: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
cost = -TT.log(_grab_probs(class_probs, target))
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/cost_layers.py:1134: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
self.word_probs = TT.exp(-cost.reshape(target_shape))
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/cost_layers.py:1152: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
self.cost = cost.sum()
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/cost_layers.py:1156: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
self.cost = self.cost_scale
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/cost_layers.py:379: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
grads = TT.grad(cost.mean(), self.params)
/Library/Python/2.7/site-packages/theano/gradient.py:1309: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
rval = tensor.ones_like(x)
/Library/Python/2.7/site-packages/theano/gradient.py:1093: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
input_grads = node.op.grad(inputs, new_output_grads)
/Library/Python/2.7/site-packages/theano/gradient.py:1275: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
grad_dict[var] = reduce(lambda x, y: x + y, terms)
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:1935: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
dC_dinps_t[dx + self.n_seqs] += dC_dXtm1
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:1938: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
outer_inp_seqs = [x[::-1] for x in inputs[1:1 + self.n_seqs]]
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:1954: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
x[:-1][::-1] for x in self.outer_sitsot_outs(outs)]
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:1982: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
outer_inp_seqs += [x[::-1][:-1] for x in self.outer_sitsot_outs(outs)]
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:1983: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
outer_inp_seqs += [x[::-1] for x in self.outer_nitsot_outs(outs)]
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:1987: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
outer_inp_seqs = [seq[:grad_steps] for seq in outer_inp_seqs]
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:2117: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
outer_inp_mitmot.append(dC_douts[idx + offset][::-1])
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:2221: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
dtype=y.dtype))
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:2282: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
gradients.append(x[::-1])
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:2299: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
gradients.append(x[::-1])
/Library/Python/2.7/site-packages/theano/scan_module/scan_op.py:2337: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
gradients.append(x[-1])
DT_RNN_Tut.py:200: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
nw_h0 = rec_layer.out[rec_layer.out.shape[0]-1]
DT_RNN_Tut.py:209: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
init_state=h0val*reset,
DT_RNN_Tut.py:211: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
nw_h0 = rec_layer.out[rec_layer.out.shape[0]-1]
/Users/macbookair/Dropbox/research year3.2/experimentalCode/GroundHog/groundhog/layers/cost_layers.py:1008: UserWarning: You are creating a TensorVariable with float64 dtype. You requested an action via the Theano flag warn_float64={ignore,warn,raise,pdb}.
emb_val = TT.dot(emb_val, W_em)
Traceback (most recent call last):
File "DT_RNN_Tut.py", line 416, in
jobman(state, None)
File "DT_RNN_Tut.py", line 225, in jobman
name='valid_fn', updates=valid_updates)
File "/Library/Python/2.7/site-packages/theano/compile/function.py", line 309, in function
output_keys=output_keys)
File "/Library/Python/2.7/site-packages/theano/compile/pfunc.py", line 487, in pfunc
no_default_updates=no_default_updates)
File "/Library/Python/2.7/site-packages/theano/compile/pfunc.py", line 214, in rebuild_collect_shared
raise TypeError(err_msg, err_sug)
TypeError: ('An update must have the same type as the original shared variable (shared_var=<TensorType(float32, vector)>, shared_var.type=TensorType(float32, vector), update_val=Subtensor{int64}.0, update_val.type=TensorType(float64, vector)).', 'If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')

[nouiz]

You need to use that flag:

warn_float64=pdb

then investigate. We don't have the time to do more then guide you. Also

[moore269]

My apologies, I'm just not very familiar with theano and a little overwhelmed.

So far, I am just trying to get the tutorial working on a small sample of data. Maybe giving you my few steps that I took might be easier to help guide me? Again, any help is appreciated.

I created train, test, and valid files in the sample_data folder each consisting of this string.
"hello world i am trying to do this right now over"
python generate.py --dest=data_words --level=words --dtype=int64 sample_data
python DT_RNN_Tut.py

and again I had only modified this part
state['path']= "data_words.npz"
state['dictionary']= "data_char_words.npz"
state['chunks'] = 'words'

I had tried with dtype=int32, but same error. Is this how I am supposed to be preparing the data?

[moore269]

Also, I had tried pdb, but wasn't sure I could conclude anything from it

[nouiz]

If you are just starting, I would suggest that you check this update to the
README we just merged:

https://github.com/lisa-groundhog/GroundHog/pull/37/files

blocks is more documented and is supported. This is probably a better
starting point.

On Mon, Sep 14, 2015 at 10:41 AM, moore269 [email protected] wrote:

Also, I had tried pdb, but wasn't sure I could conclude anything from it

—
Reply to this email directly or view it on GitHub
#36 (comment)
.

[moore269]

oh, this looks very useful, thanks. Is the RNN LM model implemented in blocks as well?

[rizar]

Please check https://github.com/mila-udem/blocks-examples for examples of
things one can do with Blocks. The Markov Chain example is the closest to
language modelling.

On 14 September 2015 at 11:11, moore269 [email protected] wrote:

oh, this looks very useful, thanks. Is the RNN LM model implemented in
blocks as well?

—
Reply to this email directly or view it on GitHub
#36 (comment)
.