tf.contrib.rnn.LayerNormBasicLSTMCell

class tf.contrib.rnn.LayerNormBasicLSTMCell

See the guide: RNN and Cells (contrib) > Core RNN Cells for use with TensorFlow's core RNN methods

LSTM unit with layer normalization and recurrent dropout.

This class adds layer normalization and recurrent dropout to a basic LSTM unit. Layer normalization implementation is based on:

https://arxiv.org/abs/1607.06450.

"Layer Normalization" Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton

and is applied before the internal nonlinearities. Recurrent dropout is base on:

https://arxiv.org/abs/1603.05118

"Recurrent Dropout without Memory Loss" Stanislau Semeniuta, Aliaksei Severyn, Erhardt Barth.

Properties

output_size

state_size

Methods

__init__(num_units, forget_bias=1.0, input_size=None, activation=tf.tanh, layer_norm=True, norm_gain=1.0, norm_shift=0.0, dropout_keep_prob=1.0, dropout_prob_seed=None)

Initializes the basic LSTM cell.

Args:

• num_units: int, The number of units in the LSTM cell.
• forget_bias: float, The bias added to forget gates (see above).
• input_size: Deprecated and unused.
• activation: Activation function of the inner states.
• layer_norm: If True, layer normalization will be applied.
• norm_gain: float, The layer normalization gain initial value. If layer_norm has been set to False, this argument will be ignored.
• norm_shift: float, The layer normalization shift initial value. If layer_norm has been set to False, this argument will be ignored.
• dropout_keep_prob: unit Tensor or float between 0 and 1 representing the recurrent dropout probability value. If float and 1.0, no dropout will be applied.
• dropout_prob_seed: (optional) integer, the randomness seed.

zero_state(batch_size, dtype)

Return zero-filled state tensor(s).

Args:

• batch_size: int, float, or unit Tensor representing the batch size.
• dtype: the data type to use for the state.

Returns:

If state_size is an int or TensorShape, then the return value is a N-D tensor of shape [batch_size x state_size] filled with zeros.

If state_size is a nested list or tuple, then the return value is a nested list or tuple (of the same structure) of 2-D tensors with the shapes [batch_size x s] for each s in state_size.

Defined in tensorflow/contrib/rnn/python/ops/rnn_cell.py.