Tensorflow：替換爲tf.nn.rnn_cell._linear（輸入，大小，0，範圍）

Question

我試圖從https://arxiv.org/pdf/1609.05473.pdf運行SequenceGAN（https://github.com/LantaoYu/SeqGAN）。
固定明顯的錯誤，就像stack更換pack後，它仍然無法運行，因爲高速公路網絡的一部分需要tf.nn.rnn_cell._linear功能：

# highway layer that borrowed from https://github.com/carpedm20/lstm-char-cnn-tensorflow 
def highway(input_, size, layer_size=1, bias=-2, f=tf.nn.relu): 
    """Highway Network (cf. http://arxiv.org/abs/1505.00387). 

    t = sigmoid(Wy + b) 
    z = t * g(Wy + b) + (1 - t) * y 
    where g is nonlinearity, t is transform gate, and (1 - t) is carry gate. 
    """ 
    output = input_ 
    for idx in range(layer_size): 
     output = f(tf.nn.rnn_cell._linear(output, size, 0, scope='output_lin_%d' % idx)) #tf.contrib.layers.linear instad doesn't work either. 
     transform_gate = tf.sigmoid(tf.nn.rnn_cell._linear(input_, size, 0, scope='transform_lin_%d' % idx) + bias) 
     carry_gate = 1. - transform_gate 

     output = transform_gate * output + carry_gate * input_ 

    return output 

的tf.nn.rnn_cell._linear功能不會出現在那裏了在Tensorflow 1.0或0.12中，我不知道如何替換它。我找不到任何這種新的實現，或tensorflow的github上的任何信息或（不幸的是非常稀疏）文檔。

有沒有人知道功能的新吊墜？ 非常感謝！

Answer 1

隨着版本1.0，東西已經四處移動。我有類似的搜索更新tf.nn.rnn_cell.LSTMCell到tf.contrib.rnn.BasicLSTMCell。

對於您的情況tf.nn.rnn_cell._linear現在生活在tf.contrib.rnn.python.ops.core_rnn_cell_impl以及BasicRNNCell的定義中。檢查BasicRNNCell docs和source code，我們在L113-L118處看到_linear的使用。

def __call__(self, inputs, state, scope=None): 
    """Most basic RNN: output = new_state = act(W * input + U * state + B).""" 
    with _checked_scope(self, scope or "basic_rnn_cell", reuse=self._reuse): 
     output = self._activation(
      _linear([inputs, state], self._num_units, True)) 
    return output, output 

的_linear方法在line 854定義爲：
Linear map: sum_i(args[i] * W[i]), where W[i] is a variable.

祝你好運！

Answer 2

爲了解決這個問題，我們可以定義一個linear（）函數。

def linear(input_, output_size, scope=None): 
    ''' 
    Linear map: output[k] = sum_i(Matrix[k, i] * args[i]) + Bias[k] 
    Args: 
     args: a tensor or a list of 2D, batch x n, Tensors. 
    output_size: int, second dimension of W[i]. 
    scope: VariableScope for the created subgraph; defaults to "Linear". 
    Returns: 
    A 2D Tensor with shape [batch x output_size] equal to 
    sum_i(args[i] * W[i]), where W[i]s are newly created matrices. 
    Raises: 
    ValueError: if some of the arguments has unspecified or wrong shape. 
    ''' 

    shape = input_.get_shape().as_list() 
    if len(shape) != 2: 
     raise ValueError("Linear is expecting 2D arguments: %s" % str(shape)) 
    if not shape[1]: 
     raise ValueError("Linear expects shape[1] of arguments: %s" % str(shape)) 
    input_size = shape[1] 

    # Now the computation. 
    with tf.variable_scope(scope or "SimpleLinear"): 
     matrix = tf.get_variable("Matrix", [output_size, input_size], dtype=input_.dtype) 
     bias_term = tf.get_variable("Bias", [output_size], dtype=input_.dtype) 

    return tf.matmul(input_, tf.transpose(matrix)) + bias_term 


def highway(input_, size, num_layers=1, bias=-2.0, f=tf.nn.relu, scope='Highway'): 
    """Highway Network (cf. http://arxiv.org/abs/1505.00387). 
    t = sigmoid(Wy + b) 
    z = t * g(Wy + b) + (1 - t) * y 
    where g is nonlinearity, t is transform gate, and (1 - t) is carry gate. 
    """ 

    with tf.variable_scope(scope): 
     for idx in range(num_layers): 
      g = f(linear(input_, size, scope='highway_lin_%d' % idx)) 

      t = tf.sigmoid(linear(input_, size, scope='highway_gate_%d' % idx) + bias) 

      output = t * g + (1. - t) * input_ 
      input_ = output 

    return output 

https://github.com/mkroutikov/tf-lstm-char-cnn/blob/7e899e6992cbf9a96e6d791e5d364eaaeec339a2/model.py

Answer 3

ruoho ruotsi的答案几乎是正確的： 然而，分別定義linear不位於tf.contrib.rnn.basicRNNCell，但在tf.contrib.rnn.python.ops.rnn_cell，或tf.contrib.rnn.python.ops.core_rnn_cell_impl。

你可以找到它們的源代碼here和here。

Answer 4

至於現在（1.2r0），您可以用 tf.contrib.layers.fully_connected(inputs=[inputs, state], num_outputs=self._num_units, biases_initializer=tf.constant_initializer(0.0), activation_fn=None) 注意替換它的是biases_initializer默認爲_linear並沒有偏向在所有的fully_connected全部爲零。

Answer 5

我在使用SkFlow的TensorFlowDNNRegressor時遇到了這個錯誤。 我第一次看到ruoho ruots的答案，我有點困惑。 但第二天我意識到他的意思。

這裏是我做的：

from tensorflow.python.ops import rnn_cell_impl 

取代tf.nn.rnn_cell._linear與rnn_cell_impl._linear