Tensorflow tfprof LSTMCell

Question

我正在使用tfprof來獲取模型轉發路徑所需的觸發器數量。 我的模型之後是3層LSTM和完全連接層。 我觀察到完全連接層的計算數量線性增長，而LSTM層的計算數量沒有變化。這怎麼可能？

tfprof Report for 1 timestamp forward path。

==================Model Analysis Report====================== 
_TFProfRoot (0/2.71m flops) 
    rnn/while/multi_rnn_cell/cell_1/lstm_cell/lstm_cell_1/MatMul (1.05m/1.05m flops) 
    rnn/while/multi_rnn_cell/cell_2/lstm_cell/lstm_cell_1/MatMul (1.05m/1.05m flops) 
    rnn/while/multi_rnn_cell/cell_0/lstm_cell/lstm_cell_1/MatMul (606.21k/606.21k flops) 
    fc_layer/MatMul (1.54k/1.54k flops) 
    rnn/while/multi_rnn_cell/cell_0/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops) 
    rnn/while/multi_rnn_cell/cell_1/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops) 
    rnn/while/multi_rnn_cell/cell_2/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops) 
    fc_layer/BiasAdd (3/3 flops) 

tfprof報告2個時間戳正向路徑。

==================Model Analysis Report====================== 
_TFProfRoot (0/2.71m flops) 
    rnn/while/multi_rnn_cell/cell_1/lstm_cell/lstm_cell_1/MatMul (1.05m/1.05m flops) 
    rnn/while/multi_rnn_cell/cell_2/lstm_cell/lstm_cell_1/MatMul (1.05m/1.05m flops) 
    rnn/while/multi_rnn_cell/cell_0/lstm_cell/lstm_cell_1/MatMul (606.21k/606.21k flops) 
    fc_layer/MatMul (3.07k/3.07k flops) 
    rnn/while/multi_rnn_cell/cell_0/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops) 
    rnn/while/multi_rnn_cell/cell_1/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops) 
    rnn/while/multi_rnn_cell/cell_2/lstm_cell/lstm_cell_1/BiasAdd (1.02k/1.02k flops) 
    fc_layer/BiasAdd (6/6 flops) 

Answer 1

tfprof對圖進行靜態分析並計算每個圖節點的浮動操作。

我假設你正在使用dynamic_rnn或類似tf.while_loop的東西。在這種情況下，一個圖形節點一次出現在圖中 ，但實際上在運行時多次運行。

在這種情況下，tfprof無法靜態計算出多少個步驟（您的單詞中的時間戳）將被運行。因此，它只計算一次浮動操作 。

現在的工作可能是你自己的時間步長。