2016-08-24 113 views
2

我想在Hive-Tez上運行查詢後獲取Map-Reduce日誌的解釋? INFO之後的內容:傳達的是什麼? 這裏我附上一個樣本Map-Reduce日誌Hive-Tez

INFO : Session is already open 
INFO : Dag name: SELECT a.Model...) 
INFO : Tez session was closed. Reopening... 
INFO : Session re-established. 
INFO : 
INFO : Status: Running (Executing on YARN cluster with App id application_14708112341234_1234) 

INFO : Map 1: -/- Map 3: -/- Map 4: -/- Map 7: -/- Reducer 2: 0/15  Reducer 5: 0/26 Reducer 6: 0/13 
INFO : Map 1: -/- Map 3: 0/118 Map 4: 0/118 Map 7: 0/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13 
INFO : Map 1: 0/118 Map 3: 0/118 Map 4: 0/118 Map 7: 0/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13 
INFO : Map 1: 0/118 Map 3: 0/118 Map 4: 0(+5)/118 Map 7: 0/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13 
INFO : Map 1: 0/118 Map 3: 0(+5)/118 Map 4: 0(+7)/118 Map 7: 0(+1)/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13 
INFO : Map 1: 0/118 Map 3: 0(+15)/118 Map 4: 0(+18)/118 Map 7: 0(+1)/1 Reducer 2: 0/15 Reducer 5: 0/26 Reducer 6: 0/13 

回答

3

您發佈的日誌是DAG執行日誌。所述DAG包含 地圖1,地圖3,地圖4,地圖的7個映射器頂點和減速器:減速機2,減速5,減速6

INFO : Map 1: -/-這意味着該頂點沒有初始化,不計算映射器的數目然而。

Map 4: 0(+7)/118 - 這意味着總共有118個映射器,其中7個並行運行,0個尚未完成,118-7 = 111個未決。

Reducer 2: 0/15 - This means that totally there are 15 reducers, 0 of them are running, 0 of them are completed (15 reducers pending). 

負數(也有在你的例子沒有這樣的)=失敗或殺死映射器或減速

+0

正是我一直在尋找的號碼!你有這樣的來源嗎? – Dimpl

+0

我沒有這方面的資源。但是你可以在tez執行期間輕鬆地檢查這個看着作業跟蹤器的東西。 – leftjoin