R中的Hadoop wordcount示例

我安裝了hadoop-3.0.0-alpha2，我試圖執行Mapreduce wordcount示例。 我創建了mapper.R和reducer.R腳本，但是當我嘗試執行作業R中的Hadoop wordcount示例

hadoop jar /home/rania/Downloads/hadoop-streaming-0.20.204.0.jar \ 
-file /home/rania/Downloads/mapper.R -mapper /home/rania/Downloads/mapper.R \ 
-file /home/rania/Downloads/reducer.R -reducer /home/rania/Downloads/reducer.R \ 
-input /readme -output /RCount

我得到以下

2017-06-04 08:12:42,252 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
2017-06-04 08:12:43,119 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address 
packageJobJar: [/home/rania/Downloads/mapper.R, /home/rania/Downloads/reducer.R] [] /tmp/streamjob5589642909909116910.jar tmpDir=null 
2017-06-04 08:12:43,303 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 
2017-06-04 08:12:43,603 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 
2017-06-04 08:12:43,734 ERROR streaming.StreamJob: Error launching job , Output path already exists : Output directory hdfs://localhost:9000/RCount already exists 
Streaming Job Failed!

出了什麼問題？謝謝！

來源

2017-06-04 Rania

嘗試將腳本運行到hdfs上尚不存在的輸出目錄。一個新的目錄將會用你選擇的任何名字創建。如果您想再次使用相同的目錄，則必須刪除其中的文件並將其刪除，然後再次使用相同的輸出目錄名稱/ RCount運行腳本

來源

2017-10-17 05:48:32 scruffyTail

R中的Hadoop wordcount示例

回答

相關問題