無法在Cosmos中使用自定義Mapreduce jar文件

Question

我創建了我自己的Mapreduce jar文件，並使用HDFS shell命令成功在Cosmos'old Hadoop cluster中進行了測試。下一步是在新集羣中測試同一個jar，所以我將它上傳到我的主文件夾（user/my.username）的 new cluster's HDFS。

當我嘗試使用下面的捲曲後啓動MapReduce工作，

curl -X POST "http://computing.cosmos.lab.fiware.org:12000/tidoop/v1/user/my.username/jobs" -d '{"jar":"dt.jar","class_name":"DistanceTest","lib_jars":"dt.jar","input":"input","output":"output"}' -H "Content-Type: application/json" -H "X-Auth-Token: xxxxxxxxxxxxxxxxxxx" 

我得到：

{ 「成功」： 「假」， 「錯誤」：255}

我嘗試了不同的jar的路徑值，我得到了相同的結果。我是否必須將我的罐子上傳到其他地方，或者我是否錯過了一些必要的步驟？

Answer 1

代碼中存在一個錯誤，已經在FIWARE Lab的全局實例中修復。

我測試過這一點：

$ curl -X POST "http://computing.cosmos.lab.fiware.org:12000/tidoop/v1/user/myuser/jobs" -d '{"jar":"mrjars/hadoop-mapreduce-examples.jar","class_name":"wordcount","lib_jars":"mrjars/hadoop-mapreduce-examples.jar","input":"testdir","output":"outputdir"}' -H "Content-Type: application/json" -H "X-Auth-Token: mytoken" 
{"success":"true","job_id": "job_1460639183882_0016"} 

請在這種情況下觀察，mrjars/hadoop-mapreduce-examples.jar是hdfs:///user/myuser下的相對路徑。在此操作中始終使用相對路徑。

$ curl -X GET "http://storage.cosmos.lab.fiware.org:14000/webhdfs/v1/user/myuser/mrjars?op=liststatus&user.name=myuser" -H "X-Auth-Token: mytoken" 
{"FileStatuses":{"FileStatus":[{"pathSuffix":"hadoop-mapreduce-examples.jar","type":"FILE","length":270283,"owner":"myuser","group":"myuser","permission":"644","accessTime":1464702215233,"modificationTime":1464702215479,"blockSize":134217728,"replication":3}]}} 

結果：

$ curl -X GET "http://storage.cosmos.lab.fiware.org:14000/webhdfs/v1/user/myuser/outputdir?op=liststatus&user.name=myuser" -H "X-Auth-Token: mytoken" 
{"FileStatuses":{"FileStatus":[{"pathSuffix":"_SUCCESS","type":"FILE","length":0,"owner":"myuser","group":"myuser","permission":"644","accessTime":1464706333691,"modificationTime":1464706333706,"blockSize":134217728,"replication":3},{"pathSuffix":"part-r-00000","type":"FILE","length":128,"owner":"myuser","group":"myuser","permission":"644","accessTime":1464706333264,"modificationTime":1464706333460,"blockSize":134217728,"replication":3}]}} 

我會上傳修復到宇宙中回購儘快GitHub的。