1. 首頁
  2. 問答
  3. 從Hue運行Sqoop作業時出錯

從Hue運行Sqoop作業時出錯

2014-12-19 143 views
1

我正在嘗試從Hue創建Sqoop作業;在「作業設計器」下，我創建了一個新作業輸入teh命令來創建sqoop作業，如下所示。從Hue運行Sqoop作業時出錯

sqoop job --create TestHiveImport -- import --connect jdbc:oracle:thin:@<ipaddress>:port/<database_name> --username <username> --password <password> --table TEST_SQOOP --hive-import --hive-table trade.test_sqoop --hive-drop-import-delims --incremental lastmodified --check-column LAST_MODIFIED --last-value "1970-01-01 00:01:01.0" -m 1

我提交的作業後，我正在低於錯誤。請幫助，我完全卡住了。

2014-12-19 09:19:40,928 INFO ActionStartXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]:start:] Start action [[email protected]:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2014-12-19 09:19:40,928 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action status=DONE 
2014-12-19 09:19:40,928 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action updated in DB! 
2014-12-19 09:19:40,950 INFO ActionEndXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]:start:] end executor for wf action 0000002-141008154216525-oozie-oozi-W with wf job 0000002-141008154216525-oozie-oozi-W 
2014-12-19 09:19:41,018 INFO ActionStartXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2014-12-19 09:19:41,179 WARN SqoopActionExecutor:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] credentials is null for the action 
2014-12-19 09:19:41,495 INFO SqoopActionExecutor:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Trying to get job [job_1415818455534_0502], attempt [1] 
2014-12-19 09:19:41,507 INFO SqoopActionExecutor:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] checking action, external ID [job_1415818455534_0502] status [RUNNING] 
2014-12-19 09:19:41,508 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action status=RUNNING 
2014-12-19 09:19:41,509 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action updated in DB! 
2014-12-19 09:19:52,651 INFO CallbackServlet:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] callback for action [[email protected]] 
2014-12-19 09:19:52,714 INFO SqoopActionExecutor:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Trying to get job [job_1415818455534_0502], attempt [1] 
2014-12-19 09:19:52,714 INFO SqoopActionExecutor:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Trying to get job [job_1415818455534_0502], attempt [1] 
2014-12-19 09:19:52,778 INFO SqoopActionExecutor:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] action completed, external ID [null] 
2014-12-19 09:19:52,784 WARN SqoopActionExecutor:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1] 
2014-12-19 09:19:52,808 INFO ActionEndXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] end executor for wf action 0000002-141008154216525-oozie-oozi-W with wf job 0000002-141008154216525-oozie-oozi-W 
2014-12-19 09:19:52,832 INFO ActionEndXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] ERROR is considered as FAILED for SLA 
2014-12-19 09:19:52,881 INFO ActionStartXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2014-12-19 09:19:52,882 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action status=DONE 
2014-12-19 09:19:52,882 WARN ActionStartXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] [***[email protected]***]Action updated in DB! 
2014-12-19 09:19:52,904 INFO ActionEndXCommand:539 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[[email protected]] end executor for wf action 0000002-141008154216525-oozie-oozi-W with wf job 0000002-141008154216525-oozie-oozi-W 
2014-12-19 09:19:52,973 WARN CoordActionUpdateXCommand:542 - USER[scbengr2] GROUP[-] TOKEN[] APP[TestSqoop] JOB[0000002-141008154216525-oozie-oozi-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100

來源

2014-12-19 user2918922

+0

它看起來像一個權限問題給我。如果你確定你的sqoop命令在bash中有效。那麼你可能想要檢查你的用戶帳戶'scbengr2'是否在寫入文件夾上有正確的權限。另外，確保您的工作流程可以訪問連接器和hive-conf.xml文件。 – 2014-12-26 20:57:26

回答

0

色相目前推出的3 ways Sqoop：

  1. 職業，設計師
  2. Sqoop應用
  3. Oozie的應用

作業設計在內部使用了Oozie，因此輸出你往上看。

我看到錯誤的命令的幾件事情你上面提供：

  1. 該命令創建一個工作，而不是運行一個。
  2. 該命令包括sqoop，這將無法正常工作。
  3. 通過oozie創建作業需要external metastore，並且該作業以某種方式具有此配置。

解決方案：

  1. 除了使用metastore的，儘量只運行sqoop命令：import --connect jdbc:oracle:thin:@<ipaddress>:port/<database_name> --username <username> --password <password> --table TEST_SQOOP --hive-import --hive-table trade.test_sqoop --hive-drop-import-delims --incremental lastmodified --check-column LAST_MODIFIED --last-value "1970-01-01 00:01:01.0" -m 1。
  2. 而不是在命令之前添加sqoop，請嘗試刪除它：job --create TestHiveImport -- import --connect jdbc:oracle:thin:@<ipaddress>:port/<database_name> --username <username> --password <password> --table TEST_SQOOP --hive-import --hive-table trade.test_sqoop --hive-drop-import-delims --incremental lastmodified --check-column LAST_MODIFIED --last-value "1970-01-01 00:01:01.0" -m 1。
  3. 您可以將Sqoop Metastore信息添加到sqoop-site.xml。請諮詢mailing lists尋求幫助。

來源

2014-12-28 18:35:21 abeaamase

相關問題

 相關問題