作業從RUNNING跳轉到PREP狀態

Question

當我運行mapreduce作業時，它從RUNNING跳轉到PREP狀態。我查看了mapreduce日誌，但沒有發現任何異常。我想知道這是否是與紗線配置有關的問題。所以，我查看了mapred-site.xml [2]的配置，看起來內存大小是正確的。儘管我已將mapreduce設置爲32GB（<name>yarn.nodemanager.resource.memory-mb</name> <value>32218</value>），但我正在運行16核和64GB內存的PC。任何建議嘗試調試這個？

[1]工作狀態

Total jobs:1 
        JobId  State   StartTime  UserName   Queue  Priority  UsedContainers RsvdContainers UsedMem   RsvdMem  NeededMem   AM info 
job_1379101056979_0001  PREP  1379101096477   root   default  NORMAL     0    0  0M    0M 

[2] mapred-site.xml

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> 
<configuration> 
<property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> 
<property> <name>mapreduce.jobhistory.done-dir</name> <value>/root/Programs/hadoop/logs/history/done</value> </property> 
<property> <name>mapreduce.jobhistory.intermediate-done-dir</name> <value>/root/Programs/hadoop/logs/history/intermediate-done-dir</value> </property> 
<property> <name>mapreduce.job.reduces</name> <value>4</value> </property> 

<!-- property> <name>yarn.nodemanager.resource.memory-mb</name> <value>8240</value> </property --> 
<property> <name>yarn.nodemanager.resource.memory-mb</name> <value>24240</value> </property> 
<property> <name>yarn.scheduler.minimum-allocation-mb</name> <value>1024</value> </property> 

<!-- property><name>mapreduce.task.files.preserve.failedtasks</name><value>true</value></property> 
<property><name>mapreduce.task.files.preserve.filepattern</name><value>*</value></property --> 

</configuration> 

我不知道發生了什麼事到這一點，所以我張貼在這裏的部分作業的日誌。我注意到作業正在運行的容器獲得了CONTAINER_STOP信號。任何人都可以幫助我發生什麼事情？

2016-10-17 09:57:23,233 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Stopping container with container Id: container_1476697963637_0001_01_000022 
2016-10-17 09:57:23,233 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=ubuntu  IP=172.30.0.231 OPERATION=Stop Container Request  TARGET=ContainerManageImpl  RESULT=SUCCESS APPID=application_1476697963637_0001 CONTAINERID=container_1476697963637_0001_01_000022 
2016-10-17 09:57:23,263 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1476697963637_0001_01_000020 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 
2016-10-17 09:57:23,263 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1476697963637_0001_01_000022 transitioned from RUNNING to KILLING 
2016-10-17 09:57:23,321 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up container container_1476697963637_0001_01_000022 
2016-10-17 09:57:23,341 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /home/ubuntu/tmp/hadoop-temp/nm-local-dir/usercache/ubuntu/appcache/application_1476697963637_0001/container_1476697963637_0001_01_000020 
2016-10-17 09:57:23,404 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 27978 for container-id container_1476697963637_0001_01_000042: 263.0 MB of 1 GB physical memory used; 1.8 GB of 2.1 GB virtual memory used 
2016-10-17 09:57:23,559 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=ubuntu  OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1476697963637_0001 CONTAINERID=container_1476697963637_0001_01_000020 
2016-10-17 09:57:23,559 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1476697963637_0001_01_000020 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 
2016-10-17 09:57:23,559 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Removing container_1476697963637_0001_01_000020 from application application_1476697963637_0001 
2016-10-17 09:57:23,559 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Considering container container_1476697963637_0001_01_000020 for log-aggregation 
2016-10-17 09:57:23,559 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1476697963637_0001 
2016-10-17 09:57:23,570 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code from container container_1476697963637_0001_01_000022 is : 143 
2016-10-17 09:57:23,571 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1476697963637_0001_01_000022 transitioned from KILLING to CONTAINER_CLEANEDUP_AFTER_KILL 
2016-10-17 09:57:23,571 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /home/ubuntu/tmp/hadoop-temp/nm-local-dir/usercache/ubuntu/appcache/application_1476697963637_0001/container_1476697963637_0001_01_000022 
2016-10-17 09:57:23,572 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=ubuntu  OPERATION=Container Finished - Killed TARGET=ContainerImpl RESULT=SUCCESS APPID=application_1476697963637_0001 CONTAINERID=container_1476697963637_0001_01_000022 
2016-10-17 09:57:23,572 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1476697963637_0001_01_000022 transitioned from CONTAINER_CLEANEDUP_AFTER_KILL to DONE 
2016-10-17 09:57:23,572 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Removing container_1476697963637_0001_01_000022 from application application_1476697963637_0001 
2016-10-17 09:57:23,572 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Considering container container_1476697963637_0001_01_000022 for log-aggregation 
2016-10-17 09:57:23,572 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1476697963637_0001 
2016-10-17 09:57:23,670 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 27820 for container-id container_1476697963637_0001_01_000040: 266.3 MB of 1 GB physical memory used; 1.8 GB of 2.1 GB virtual memory used 

Answer 1

我有這個問題;重新開始cloudera和紗線解決它。

如果重新啓動不起作用，請嘗試檢查job.properties中的端口 - 端口namenode和jobtracker可能有問題。確保您的jobtracker端口在job.properties文件中正確。

另請檢查map-reduce羣集插槽。它可能會用完插槽。