2015-08-08 78 views
2

我的namenode未啓動。Namenode未啓動 - namenode加入中的異常

在嘗試重新啓動之前嘗試格式化並刪除tmp目錄。但它沒有出現。

目前我正在嘗試爲兩節點羣集。我從單個節點機器克隆了兩個節點。並將屬性更改爲類似於名稱節點,作業跟蹤器和輔助名稱節點的屬性。剩下的就是另一個。

試圖啓動名稱節點。我在日誌中得到以下異常。嘗試搜索,但沒有找到任何具體的問題。另外我已經設置了較少的ssh密碼,以防因任何權限被拒絕。

2015-08-08 12:40:59,005 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: 
/************************************************************ 
STARTUP_MSG: Starting NameNode 
STARTUP_MSG: host = HNNAME/192.168.136.170 
STARTUP_MSG: args = [] 
STARTUP_MSG: version = 2.0.0-cdh4.7.0 
STARTUP_MSG: classpath = /etc/hadoop/conf:/usr/lib/hadoop/lib/mockito-all-1.8.5.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/zookeeper-3.4.5-cdh4.7.0.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/avro-1.7.4.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/cloudera-jets3t-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop/lib/log4j-1.2.17.jar:/usr/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/xz-1.0.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/junit-4.8.2.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/.//parquet-column-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//parquet-hive-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//parquet-column-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-scrooge-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//hadoop-common-2.0.0-cdh4.7.0-tests.jar:/usr/lib/hadoop/.//parquet-hadoop-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//parquet-generator-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//hadoop-auth.jar:/usr/lib/hadoop/.//parquet-pig-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//hadoop-annotations-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop/.//hadoop-common-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-generator-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//parquet-avro-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-cascading-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//parquet-thrift-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//parquet-thrift-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//parquet-generator-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-cascading-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-common-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//parquet-avro-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//parquet-avro-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//parquet-cascading-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//parquet-encoding-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-pig-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//parquet-thrift-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-common-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//parquet-hive-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//hadoop-auth-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-pig-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-encoding-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//parquet-hive-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//parquet-format-1.0.0-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-common-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-format-1.0.0-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//hadoop-annotations.jar:/usr/lib/hadoop/.//parquet-test-hadoop2-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-hadoop-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//parquet-encoding-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//hadoop-common.jar:/usr/lib/hadoop/.//parquet-hadoop-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-scrooge-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-column-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//parquet-scrooge-1.2.5-cdh4.7.0-javadoc.jar:/usr/lib/hadoop/.//parquet-pig-bundle-1.2.5-cdh4.7.0-sources.jar:/usr/lib/hadoop/.//parquet-pig-bundle-1.2.5-cdh4.7.0.jar:/usr/lib/hadoop/.//parquet-format-1.0.0-cdh4.7.0-javadoc.jar:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/usr/lib/hadoop-hdfs/lib/asm-3.2.jar:/usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar:/usr/lib/hadoop-hdfs/lib/jersey-core-1.8.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh4.7.0.jar:/usr/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/usr/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/usr/lib/hadoop-hdfs/lib/commons-lang-2.5.jar:/usr/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/usr/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/usr/lib/hadoop-hdfs/lib/commons-io-2.1.jar:/usr/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/usr/lib/hadoop-hdfs/lib/commons-codec-1.4.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.0.0-cdh4.7.0-tests.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs.jar:/usr/lib/hadoop-yarn/lib/netty-3.2.4.Final.jar:/usr/lib/hadoop-yarn/lib/paranamer-2.3.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.8.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.8.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/avro-1.7.4.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.8.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.17.jar:/usr/lib/hadoop-yarn/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-yarn/lib/xz-1.0.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.1.jar:/usr/lib/hadoop-yarn/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests-2.0.0-cdh4.7.0-tests.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-client-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-unmanaged-am-launcher.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-site.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-client.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-tests.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-site-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-mapreduce/lib/netty-3.2.4.Final.jar:/usr/lib/hadoop-mapreduce/lib/paranamer-2.3.jar:/usr/lib/hadoop-mapreduce/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-mapreduce/lib/javax.inject-1.jar:/usr/lib/hadoop-mapreduce/lib/guice-3.0.jar:/usr/lib/hadoop-mapreduce/lib/aopalliance-1.0.jar:/usr/lib/hadoop-mapreduce/lib/asm-3.2.jar:/usr/lib/hadoop-mapreduce/lib/jersey-core-1.8.jar:/usr/lib/hadoop-mapreduce/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/jersey-guice-1.8.jar:/usr/lib/hadoop-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-mapreduce/lib/avro-1.7.4.jar:/usr/lib/hadoop-mapreduce/lib/jersey-server-1.8.jar:/usr/lib/hadoop-mapreduce/lib/log4j-1.2.17.jar:/usr/lib/hadoop-mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/lib/hadoop-mapreduce/lib/xz-1.0.jar:/usr/lib/hadoop-mapreduce/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-mapreduce/lib/commons-io-2.1.jar:/usr/lib/hadoop-mapreduce/lib/commons-compress-1.4.1.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples.jar:/usr/lib/hadoop-mapreduce/.//hadoop-archives-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-datajoin.jar:/usr/lib/hadoop-mapreduce/.//hadoop-distcp-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient.jar:/usr/lib/hadoop-mapreduce/.//hadoop-streaming-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-shuffle-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-rumen-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins.jar:/usr/lib/hadoop-mapreduce/.//hadoop-extras-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-datajoin-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-rumen.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-app.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common.jar:/usr/lib/hadoop-mapreduce/.//hadoop-streaming.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-gridmix-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-extras.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-jobclient-2.0.0-cdh4.7.0-tests.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-common-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-archives.jar:/usr/lib/hadoop-mapreduce/.//hadoop-distcp.jar:/usr/lib/hadoop-mapreduce/.//hadoop-gridmix.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-examples-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-hs-plugins-2.0.0-cdh4.7.0.jar:/usr/lib/hadoop-mapreduce/.//hadoop-mapreduce-client-core-2.0.0-cdh4.7.0.jar 
STARTUP_MSG: build = git://centos32-6-slave.sf.cloudera.com/data/1/jenkins/workspace/generic-package-centos32-6/topdir/BUILD/hadoop-2.0.0-cdh4.7.0/src/hadoop-common-project/hadoop-common -r 8e266e052e423af592871e2dfe09d54c03f6a0e8; compiled by 'jenkins' on Wed May 28 10:12:25 PDT 2014 
STARTUP_MSG: java = 1.6.0_45 
************************************************************/ 
2015-08-08 12:40:59,010 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] 
2015-08-08 12:40:59,576 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 
2015-08-08 12:40:59,718 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 
2015-08-08 12:40:59,718 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system started 
2015-08-08 12:41:00,059 WARN org.apache.hadoop.hdfs.server.common.Util: Path /storage/name should be specified as a URI in configuration files. Please update hdfs configuration. 
2015-08-08 12:41:00,060 WARN org.apache.hadoop.hdfs.server.common.Util: Path /storage/name should be specified as a URI in configuration files. Please update hdfs configuration. 
2015-08-08 12:41:00,061 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of dataloss due to lack of redundant storage directories! 
2015-08-08 12:41:00,061 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one namespace edits storage directory (dfs.namenode.edits.dir) configured. Beware of dataloss due to lack of redundant storage directories! 
2015-08-08 12:41:00,069 WARN org.apache.hadoop.hdfs.server.common.Util: Path /storage/name should be specified as a URI in configuration files. Please update hdfs configuration. 
2015-08-08 12:41:00,069 WARN org.apache.hadoop.hdfs.server.common.Util: Path /storage/name should be specified as a URI in configuration files. Please update hdfs configuration. 
2015-08-08 12:41:00,101 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsLock is fair:true 
2015-08-08 12:41:00,165 INFO org.apache.hadoop.hdfs.server.blockmanagement.HeartbeatManager: Setting heartbeat recheck interval to 30000 since dfs.namenode.stale.datanode.interval is less than dfs.namenode.heartbeat.recheck-interval 
2015-08-08 12:41:00,180 INFO org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000 
2015-08-08 12:41:00,187 INFO org.apache.hadoop.util.GSet: Computing capacity for map BlocksMap 
2015-08-08 12:41:00,187 INFO org.apache.hadoop.util.GSet: VM type  = 32-bit 
2015-08-08 12:41:00,196 INFO org.apache.hadoop.util.GSet: 2.0% max memory 966.7 MB = 19.3 MB 
2015-08-08 12:41:00,196 INFO org.apache.hadoop.util.GSet: capacity  = 2^22 = 4194304 entries 
2015-08-08 12:41:01,099 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: dfs.block.access.token.enable=false 
2015-08-08 12:41:01,100 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: defaultReplication   = 1 
2015-08-08 12:41:01,100 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplication    = 512 
2015-08-08 12:41:01,100 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: minReplication    = 1 
2015-08-08 12:41:01,100 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxReplicationStreams  = 2 
2015-08-08 12:41:01,100 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: shouldCheckForEnoughRacks = false 
2015-08-08 12:41:01,100 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: replicationRecheckInterval = 3000 
2015-08-08 12:41:01,100 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: encryptDataTransfer  = false 
2015-08-08 12:41:01,100 INFO org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: maxNumBlocksToLog   = 1000 
2015-08-08 12:41:01,110 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner    = hdfs (auth:SIMPLE) 
2015-08-08 12:41:01,110 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup   = supergroup 
2015-08-08 12:41:01,111 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled = false 
2015-08-08 12:41:01,111 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: HA Enabled: false 
2015-08-08 12:41:01,115 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Append Enabled: true 
2015-08-08 12:41:01,547 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occuring more than 10 times 
2015-08-08 12:41:01,549 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033 
2015-08-08 12:41:01,549 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0 
2015-08-08 12:41:01,549 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.namenode.safemode.extension  = 30000 
2015-08-08 12:41:01,562 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /storage/name/in_use.lock acquired by nodename [email protected] 
2015-08-08 12:41:01,640 INFO org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Recovering unfinalized segments in /storage/name/current 
2015-08-08 12:41:01,772 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Loading image file /storage/name/current/fsimage_0000000000000038306 using no compression 
2015-08-08 12:41:01,772 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Number of files = 4012 
2015-08-08 12:41:01,932 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Number of files under construction = 1 
2015-08-08 12:41:01,941 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Image file of size 343797 loaded in 0 seconds. 
2015-08-08 12:41:01,941 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Loaded image for txid 38306 from /storage/name/current/fsimage_0000000000000038306 
2015-08-08 12:41:01,944 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Reading org[email protected]c623af expecting start txid #38307 
2015-08-08 12:41:01,965 INFO org.apache.hadoop.hdfs.server.namenode.EditLogInputStream: Fast-forwarding stream '/storage/name/current/edits_0000000000000038307-0000000000000038308' to transaction ID 38307 
2015-08-08 12:41:01,985 INFO org.apache.hadoop.hdfs.server.namenode.FSImage: Edits file /storage/name/current/edits_0000000000000038307-0000000000000038308 of size 30 edits # 2 loaded in 0 seconds 
2015-08-08 12:41:02,045 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Starting log segment at 38309 
2015-08-08 12:41:02,154 WARN org.apache.hadoop.hdfs.server.namenode.FileJournalManager: Unable to start log segment 38309 at /storage/name/current/edits_inprogress_0000000000000038309: /storage/name/current/edits_inprogress_0000000000000038309 (Permission denied) 
2015-08-08 12:41:02,154 ERROR org.apache.hadoop.hdfs.server.namenode.NNStorage: Error reported on storage directory Storage Directory /storage/name 
2015-08-08 12:41:02,154 WARN org.apache.hadoop.hdfs.server.namenode.NNStorage: About to remove corresponding storage: /storage/name 
2015-08-08 12:41:02,155 ERROR org.apache.hadoop.hdfs.server.namenode.FSEditLog: Error: starting log segment 38309 failed for (journal JournalAndStream(mgr=FileJournalManager(root=/storage/name), stream=null)) 
java.io.FileNotFoundException: /storage/name/current/edits_inprogress_0000000000000038309 (Permission denied) 
    at java.io.RandomAccessFile.open(Native Method) 
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:216) 
    at org.apache.hadoop.hdfs.server.namenode.EditLogFileOutputStream.<init>(EditLogFileOutputStream.java:74) 
    at org.apache.hadoop.hdfs.server.namenode.FileJournalManager.startLogSegment(FileJournalManager.java:105) 
    at org.apache.hadoop.hdfs.server.namenode.JournalSet$JournalAndStream.startLogSegment(JournalSet.java:89) 
    at org.apache.hadoop.hdfs.server.namenode.JournalSet$2.apply(JournalSet.java:197) 
    at org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:347) 
    at org.apache.hadoop.hdfs.server.namenode.JournalSet.startLogSegment(JournalSet.java:194) 
    at org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegment(FSEditLog.java:923) 
    at org.apache.hadoop.hdfs.server.namenode.FSEditLog.openForWrite(FSEditLog.java:264) 
    at org.apache.hadoop.hdfs.server.namenode.FSImage.openEditLogForWrite(FSImage.java:574) 
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:747) 
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:531) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:403) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:445) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:621) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:606) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1177) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1241) 
2015-08-08 12:41:02,156 ERROR org.apache.hadoop.hdfs.server.namenode.FSEditLog: Disabling journal JournalAndStream(mgr=FileJournalManager(root=/storage/name), stream=null) 
2015-08-08 12:41:02,156 ERROR org.apache.hadoop.hdfs.server.namenode.FSEditLog: Error: starting log segment 38309 failed for too many journals 
2015-08-08 12:41:02,157 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping NameNode metrics system... 
2015-08-08 12:41:02,157 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system stopped. 
2015-08-08 12:41:02,158 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system shutdown complete. 
2015-08-08 12:41:02,158 FATAL org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode join 
java.io.IOException: Unable to start log segment 38309: too few journals successfully started. 
    at org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegment(FSEditLog.java:925) 
    at org.apache.hadoop.hdfs.server.namenode.FSEditLog.openForWrite(FSEditLog.java:264) 
    at org.apache.hadoop.hdfs.server.namenode.FSImage.openEditLogForWrite(FSImage.java:574) 
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:747) 
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:531) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:403) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:445) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:621) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:606) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1177) 
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1241) 
Caused by: java.io.IOException: starting log segment 38309 failed for too many journals 
    at org.apache.hadoop.hdfs.server.namenode.JournalSet.mapJournalsAndReportErrors(JournalSet.java:374) 
    at org.apache.hadoop.hdfs.server.namenode.JournalSet.startLogSegment(JournalSet.java:194) 
    at org.apache.hadoop.hdfs.server.namenode.FSEditLog.startLogSegment(FSEditLog.java:923) 
    ... 10 more 
2015-08-08 12:41:02,159 INFO org.apache.hadoop.util.ExitUtil: Exiting with status 1 
2015-08-08 12:41:02,160 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG: 
/************************************************************ 
SHUTDOWN_MSG: Shutting down NameNode at HNNAME/192.168.136.170 
************************************************************/ 
+0

運行namenode的用戶是否擁有對tmp文件夾的寫入權限?你還可以發佈你的配置文件嗎? – mzzzzb

+0

是的。我正在使用root,它對tmp drwxrwxrwx具有權限。 4個HDFS 4096的Hadoop 08月08 13時13 TMP –

+0

CORE-SITE.XML <結構> hadoop.tmp.dir /usr/lib中/ hadoop的/ TMP fs.default.name HDFS:// HNNAME:8020

回答

1

我想你沒有正確給予namenode元數據位置的權限。爲確保其正常工作,請按照以下步驟驗證您的程序。

假設的Namenode元數據位置:/storage/name

  • mkdir -p /storage/name
  • chown -R hdfs:hadoop /storage/name
  • sudo -u hdfs hadoop namenode -format
  • service hdfs-namenode start(假設CDH rpm安裝時,使用該變化與您所使用的安裝方法。)

Hadoop守護進程將以hdfs用戶身份啓動,並且如果元數據位置權限未設置爲hadoop用戶和hadoop超級用戶組,則會出現上述錯誤。

如果您觀察上面生成的日誌,FileSystem所有者:fsOwner爲hdfs,超級組爲supergroup。異常是FileNotFound,因爲負責啓動namenode的服務無法訪問文件系統,因爲它沒有所需的權限。

+0

只是5-6個小時的尋找錯誤在找到你的答案之前:)非常感謝!理解所有這些hadoop/cloudera/hdfs的東西還有很多事情要做,但是你讓我更進一步:) – DenisFLASH