我hadoop2.6.5 HA，當從mysql導入數據時，運行錯誤java.lang.StackOverflowError

-1

在我的Hadoop2.6.5 HA中，當我使用sqoop 1.4.6從mysql數據導入數據時，出現以下錯誤。我hadoop2.6.5 HA，當從mysql導入數據時，運行錯誤java.lang.StackOverflowError

OS：Centos6.5bit

有誰面臨同樣的proble？如果任何地方我想念它？

[[email protected] app]$ sqoop import --connect jdbc:mysql://localhost:3306/hive --username hive --password hive --table DBS --m 1 --target-dir /user/test3 
Warning: /home/hadoop/app/sqoop/../hcatalog does not exist! HCatalog jobs will fail. 
Please set $HCAT_HOME to the root of your HCatalog installation. 
Warning: /home/hadoop/app/sqoop/../accumulo does not exist! Accumulo imports will fail. 
Please set $ACCUMULO_HOME to the root of your Accumulo installation. 
17/09/01 16:54:11 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6 
17/09/01 16:54:11 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead. 
17/09/01 16:54:11 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 
17/09/01 16:54:11 INFO tool.CodeGenTool: Beginning code generation 
17/09/01 16:54:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `DBS` AS t LIMIT 1 
17/09/01 16:54:11 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `DBS` AS t LIMIT 1 
17/09/01 16:54:11 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /home/hadoop/app/hadoop 
Note: /tmp/sqoop-hadoop/compile/53650552d9b1969139bf57841c0c9aa1/DBS.java uses or overrides a deprecated API. 
Note: Recompile with -Xlint:deprecation for details. 
17/09/01 16:54:14 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/53650552d9b1969139bf57841c0c9aa1/DBS.jar 
17/09/01 16:54:14 WARN manager.MySQLManager: It looks like you are importing from mysql. 
17/09/01 16:54:14 WARN manager.MySQLManager: This transfer can be faster! Use the --direct 
17/09/01 16:54:14 WARN manager.MySQLManager: option to exercise a MySQL-specific fast path. 
17/09/01 16:54:14 INFO manager.MySQLManager: Setting zero DATETIME behavior to convertToNull (mysql) 
17/09/01 16:54:14 INFO mapreduce.ImportJobBase: Beginning import of DBS 
SLF4J: Class path contains multiple SLF4J bindings. 
SLF4J: Found binding in [jar:file:/home/hadoop/app/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
SLF4J: Found binding in [jar:file:/home/hadoop/app/hbase/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 
17/09/01 16:54:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
17/09/01 16:54:14 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar 
17/09/01 16:54:14 WARN fs.FileSystem: "cluster1" is a deprecated filesystem name. Use "hdfs://cluster1/" instead. 
17/09/01 16:54:20 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps 
17/09/01 16:54:20 WARN fs.FileSystem: "cluster1" is a deprecated filesystem name. Use "hdfs://cluster1/" instead. 
Exception in thread "main" java.lang.StackOverflowError 
     at org.apache.commons.collections.map.AbstractMapDecorator.containsKey(AbstractMapDecorator.java:83) 
     at org.apache.hadoop.conf.Configuration.isDeprecated(Configuration.java:558) 
     at org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:605) 
     at org.apache.hadoop.conf.Configuration.get(Configuration.java:1185) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:470) 
     at org.apache.hadoop.fs.FileContext.getFileContext(FileContext.java:444) 
[[email protected] app]$

來源

2017-09-01 jiandanjie.com

的StackOverFlowError發生因爲Java調用堆棧的增長並沒有在運行的JVM，以容納更多的細節調用堆棧更多的內存。

請仔細參閱Java錯誤堆棧和注意 - 有以下兩種功能

public static FileContext getFileContext(final URI defaultFsUri, final Configuration aConf) 
public static FileContext getFileContext(final Configuration aConf)

在一定條件下1 getFileContext調用第二getFileContext和第二getFileContext調用第一getFileContext。這意味着它的遞歸調用。

對於您的情況，core-site.xml必須包含屬性fs.defaultFS且值cluster1。相反，它應該是hdfs://cluster1。如果您將fs.defaultFS設置爲正確的URL hdfs://cluster1，遞歸調用將會消失。

來源

2017-09-01 11:49:44 Shubhangi

謝謝，當我改變fs.defaultFS與價值hdfs：// cluster1，問題已經解決 –

@jiandanjire很高興它的工作。請接受答案。 – Shubhangi

在你的情況下，core-site.xml必須包含值爲cluster1的屬性fs.defaultFS。相反，它應該是hdfs：// cluster1。如果將fs.defaultFS設置爲正確的URL hdfs：// cluster1，則遞歸調用將消失。

來源

2017-09-08 02:03:32

我hadoop2.6.5 HA，當從mysql導入數據時，運行錯誤java.lang.StackOverflowError

回答

相關問題