2017-05-29 79 views
-1

在本地系統中得到錯誤程序運行時可作爲 我的內存大小爲3GB,需要解決星火對象運行時錯誤

Exception in thread "main" java.lang.IllegalArgumentException: System memory 259522560 must be at least 471859200. Please increase heap size using the --driver-memory option or spark.driver.memory in Spark configuration. 
    at org.apache.spark.memory.UnifiedMemoryManager$.getMaxMemory(UnifiedMemoryManager.scala:216) 
    at org.apache.spark.memory.UnifiedMemoryManager$.apply(UnifiedMemoryManager.scala:198) 
    at org.apache.spark.SparkEnv$.create(SparkEnv.scala:330) 
    at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:174) 
    at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:257) 
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:432) 
    at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2313) 
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868) 
    at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860) 
    at scala.Option.getOrElse(Option.scala:121) 
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860) 
    at SparkCore.cartesianTransformation$.main(cartesianTransformation.scala:11) 
    at SparkCore.cartesianTransformation.main(cartesianTransformation.scala) 
+0

如果您不發送任何代碼,我們如何幫助您? –

回答

0

看來你的火花驅動程序在小內存中運行的嘗試增加的大小驅動器內存。 您可以使用--driver-memory 4g向驅動程序提供內存大小。

希望這會有所幫助!

+0

仍然是相同的錯誤 – surya

+0

你能看到在Web UI有多少內存已被使用? –

+0

並嘗試在VM參數中添加-Xmx1024m -Xms512m –