阿帕奇火花「Py4JError：從Java方面的答案是空的」

Question

每次我得到這個錯誤... 我用蘇打水...... 我的conf文件：

***"spark.driver.memory 65g 
spark.python.worker.memory 65g 
spark.master local[*]"*** 

的數據量是約5 Gb。 有沒有關於這個錯誤的其他信息... 有誰知道它爲什麼會發生？謝謝！

***"ERROR:py4j.java_gateway:Error while sending or receiving. 
Traceback (most recent call last): 
    File "/data/analytics/Spark1.6.1/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 746, in send_command 
    raise Py4JError("Answer from Java side is empty") 
Py4JError: Answer from Java side is empty 
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server 
Traceback (most recent call last): 
    File "/data/analytics/Spark1.6.1/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 690, in start 
    self.socket.connect((self.address, self.port)) 
    File "/usr/local/anaconda/lib/python2.7/socket.py", line 228, in meth 
    return getattr(self._sock,name)(*args) 
error: [Errno 111] Connection refused 
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server 
Traceback (most recent call last): 
    File "/data/analytics/Spark1.6.1/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 690, in start 
    self.socket.connect((self.address, self.port)) 
    File "/usr/local/anaconda/lib/python2.7/socket.py", line 228, in meth 
    return getattr(self._sock,name)(*args) 
error: [Errno 111] Connection refused 
ERROR:py4j.java_gateway:An error occurred while trying to connect to the Java server 
Traceback (most recent call last): 
    File "/data/analytics/Spark1.6.1/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py", line 690, in start 
    self.socket.connect((self.address, self.port)) 
    File "/usr/local/anaconda/lib/python2.7/socket.py", line 228, in meth 
    return getattr(self._sock,name)(*args) 
error: [Errno 111] Connection refused"*** 

Answer 1

你有沒有嘗試設置你的星火配置文件spark.executor.memory和spark.driver.memory？

有關更多信息，請參見https://stackoverflow.com/a/22742982/5453184。