2016-07-27 73 views
3

我剛剛在Ubuntu主機上使用「sbt assembly」構建了spark 2.0。 一切都完成很好,但是,當我試圖提交pyspark工作:如何從源代碼正確構建spark 2.0,以包含pyspark?

bin/spark-submit --master spark://localhost:7077 examples/src/main/python/pi.py 1000 

我得到這個錯誤:

Failed to find Spark jars directory (/home/ubuntu/spark/spark-2.0.0/assembly/target/scala-2.10/jars). 
You need to build Spark with the target "package" before running this program. 

我應該以重新構建火花2.0包括pyspark做?

回答

6

嘗試:

  1. Install sbt

  2. 體形:

    https://github.com/apache/spark.git 
    cd spark 
    git checkout v2.0.0 
    sbt package