卡桑德拉火花連接器 - 的NoSuchMethodError：scala.runtime.ObjectRef.zero（）Lscala /運行/ ObjectRef

Question

I am trying to connect spark with cassandra database but I got the error mentioned below. I think there should some mismatch with the versions.

代碼：

SparkConf conf = new SparkConf().setAppName("kafka-sandbox").setMaster("local[2]"); 
    conf.set("spark.cassandra.connection.host", "192.168.34.1");//connection for cassandra database 
    JavaSparkContext sc = new JavaSparkContext(conf); 
    CassandraConnector connector = CassandraConnector.apply(sc.getConf()); 
    final Session session = connector.openSession();//error in this line 
    final PreparedStatement prepared = session.prepare("INSERT INTO spark_test5.messages JSON?"); 

---

error: 


    Exception in thread "main" java.lang.NoSuchMethodError: scala.runtime.ObjectRef.zero()Lscala/runtime/ObjectRef; 
     at com.datastax.spark.connector.cql.CassandraConnector$.com$datastax$spark$connector$cql$CassandraConnector$$createSession(CassandraConnector.scala) 
     at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$3.apply(CassandraConnector.scala:149) 
     at com.datastax.spark.connector.cql.CassandraConnector$$anonfun$3.apply(CassandraConnector.scala:149) 
     at com.datastax.spark.connector.cql.RefCountedCache.createNewValueAndKeys(RefCountedCache.scala:31) 
     at com.datastax.spark.connector.cql.RefCountedCache.acquire(RefCountedCache.scala:56) 
     at com.datastax.spark.connector.cql.CassandraConnector.openSession(CassandraConnector.scala:82) 

---

pom.xml: 

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 
    <modelVersion>4.0.0</modelVersion> 
    <groupId>SparkPoc</groupId> 
    <artifactId>Spark-Poc</artifactId> 
    <version>0.0.1-SNAPSHOT</version> 
    <dependencies> 
    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-streaming_2.10</artifactId> 
     <version>2.0.0</version> 
     <scope>provided</scope> 
    </dependency> 
    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-core_2.10</artifactId> 
     <version>2.0.1</version> 
    </dependency> 
    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-streaming-kafka-0-8_2.10</artifactId> 
     <version>2.0.0</version> 
    </dependency> 
    <dependency> 
     <groupId>com.datastax.spark</groupId> 
     <artifactId>spark-cassandra-connector_2.11</artifactId> 
     <version>2.0.0-M3</version> 
    </dependency> 
    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-sql_2.11</artifactId> 
     <version>2.0.1</version> 
    </dependency> 
    </dependencies> 
<build> 
    <plugins> 
    <plugin> 
     <groupId>org.apache.maven.plugins</groupId> 
     <artifactId>maven-compiler-plugin</artifactId> 
     <version>3.3</version> 
     <configuration> 
      <source>1.8</source> 
      <target>1.8</target> 
     </configuration> 
    </plugin> 
    <plugin> 
     <groupId>org.apache.maven.plugins</groupId> 
     <artifactId>maven-assembly-plugin</artifactId> 
     <version>2.4.1</version> 
     <configuration> 
      <!-- get all project dependencies --> 
      <descriptorRefs> 
        <descriptorRef>jar-with-dependencies</descriptorRef> 
      </descriptorRefs> 
      <!-- MainClass in mainfest make a executable jar --> 
      <archive> 
        <manifest> 
          <mainClass>com.nwf.Consumer</mainClass> 
        </manifest> 
      </archive> 
     </configuration> 
     <executions> 
      <execution> 
        <id>make-assembly</id> 
        <!-- bind to the packaging phase --> 
        <phase>package</phase> 
        <goals> 
          <goal>single</goal> 
        </goals> 
      </execution> 
    </executions> 
    </plugin> 
    </plugins> 
</build> 
</project> 

火花版本：版本2.0.0

Scala版本：版本2.11.8

Answer 1

根據您的pom.xml你混的Scala版本不同的依存關係：

• 火花streaming_ 2.10
• spark-core_ 2.10
• 火花流 - 卡夫卡0-8_ 2.10
• 火花卡桑德拉 - connector_ 2.11
• 火花SQL_ 2.11

所有的依賴應該有相同的階版本。請嘗試將所有內容更改爲_2.11。

Answer 2

scala.runtime.VolatileObjectRef上的零（）已經在Scala 2.11中引入了您可能有一個針對Scala 2.11編譯並在Scala 2.10運行時運行的庫。

見

2.10：https://github.com/scala/scala/blob/2.10.x/src/library/scala/runtime/VolatileObjectRef.java V2.11：https://github.com/scala/scala/blob/2.11.x/src/library/scala/runtime/VolatileObjectRef.java

Answer 3

In my pom.xml I changed sacala version from 2.10 to 2.11. 
Given below is the updated pom.xml 


---------- 
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 
    <modelVersion>4.0.0</modelVersion> 
    <groupId>SparkPoc</groupId> 
    <artifactId>Spark-Poc</artifactId> 
    <version>0.0.1-SNAPSHOT</version> 
    <dependencies> 
    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-streaming_2.11</artifactId> 
     <version>2.0.0</version> 
     <scope>provided</scope> 
    </dependency> 
    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-core_2.11</artifactId> 
     <version>2.0.1</version> 
    </dependency> 
    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-streaming-kafka-0-8_2.11</artifactId> 
     <version>2.0.0</version> 
    </dependency> 
    <dependency> 
     <groupId>com.datastax.spark</groupId> 
     <artifactId>spark-cassandra-connector_2.11</artifactId> 
     <version>2.0.0-M3</version> 
    </dependency> 
    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-sql_2.11</artifactId> 
     <version>2.0.1</version> 
    </dependency> 
    </dependencies> 
<build> 
    <plugins> 
    <plugin> 
     <groupId>org.apache.maven.plugins</groupId> 
     <artifactId>maven-compiler-plugin</artifactId> 
     <version>3.3</version> 
     <configuration> 
      <source>1.8</source> 
      <target>1.8</target> 
     </configuration> 
    </plugin> 
    <plugin> 
     <groupId>org.apache.maven.plugins</groupId> 
     <artifactId>maven-assembly-plugin</artifactId> 
     <version>2.4.1</version> 
     <configuration> 
      <!-- get all project dependencies --> 
      <descriptorRefs> 
        <descriptorRef>jar-with-dependencies</descriptorRef> 
      </descriptorRefs> 
      <!-- MainClass in mainfest make a executable jar --> 
      <archive> 
        <manifest> 
          <mainClass>com.nwf.Consumer</mainClass> 
        </manifest> 
      </archive> 
     </configuration> 
     <executions> 
      <execution> 
        <id>make-assembly</id> 
        <!-- bind to the packaging phase --> 
        <phase>package</phase> 
        <goals> 
          <goal>single</goal> 
        </goals> 
      </execution> 
    </executions> 
    </plugin> 
    </plugins> 
</build> 
</project>