Maven pom.xml錯誤，同時設置Apache的火花

Question

我想設置Apache流。在下面的pom.xml中。當我加入

<dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-core_2.10</artifactId> 
     <version>2.1.0</version> 
    </dependency> 

POM文件越來越錯誤，他說：在這行找到多個註釋： - 故障轉移org.glassfish.hk2：HK2-utils的：罈子：2.4.0-B34

http://maven.apache.org/xsd/maven-4.0.0.xsd「> 4.0.0 com.iot.app.kafka IOT-卡夫卡生產者 1.0.0 的IoT卡夫卡生產者

<dependencies> 
    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-core_2.10</artifactId> 
     <version>2.1.0</version> 
    </dependency> 


    <dependency> 
     <groupId>org.apache.kafka</groupId> 
     <artifactId>kafka-clients</artifactId> 
     <version>0.9.0.0</version> 
    </dependency> 

    <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka_2.10 --> 
    <dependency> 
     <groupId>org.apache.spark</groupId> 
     <artifactId>spark-streaming-kafka_2.10</artifactId> 
     <version>1.6.3</version> 
    </dependency> 


    <dependency> 
     <groupId>org.spark-project.spark</groupId> 
     <artifactId>unused</artifactId> 
     <version>1.0.0</version> 
     <scope>provided</scope> 
    </dependency> 

    <dependency> 
     <groupId>com.fasterxml.jackson.core</groupId> 
     <artifactId>jackson-core</artifactId> 
     <version>2.6.6</version> 
    </dependency> 
    <dependency> 
     <groupId>com.fasterxml.jackson.core</groupId> 
     <artifactId>jackson-databind</artifactId> 
     <version>2.6.6</version> 
    </dependency> 
    <dependency> 
     <groupId>com.fasterxml.jackson.core</groupId> 
     <artifactId>jackson-annotations</artifactId> 
     <version>2.6.6</version> 
    </dependency> 
    <dependency> 
     <groupId>log4j</groupId> 
     <artifactId>log4j</artifactId> 
     <version>1.2.17</version> 
    </dependency> 
    <dependency> 
     <groupId>junit</groupId> 
     <artifactId>junit</artifactId> 
     <version>4.12</version> 
    </dependency> 
    <dependency> 
     <groupId>com.google.guava</groupId> 
     <artifactId>guava</artifactId> 
     <version>19.0</version> 
    </dependency> 
</dependencies> 

請幫助我的人在adavnce

Answer 1

您需要使用的spark-core和spark-streaming-kafka相同版本解決這個issue.Thans。請嘗試將spark-streaming-kafka依賴項更改爲：

<dependency> 
    <groupId>org.apache.spark</groupId> 
    <artifactId>spark-streaming-kafka-0-8_2.10</artifactId> 
    <version>2.1.0</version> 
</dependency>