Avro反序列化錯誤 - ArrayIndexOutOfBoundsException

Question

我的工作流程是 - 我創建avsc文件，使用avrogencpp工具生成C++類並在我的C++應用程序中創建avro二進制編碼數據。

我想了解爲什麼方案2不起作用。

方案1

test.avsc

{ 
"namespace": "com.company.project", 
"name": "Component_DeviceInfo", 
"type": "record", 
"doc": "Identifies a client device", 
"fields": [ 
    { 
     "name": "deviceId", 
     "type": [ 
      "null", 
      "string" 
     ], 
     "default": null, 
     "doc": "Multicast Data Client Device Id. Usually unique MAC address" 
    }, 
    { 
     "name": "zoneId", 
     "type": [ 
      "null", 
      "string" 
     ], 
     "default": null, 
     "doc": "Zone id where device belongs to" 
    } 
] 
} 

編碼器 - C++

Component_DeviceInfo deviceInfo; 
    deviceInfo.deviceId.set_string("device1"); 
    deviceInfo.zoneId.set_string("zone1"); 
    std::vector <char>tele_bytes_; 
    std::auto_ptr<avro::OutputStream> out = avro::memoryOutputStream(1); 
    avro::EncoderPtr enc = avro::binaryEncoder(); 
    enc->init(*out); 
    avro::encode(*enc, deviceInfo); 
    out->flush(); 

    size_t byte_count = out->byteCount(); 
    DBG("BYTE COUNT " << byte_count); 

    std::auto_ptr<avro::InputStream> in = avro::memoryInputStream(*out); 
    avro::StreamReader reader(*in); 
    std::vector<uint8_t> row_data(byte_count); 
    reader.readBytes(&row_data[0], byte_count); 

的Java解碼器

@Override 
    public Object deserializeByteArr(Schema schema, final byte[] data){ 
     DatumReader<GenericRecord> genericDatumReader = new SpecificDatumReader<>(schema); 
     Decoder decoder = DecoderFactory.get().binaryDecoder(data, null); 
     try { 
      GenericRecord userData = genericDatumReader.read(null, decoder); 
      System.out.println(userData); 
     } catch (IOException e) { 
      e.printStackTrace(); 
     } 
     return null; 
    } 

方案2 - 不工作

請注意，我已經更新的模式和再生對應模式

C++文件

test.avsc

[ 
    { 
     "namespace": "com.company.project", 
     "name": "Component_DeviceInfo", 
     "type": "record", 
     "doc": "Identifies a client device", 
     "fields": [ 
      { 
       "name": "deviceId", 
       "type": [ 
        "null", 
        "string" 
       ], 
       "default": null, 
       "doc": "Unique MAC address" 
      }, 
      { 
       "name": "zoneId", 
       "type": [ 
        "null", 
        "string" 
       ], 
       "default": null, 
       "doc": "Zone id where Client device belongs to" 
      } 
     ] 
    }, 
    { 
     "namespace": "com.company.project", 
     "name": "Component_EventList", 
     "type": "record", 
     "doc": "Component Event list", 
     "fields": [ 
      { 
       "name": "deviceInfo", 
       "type": [ 
        "null", 
        "com.company.project.Component_DeviceInfo" 
       ], 
       "default": null, 
       "doc": "Device information such as device id and zone id" 
      } 
     ] 
    } 
] 

編碼器C++

Component_DeviceInfo deviceInfo; 
    deviceInfo.deviceId.set_string("device1"); 
    deviceInfo.zoneId.set_string("zone1"); 

    std::vector <char>tele_bytes_; 

    Component_EventList ComponentEventList; 
    ComponentEventList.deviceInfo.set_Component_DeviceInfo(deviceInfo); 

    std::auto_ptr<avro::OutputStream> out = avro::memoryOutputStream(1); 
    avro::EncoderPtr enc = avro::binaryEncoder(); 
    enc->init(*out); 
    avro::encode(*enc, ComponentEventList); 
    out->flush(); 

    size_t byte_count = out->byteCount(); 
    DBG("BYTE COUNT " << byte_count); 

    std::auto_ptr<avro::InputStream> in = avro::memoryInputStream(*out); 
    avro::StreamReader reader(*in); 
    std::vector<uint8_t> row_data(byte_count); 
    reader.readBytes(&row_data[0], byte_count); 

輸出

org.springframework.kafka.listener.ListenerExecutionFailedException: Listener method 'public void com.company.telemetry.services.consumer.TelemetryConsumerService.consume(org.apache.kafka.clients.consumer.ConsumerRecord<java.lang.String, byte[]>)' threw exception; nested exception is java.lang.ArrayIndexOutOfBoundsException: 7 
    at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:188) ~[spring-kafka-1.1.6.RELEASE.jar:na] 
    at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:72) ~[spring-kafka-1.1.6.RELEASE.jar:na] 
    at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:47) ~[spring-kafka-1.1.6.RELEASE.jar:na] 
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:794) [spring-kafka-1.1.6.RELEASE.jar:na] 
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:738) [spring-kafka-1.1.6.RELEASE.jar:na] 
    at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:570) [spring-kafka-1.1.6.RELEASE.jar:na] 
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_91] 
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_91] 
    at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91] 
Caused by: java.lang.ArrayIndexOutOfBoundsException: 7 
    at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:402) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.io.parsing.Parser.advance(Parser.java:88) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:155) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:155) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:155) ~[avro-1.7.7.jar:1.7.7] 
    at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142) ~[avro-1.7.7.jar:1.7.7] 
    at com.company.telemetry.services.serde.AvroByteArrDeserializer.deserializeByteArr(AvroByteArrDeserializer.java:32) ~[classes/:na] 
    at com.company.telemetry.services.TelemetryService.handleByteArr(TelemetryService.java:59) ~[classes/:na] 
    at com.company.telemetry.services.consumer.TelemetryConsumerService.consume(TelemetryConsumerService.java:39) ~[classes/:na] 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_91] 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_91] 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91] 
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91] 
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:180) ~[spring-messaging-4.3.11.RELEASE.jar:4.3.11.RELEASE] 
    at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:112) ~[spring-messaging-4.3.11.RELEASE.jar:4.3.11.RELEASE] 
    at org.springframework.kafka.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:48) ~[spring-kafka-1.1.6.RELEASE.jar:na] 
    at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:174) ~[spring-kafka-1.1.6.RELEASE.jar:na] 
    ... 8 common frames omitted 

我將不勝感激，如果有人可以告訴我怎麼回事！謝謝 ！

Answer 1

的問題是我用錯了類創建Avro的編碼數據。但更大的問題竟然是頂級聯盟 - https://issues.apache.org/jira/browse/AVRO-2095

Answer 2

如果你定義了DeviceInfo內聯而不是avsc中數組的元素，它是否工作？

Answer 3

我相信你所做的是在你的第二個模式中定義一個聯合。

你可能想看看這個，我沒有找到所有有用的，誠實的。

http://apache-avro.679487.n3.nabble.com/avrogencpp-and-multiple-avsc-td4028378.html