Hadoop：如何將雙精度數組作爲鍵值對中的值？

Question

我有一個問題，我需要聚合一些載體，以便找到一些統計數據。例如，我有雙打矢量，我需要總結它們。我的向量是這樣的：

 1,0,3,4,5 
     2,3,4,5,6 
     3,4,5,5,6 

到目前爲止，我的鍵值對是（String，String）。但每次我需要添加這些矢量時，首先我必須將它們轉換爲雙數組，然後將它們相加，最後將聚合矢量轉換爲字符串。我認爲如果我可以在窗體中有鍵值對（String，double array），速度會快很多。沒有必要來回轉換它們。我的問題是，我不能找到一種方法來將雙數組作爲值。有沒有簡單的方法，而不是創建一個新的自定義類型？

Answer 1

你的意思是這樣的嗎？

Map<String, List<Double>> arrays = new HashMap<String, List<Double>>(); 

double[] array; 
arrays.put("ArrayKey", Arrays.asList(array)); 

，那麼你可以調用地圖的方法：

map(String key, String arrayKey) { 
    List<Double> value = arrays.get(arrayKey); 
} 

你也可以序列化的雙陣列，然後反序列化回：

package test; 

import org.apache.commons.codec.binary.Base64InputStream; 
import org.apache.commons.codec.binary.Base64OutputStream; 

import java.io.*; 
import java.util.Arrays; 

public class Test { 

    public static void main(String[] args) throws IOException, ClassNotFoundException { 
     double[] array = {0.0, 1.1, 2.2, 3.3}; 
     String stringValue = serialize(array); 
     map("Key", stringValue); 
    } 

    public static void map(String key, String value) throws ClassNotFoundException, IOException { 
     double[] array = deserialize(value); 
     System.out.println("Key=" + key + "; Value=" + Arrays.toString(array)); 
    } 

    public static String serialize(double[] array) throws IOException { 
     ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(); 
     Base64OutputStream base64OutputStream = new Base64OutputStream(byteArrayOutputStream); 
     ObjectOutputStream oos = new ObjectOutputStream(base64OutputStream); 
     oos.writeObject(array); 
     oos.flush(); 
     oos.close(); 
     return byteArrayOutputStream.toString(); 
    } 

    public static double[] deserialize(String stringArray) throws IOException, ClassNotFoundException { 
     ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(stringArray.getBytes()); 
     Base64InputStream base64InputStream = new Base64InputStream(byteArrayInputStream); 
     ObjectInputStream iis = new ObjectInputStream(base64InputStream); 
     return (double[]) iis.readObject(); 
    } 
} 

OUTPUT：

Key=Key; Value=[0.0, 1.1, 2.2, 3.3] 

映射速度更快，但s erialization會更加有用，如果你使用節點和集羣爲（如果你需要你的數組傳遞到另一個JVM）：

private static class SpeedTest { 
     private static final Map<String, List> arrays = new HashMap<String, List>(); 

     public static void test(final double[] array) throws IOException, ClassNotFoundException { 
      final String str = serialize(array); 
      final int amount = 10 * 1000; 

      long timeStamp = System.currentTimeMillis(); 
      for (int i = 0; i < amount; i++) { 
       serialize(array); 
      } 
      System.out.println("Serialize: " + (System.currentTimeMillis() - timeStamp) + " ms"); 

      timeStamp = System.currentTimeMillis(); 
      for (int i = 0; i < amount; i++) { 
       deserialize(str); 
      } 
      System.out.println("Deserialize: " + (System.currentTimeMillis() - timeStamp) + " ms"); 

      arrays.clear(); 
      timeStamp = System.currentTimeMillis(); 
      // Prepaire map, that contains reference for all arrays. 
      for (int i = 0; i < amount; i++) { 
       arrays.put("key_" + i, Arrays.asList(array)); 
      } 
      // Getting array by its key in map. 
      for (int i = 0; i < amount; i++) { 
       arrays.get("key_" + i).toArray(); 
      } 
      System.out.println("Mapping: " + (System.currentTimeMillis() - timeStamp) + " ms"); 
     } 
    } 

OUTPUT：

Serialize: 298 ms 
Deserialize: 254 ms 
Mapping: 27 ms