mapreduce ---自定義數據類型

Question

當我做一個mapreduce程序，我遇到的關鍵是一個元組（A，B）（A和B都是整數集）。我怎樣才能定製這種數據類型？

public static class MapClass extends Mapper<Object,Text,Tuple,Tuple>.... 

public class Tuple implements WritableComparable<Tuple>{ 


     @Override 
     public void readFields(DataInput arg0) throws IOException { 
      // TODO Auto-generated method stub 

     } 

     @Override 
     public void write(DataOutput arg0) throws IOException { 
      // TODO Auto-generated method stub 

     } 

     @Override 
     public int compareTo(Tuple o) { 
      // TODO Auto-generated method stub 
      return 0; 
     } 
    } 

Answer 1

就快，只需添加變量A和B，然後完成序列化方法和的compareTo：

public class Tuple implements WritableComparable<Tuple>{ 
    public Set<Integer> a = new TreeSet<Integer>; 
    public Set<Integer> b = new TreeSet<Integer>; 

    @Override 
    public void readFields(DataInput arg0) throws IOException { 
     a.clear(); 
     b.clear(); 

     int count = arg0.readInt(); 
     while (count-- > 0) { 
      a.add(arg0.readInt()); 
     } 

     count = arg0.readInt(); 
     while (count-- > 0) { 
      b.add(arg0.readInt()); 
     } 
    } 

    @Override 
    public void write(DataOutput arg0) throws IOException { 
     arg0.writeInt(a.size()); 
     for (int v : a) { 
      arg0.writeInt(v); 
     } 
     arg0.writeInt(b.size()); 
     for (int v : b) { 
      arg0.writeInt(v); 
     } 
    } 

    @Override 
    public int compareTo(Tuple o) { 
     // you'll need to implement how you want to compare the two sets between objects 
    } 
} 

Answer 2

實現Hadoop中的自定義數據類型，則必須實現WritableComparable接口併爲readFields（）write（）方法提供自定義實現。 除了執行readFiled之外，寫入方法必須覆蓋java對象的equals和hashcode方法。

如果自定義數據類型實現的密鑰必須實現可比較的接口。