擴展了擴展Hadoop映射器的類

這是擴展Mapper類的Hadoop中Map類[1]的一個示例。 [3]是Hadoop的Mapper類。擴展了擴展Hadoop映射器的類

我想創建我的MyExampleMapper，它擴展ExampleMapper，這也擴展了hadoop的Mapper [2]。我這樣做是因爲我只想在ExampleMapper中設置一個屬性，以便在創建MyExampleMapper或其他示例時，我不必自己設置屬性，因爲我擴展了ExampleMapper。是否有可能做到這一點？

[1]實施例映射器

import org.apache.hadoop.mapreduce.Mapper; 

public class ExampleMapper 
    extends Mapper<Object, Text, Text, IntWritable>{ 

    private final static IntWritable one = new IntWritable(1); 
    private Text word = new Text(); 

    public void map(Object key, Text value, Context context) throws IOException, InterruptedException { 
    StringTokenizer itr = new StringTokenizer(value.toString()); 
    while (itr.hasMoreTokens()) { 
     word.set(itr.nextToken()); 
     context.write(word, one); 
    } 
    } 
}

[2]我想什麼

import org.apache.hadoop.mapreduce.Mapper; 

public class MyExampleMapper 
    extends ExampleMapper<Object, Text, Text, IntWritable>{ 

    private final static IntWritable one = new IntWritable(1); 
    private Text word = new Text(); 

    public void map(Object key, Text value, Context context) throws IOException, InterruptedException { 
    StringTokenizer itr = new StringTokenizer(value.toString()); 

    String result = System.getProperty("job.examplemapper") 

    if (result.equals("true")) { 
     while (itr.hasMoreTokens()) { 
     word.set(itr.nextToken()); 
     context.write(word, one); 
     } 
    } 
    } 
} 


public class ExampleMapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT> 
    extends Mapper{ 

    System.setProperty("job.examplemapper", "true"); 
}

[3]這是Hadoop的映射器類

public class Mapper<KEYIN, VALUEIN, KEYOUT, VALUEOUT> { 
    public Mapper() { 
    } 

    protected void setup(Mapper.Context context) throws IOException, InterruptedException { 
    } 

    protected void map(KEYIN key, VALUEIN value, Mapper.Context context) throws IOException, InterruptedException { 
     context.write(key, value); 
    } 

    protected void cleanup(Mapper.Context context) throws IOException, InterruptedException { 
    } 

    public void run(Mapper.Context context) throws IOException, InterruptedException { 
     this.setup(context); 

     try { 
      while(context.nextKeyValue()) { 
       this.map(context.getCurrentKey(), context.getCurrentValue(), context); 
      } 
     } finally { 
      this.cleanup(context); 
     } 

    } 

    public class Context extends MapContext<KEYIN, VALUEIN, KEYOUT, VALUEOUT> { 
     public Context(Configuration var1, TaskAttemptID conf, RecordReader<KEYIN, VALUEIN> taskid, RecordWriter<KEYOUT, VALUEOUT> reader, OutputCommitter writer, StatusReporter committer, InputSplit reporter) throws IOException, InterruptedException { 
      super(conf, taskid, reader, writer, committer, reporter, split); 
     } 
    } 
}

來源

2016-08-06 xeon

import org.apache.hadoop.mapreduce.Mapper; 

public class ExampleMapper<T, X, Y, Z> extends Mapper<T, X, Y, Z> { 
    static { 
     System.setProperty("job.examplemapper", "true"); 
    } 
}

然後延伸它在你的程序

中

public class MyExampleMapper 
    extends ExampleMapper<Object, Text, Text, IntWritable>{ 

    private final static IntWritable one = new IntWritable(1); 
    private Text word = new Text(); 

    public void map(Object key, Text value, Context context) throws IOException, InterruptedException { 
    StringTokenizer itr = new StringTokenizer(value.toString()); 

    String result = System.getProperty("job.examplemapper") 

    if (result.equals("true")) { 
     while (itr.hasMoreTokens()) { 
     word.set(itr.nextToken()); 
     context.write(word, one); 
     } 
    } 
    } 
}

來源

2016-08-07 07:43:50

擴展了擴展Hadoop映射器的類

回答

相關問題