MapReduce来源于函数式编程的Map,Reduce概念,Map是映射,Reduce是规约。说了非常简单,就真的是非常简单的,先看Hadoop自带的sample-WordCount ,再看Nutch里的Indexer,Fetcher两个实战的例子,最后阅读Hadoop wiki 上的HadoopMapReduce,很快就可以上手:
public static void main(String[] args) main() { JobConf conf = new JobConf(WordCount. class ); conf.setJobName( " wordcount " );
jobConf.setInputFormat(MyInputFormat. class ); jobConf.setOutputFormat(MyOutputFormat. class );
jobConf.setInputKeyClass(BytesWritable. class ); jobConf.setInputValueClass(BytesWritable. class ); jobConf.setOutputKeyClass(BytesWritable. class ); jobConf.setOutputValueClass(BytesWritable. class );
jobConf.setMapperClass(MyMapper. class ); jobConf.setReducerClass(MyReducer. class );