分享

eclipse 运行mapreduce去重程序报空值异常

xianyang 发表于 2015-6-14 17:00:46 [显示全部楼层] 只看大图 回帖奖励 阅读模式 关闭右栏 1 11720
public class DataDistinct {

public static class Map extends Mapper<Object,Text,Text,Text>{

  private static Text line=new Text();
  public void map(Object key,Text value,Context context) throws IOException, InterruptedException{
   line=value;
   context.write(line, new Text(""));
  }
}

public static class Reduce extends Reducer<Text,Text,Text,Text>{
  public void reduce(Text key,Iterable<Text> values,Context context) throws IOException, InterruptedException{
   context.write(key, new Text(""));
  }
}

public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException{
  Configuration conf=new Configuration();
  conf.set("fs.default.name", "hdfs://192.168.1.124:9000");
  conf.set("mapreduce.jobtracker.address","hdfs://nameserver:9001");
  String[] ioStr=new String[]{"dedup_in","dedup_out"};
  String[] otherStr=new GenericOptionsParser(conf,ioStr).getRemainingArgs();
  if(otherStr.length!=2){
   System.out.println("目录错误");
   System.exit(2);
  }
  

  Job job=new Job(conf,"Data Deduplication");
  job.setJarByClass(DataDistinct.class);
  
  job.setMapperClass(Map.class);

  job.setCombinerClass(Reduce.class);
  job.setReducerClass(Reduce.class);
  
  job.setOutputKeyClass(Text.class);
  job.setOutputValueClass(Text.class);
  
  FileInputFormat.addInputPath(job, new Path(otherStr[0]));
  FileOutputFormat.setOutputPath(job,new Path(otherStr[1]));
  
  System.exit(job.waitForCompletion(true)?0:1);
  
}
}
一运行就报空值异常,为什么啊,新手求助

捕获.PNG

已有(1)人评论

跳转到指定楼层
tntzbzc 发表于 2015-6-14 21:58:44
这里不对

context.write(line, new Text(""));
new Text("")为空了。
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条