不是org.apache.hadoop.mapreduce.Mapper

Question

我正在写一个mapreduce项目。

我想把一个数组从mapper发送到reducer。

但它有一个错误，我无法修复它。

我导入这些类：

 import java.io.DataInput; 
    import java.io.DataOutput; 
    import java.io.EOFException; 
    import java.io.IOException; 
    import java.net.Socket; 
    import java.util.HashMap; 
    import java.util.HashSet; 
    import java.util.Map; 
    import java.util.Set; 
    import org.apache.hadoop.conf.Configured;
    import java.io.BufferedInputStream;
    import java.io.BufferedReader;
    import java.util.*;
    import java.io.File;
    import java.io.FileInputStream;
    import java.io.FileOutputStream;
    import java.io.FileReader;
    import java.io.IOException;
    import java.io.InputStream;
    import java.io.InputStreamReader;
    import java.io.ObjectInputStream;
    import java.io.ObjectOutputStream;
    import java.io.OutputStream;
    import java.util.Iterator;
    import hadoop.DENCLUE;
    //import javafx.scene.text.Text;
    import sun.security.krb5.Config;
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.fs.FileSystem;
    import org.apache.hadoop.fs.Path;
    import org.apache.hadoop.fs.viewfs.Constants;
    import org.apache.hadoop.mapred.FileInputFormat;
    import org.apache.hadoop.mapred.FileOutputFormat;
    import org.apache.hadoop.mapred.JobClient;
    import org.apache.hadoop.mapred.JobConf;
    import org.apache.hadoop.mapred.MapReduceBase;
    import org.apache.hadoop.mapred.Mapper;
    import org.apache.hadoop.mapred.OutputCollector;
    import org.apache.hadoop.mapred.Reducer;
    import org.apache.hadoop.mapred.Reporter;
    import org.apache.hadoop.mapred.TextInputFormat;
    import org.apache.hadoop.mapred.TextOutputFormat;
    //import org.apache.hadoop.mapred.jobcontrol.Job;
    import org.apache.hadoop.util.Tool; 
    import org.apache.hadoop.util.ToolRunner;
    import org.omg.CORBA.PUBLIC_MEMBER;
    import com.sun.org.apache.bcel.internal.generic.NEW;
    import org.apache.hadoop.conf.*;
    import org.apache.hadoop.fs.*;
    import org.apache.hadoop.io.*;
    import org.apache.hadoop.mapreduce.Job;
    import org.apache.hadoop.mapreduce.lib.input.*;
    import org.apache.hadoop.mapreduce.lib.output.*;
    import org.apache.hadoop.mapreduce.Mapper.Context;
    import org.apache.hadoop.mapreduce.task.JobContextImpl;
    import org.apache.hadoop.mapreduce.lib.input.*;
    import org.apache.hadoop.mapreduce.lib.output.*;
    import org.apache.hadoop.util.*;
    import java.io.DataOutput;
    import java.io.DataInput;
    import java.io.IOException;

这是我的Map类：

    public static class Mapn extends MapReduceBase implements Mapper<LongWritable, Text, Text, Text> {
        @SuppressWarnings("rawtypes")
        Context con ;
        @SuppressWarnings("unchecked")
        public void map(LongWritable key, Text value, OutputCollector< Text,Text > 

            output, Reporter reporter) throws IOException  {

                String line = value.toString();
                String[] words=line.split(",");
                for(String word: words )
                {
                      Text outputKey = new Text(word.toUpperCase().trim());

                  try {
                    con.write(outputKey, words);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                }

这就是工作：

public static void main(String[] args) throws Exception {


    Configuration c=new Configuration();
    String[] files=new GenericOptionsParser(c,args).getRemainingArgs();
    Path input=new Path(files[0]);
    Path output=new Path(files[1]);
    Job j=new Job(c,"wnt");
    j.setJarByClass(projectmr.class);
    j.setMapperClass(Mapn.class);
    j.setReducerClass(Reduce.class);
    j.setOutputKeyClass(Text.class);
    j.setOutputValueClass(Text.class);
    FileInputFormat.addInputPaths(j, input);
    FileOutputFormat.setOutputPath(j, output);
    System.exit(j.waitForCompletion(true)?0:1);

下面是我得到的错误：

Exception in thread "main" java.lang.RuntimeException: class hadoop.projectmr$Mapn not org.apache.hadoop.mapreduce.Mapper
    at org.apache.hadoop.conf.Configuration.setClass(Configuration.java:1969)
    at org.apache.hadoop.mapreduce.Job.setMapperClass(Job.java:891)
    at hadoop.projectmr.main(projectmr.java:191)

Answer 1

这是旧的Hadoop 1 API

import org.apache.hadoop.mapred.*;

您应该从中的类导入

org.apache.hadoop.mapreduce.*;

如错误所示

不是org.apache.hadoop.mapreduce.Mapper

因此，基本上，您不需要MapReduceBase，并且Mapper现在是一个类，而不是接口

所以，你现在应该有

public static class MyMapper extends Mapper<Kin, Vin, Kout, Vout>

Look at the WordCount code