构建Giraph时出现编译错误

Question

我正在尝试构建Giraph。我有: java版本"1.7.0_25"，Apache Maven3.0.4，Hadoop1.0.4。我正在遵循此页面中的说明：https://cwiki.apache.org/confluence/display/GIRAPH/Quick+Start+Guide

当我运行: mvn compile时，我得到以下错误：

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Apache Giraph Parent
[INFO] Apache Giraph Core
[INFO] Apache Giraph Examples
[INFO] Apache Giraph Accumulo I/O
[INFO] Apache Giraph HBase I/O
[INFO] Apache Giraph HCatalog I/O
[INFO] Apache Giraph Hive I/O
[INFO] Apache Giraph Rexster I/O
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Giraph Parent 1.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- mavanagaiata:0.5.0:branch (git-commit) @ giraph-parent ---
[INFO]
[INFO] --- mavanagaiata:0.5.0:commit (git-commit) @ giraph-parent ---
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Giraph Core 1.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- mavanagaiata:0.5.0:branch (git-commit) @ giraph-core ---
[INFO]
[INFO] --- mavanagaiata:0.5.0:commit (git-commit) @ giraph-core ---
[INFO]
[INFO] --- maven-enforcer-plugin:1.0.1:enforce (enforce-maven) @ giraph-core ---
[INFO]
[INFO] --- munge-maven-plugin:1.0:munge (munge) @ giraph-core ---
[INFO]
[INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ giraph-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ giraph-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 532 source files to /home/tom/giraph/giraph-core/target/munged/classes
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayInputStream.java:[34,32] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-    core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayInputStream.java:[37,29] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayInputStream.java:[39,25] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayOutputStream.java:[33,29] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayOutputStream.java:[35,25] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayOutputStream.java:[66,32] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/master/BspServiceMaster.java: Some input files use or override a deprecated API.
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/master/BspServiceMaster.java: Recompile with -Xlint:deprecation for details.
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/metrics/GiraphMetricsRegistry.java: Some input files use unchecked or unsafe operations.
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/metrics/GiraphMetricsRegistry.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --- maven-assembly-plugin:2.4:single (build-fat-jar) @ giraph-core ---
[INFO] Building jar: /home/tom/giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (default) @ giraph-core ---
[INFO] Executing tasks

main:
 [java] java.lang.NoClassDefFoundError: org/apache/hadoop/io/Writable
 [java]     at org.apache.tools.ant.taskdefs.ExecuteJava.execute(ExecuteJava.java:194)
 [java]     at org.apache.tools.ant.taskdefs.Java.run(Java.java:771)
 [java]     at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:221)
 [java]     at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:135)
 [java]     at org.apache.tools.ant.taskdefs.Java.execute(Java.java:108)
 [java]     at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
 [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 [java]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 [java]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 [java]     at java.lang.reflect.Method.invoke(Method.java:606)
 [java]     at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
 [java]     at org.apache.tools.ant.Task.perform(Task.java:348)
 [java]     at org.apache.tools.ant.Target.execute(Target.java:390)
 [java]     at org.apache.tools.ant.Target.performTasks(Target.java:411)
 [java]     at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
 [java]     at org.apache.tools.ant.Project.executeTarget(Project.java:1366)
 [java]     at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:270)
 [java]     at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
 [java]     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
 [java]     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
 [java]     at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
 [java]     at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
 [java]     at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
 [java]     at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
 [java]     at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
 [java]     at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
 [java]     at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
 [java]     at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
 [java]     at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
 [java]     at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
 [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 [java]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 [java]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 [java]     at java.lang.reflect.Method.invoke(Method.java:606)
 [java]     at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
 [java]     at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
 [java]     at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
 [java]     at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
 [java] Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/io/Writable
 [java]     at java.lang.ClassLoader.defineClass1(Native Method)
 [java]     at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
 [java]     at org.apache.tools.ant.AntClassLoader.defineClassFromData(AntClassLoader.java:1124)
 [java]     at org.apache.tools.ant.AntClassLoader.getClassFromStream(AntClassLoader.java:1295)
 [java]     at org.apache.tools.ant.AntClassLoader.findClassInComponents(AntClassLoader.java:1351)
 [java]     at org.apache.tools.ant.AntClassLoader.findClass(AntClassLoader.java:1311)
 [java]     at org.apache.tools.ant.AntClassLoader.loadClass(AntClassLoader.java:1070)
 [java]     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 [java]     at java.lang.ClassLoader.defineClass1(Native Method)
 [java]     at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
 [java]     at org.apache.tools.ant.AntClassLoader.defineClassFromData(AntClassLoader.java:1124)
 [java]     at org.apache.tools.ant.AntClassLoader.getClassFromStream(AntClassLoader.java:1295)
 [java]     at org.apache.tools.ant.AntClassLoader.findClassInComponents(AntClassLoader.java:1351)
 [java]     at org.apache.tools.ant.AntClassLoader.findClass(AntClassLoader.java:1311)
 [java]     at org.apache.tools.ant.AntClassLoader.loadClass(AntClassLoader.java:1070)
 [java]     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 [java]     at java.lang.ClassLoader.defineClass1(Native Method)
 [java]     at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
 [java]     at org.apache.tools.ant.AntClassLoader.defineClassFromData(AntClassLoader.java:1124)
 [java]     at org.apache.tools.ant.AntClassLoader.getClassFromStream(AntClassLoader.java:1295)
 [java]     at org.apache.tools.ant.AntClassLoader.findClassInComponents(AntClassLoader.java:1351)
 [java]     at org.apache.tools.ant.AntClassLoader.findClass(AntClassLoader.java:1311)
 [java]     at org.apache.tools.ant.AntClassLoader.loadClass(AntClassLoader.java:1070)
 [java]     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 [java]     at org.apache.giraph.conf.GiraphConstants.<clinit>(GiraphConstants.java:129)
 [java]     at org.apache.giraph.conf.AllOptions.main(AllOptions.java:153)
 [java]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 [java]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 [java]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 [java]     at java.lang.reflect.Method.invoke(Method.java:606)
 [java]     at org.apache.tools.ant.taskdefs.ExecuteJava.run(ExecuteJava.java:217)
 [java]     at org.apache.tools.ant.taskdefs.ExecuteJava.execute(ExecuteJava.java:152)
 [java]     ... 37 more
 [java] Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.io.Writable
 [java]     at org.apache.tools.ant.AntClassLoader.findClassInComponents(AntClassLoader.java:1361)
 [java]     at org.apache.tools.ant.AntClassLoader.findClass(AntClassLoader.java:1311)
 [java]     at org.apache.tools.ant.AntClassLoader.loadClass(AntClassLoader.java:1070)
 [java]     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
 [java]     ... 69 more
 [java] Java Result: -1
[INFO] Executed tasks
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Giraph Examples 1.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Giraph Parent .............................. SUCCESS [1.604s]
[INFO] Apache Giraph Core ................................ SUCCESS [38.503s]
[INFO] Apache Giraph Examples ............................ FAILURE [0.095s]
[INFO] Apache Giraph Accumulo I/O ........................ SKIPPED
[INFO] Apache Giraph HBase I/O ........................... SKIPPED
[INFO] Apache Giraph HCatalog I/O ........................ SKIPPED
[INFO] Apache Giraph Hive I/O ............................ SKIPPED
[INFO] Apache Giraph Rexster I/O ......................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 41.175s
[INFO] Finished at: Tue Aug 06 11:02:24 PDT 2013
[INFO] Final Memory: 46M/439M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project giraph-examples: Could not resolve dependencies for project org.apache.giraph:giraph-examples:jar:1.1.0-SNAPSHOT: Failure to find org.apache.giraph:giraph-core:jar:tests:1.1.0-SNAPSHOT in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced ->         [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :giraph-examples

我尝试运行mvn -Phadoop_non_secure编译，但得到以下错误：

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] Apache Giraph Parent
[INFO] Apache Giraph Core
[INFO] Apache Giraph Examples
[INFO] Apache Giraph Accumulo I/O
[INFO] Apache Giraph HBase I/O
[INFO] Apache Giraph HCatalog I/O
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Giraph Parent 1.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- mavanagaiata:0.5.0:branch (git-commit) @ giraph-parent ---
[INFO]
[INFO] --- mavanagaiata:0.5.0:commit (git-commit) @ giraph-parent ---
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Apache Giraph Core 1.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- mavanagaiata:0.5.0:branch (git-commit) @ giraph-core ---
[INFO]
[INFO] --- mavanagaiata:0.5.0:commit (git-commit) @ giraph-core ---
[INFO]
[INFO] --- maven-enforcer-plugin:1.0.1:enforce (enforce-maven) @ giraph-core ---
[INFO]
[INFO] --- munge-maven-plugin:1.0:munge (munge) @ giraph-core ---
[INFO]
[INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ giraph-core ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @ giraph-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 525 source files to /home/tom/giraph/giraph-core/target/munged/classes
[INFO] -------------------------------------------------------------
[WARNING] COMPILATION WARNING :
[INFO] -------------------------------------------------------------
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayInputStream.java:[34,32] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayInputStream.java:[37,29] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayInputStream.java:[39,25] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayOutputStream.java:[33,29] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayOutputStream.java:[35,25] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/utils/UnsafeByteArrayOutputStream.java:[66,32] sun.misc.Unsafe is internal proprietary API and may be removed in a future release
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/master/BspServiceMaster.java: /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/master/BspServiceMaster.java uses or overrides a deprecated API.
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/master/BspServiceMaster.java: Recompile with -Xlint:deprecation for details.
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/metrics/GiraphMetricsRegistry.java: Some input files use unchecked or unsafe operations.
[WARNING] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/metrics/GiraphMetricsRegistry.java: Recompile with -Xlint:unchecked for details.
[INFO] 10 warnings
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/io/formats/GraphvizOutputFormat.java:[30,35] cannot find symbol
  symbol:   class JobStatus
  location: package org.apache.hadoop.mapreduce
[ERROR] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/io/formats/GraphvizOutputFormat.java:[187,68] package JobStatus does not exist
[ERROR] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/io/formats/GraphvizOutputFormat.java:[187,5] method does not override or implement a method from a supertype
[ERROR] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/io/formats/GraphvizOutputFormat.java:[204,15] cannot find symbol
  symbol:   method commitJob(org.apache.hadoop.mapreduce.JobContext)
  location: variable delegate of type org.apache.hadoop.mapreduce.OutputCommitter
[ERROR] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/io/formats/GraphvizOutputFormat.java:[202,5] method does not override or implement a method from a supertype
[INFO] 5 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Giraph Parent .............................. SUCCESS [1.637s]
[INFO] Apache Giraph Core ................................ FAILURE [12.114s]
[INFO] Apache Giraph Examples ............................ SKIPPED
[INFO] Apache Giraph Accumulo I/O ........................ SKIPPED
[INFO] Apache Giraph HBase I/O ........................... SKIPPED
[INFO] Apache Giraph HCatalog I/O ........................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.567s
[INFO] Finished at: Tue Aug 06 11:06:34 PDT 2013
[INFO] Final Memory: 28M/409M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project giraph-core: Compilation failure: Compilation failure:
[ERROR] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/io/formats/GraphvizOutputFormat.java:[30,35] cannot find symbol
[ERROR] symbol:   class JobStatus
[ERROR] location: package org.apache.hadoop.mapreduce
[ERROR] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/io/formats/GraphvizOutputFormat.java:[187,68] package JobStatus does not exist
[ERROR] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/io/formats/GraphvizOutputFormat.java:[187,5] method does not override or implement a method from a supertype
[ERROR] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/io/formats/GraphvizOutputFormat.java:[204,15] cannot find symbol
[ERROR] symbol:   method commitJob(org.apache.hadoop.mapreduce.JobContext)
[ERROR] location: variable delegate of type org.apache.hadoop.mapreduce.OutputCommitter
[ERROR] /home/tom/giraph/giraph-core/target/munged/main/org/apache/giraph/io/formats/GraphvizOutputFormat.java:[202,5] method does not override or implement a method from a supertype
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :giraph-core

我也尝试了在线源代码库，但仍然收到相同的错误。有人能帮我解决这个问题吗？

我应该在PATH和CLASSPATH中添加什么？

Answer 1

我猜你正在尝试构建Giraph-1.1.0-SNAPSHOT，它还没有发布。这只是一个git快照，它可能在任何地方失败。我建议你使用giraph 1.0.0，它要稳定得多。

Answer 2

这可能是因为你没有安装Munge插件。你安装了吗？