运行铲运机导入作业时出错

Question

我正在尝试使用scoop将数据从MySQL数据库导入到hdfs，并且我得到了“不能为ClassWriter生成列”错误。

这是我的独家命令：

sqoop import --connect jdbc:mysql://xxxxx.xxxx.us-east-1.rds.amazonaws.com/xxxx --username xxxx --password xxxx --table xxxx --null-string 'NA' --null-non-string '\\N' --warehouse-dir /user/romeo/finalproject/cardMember

下面是错误堆栈：

com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: 20/03/08 16:11:27错误manager.SqlManager:错误执行语句：

通信链接失败

最后一次成功发送到服务器的数据包是0毫秒前。驱动程序尚未收到来自服务器的任何数据包。com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:通信链路故障

最后一次成功发送到服务器的数据包是0毫秒前。驱动程序尚未收到来自服务器的任何数据包。( sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native方法)在sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)，sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)，java.lang.reflect.Constructor.newInstance(Constructor.java:526)，com.mysql.jdbc.Util.handleNewInstance(Util.java:411)，com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121)在com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2484)，com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2521)，com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2306)，com.mysql.jdbc.ConnectionImpl.(ConnectionImpl.java:839)，com.mysql.jdbc.JDBC4Connection。( sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native )在sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)，sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)，java.lang.reflect.Constructor.newInstance(Constructor.java:526)，com.mysql.jdbc.Util.handleNewInstance(Util.java:411)，com.mysql.jdbc。ConnectionImpl.getInstance(ConnectionImpl.java:421) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:350) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:215) at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:903) at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)在org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:762) at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:785) at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:288) at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:259) at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:245)在org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:333) at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1858) at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1657) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) atorg.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252)由: java.net.UnknownHostException: upgradawsrds.cpclxrkdvwmz.us-east1.rds.amazonaws.com at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)在java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)引起在java.net.InetAddress.getAllByName0(InetAddress.java:1246) at java.net.InetAddress.getAllByName(InetAddress.java:1162) at java.net.InetAddress.getAllByName(InetAddress.java:1098) at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:249) at com.mysql.jdbc.MysqlIO.(MysqlIO.java:307) . 33 /03/08 16:11:27 ERROR tool.ImportTool:导入失败:在org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)上为ClassWriter生成的java.io.IOException:无列org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

知道怎么解决这个问题吗？提前谢谢你。

Answer 1

从堆栈跟踪中可以很清楚地看出，这是由关于URI java.net.UnknownHostException的upgradawsrds.cpclxrkdvwmz.us-east1.rds.amazonaws.com.引起的。URI不正确，或者它指向的DNS服务器没有解析。更新URI以更新DNS。