sqoop启动作业报错

sqoop:000> start job --name hdfs2mysql
Exception has occurred during processing command
Exception: org.apache.sqoop.common.SqoopException Message: DRIVER_0002:Given job is already running - Job with name hdfs2mysql
Stack trace:
Caused by: Exception: org.apache.sqoop.common.SqoopException Message: DRIVER_0002:Given job is already running - Job with name hdfs2mysql
Stack trace:
   at  org.apache.sqoop.driver.JobManager (JobManager.java:326)
   at  org.apache.sqoop.handler.JobRequestHandler (JobRequestHandler.java:353)
   at  org.apache.sqoop.handler.JobRequestHandler (JobRequestHandler.java:114)
   at  org.apache.sqoop.server.v1.JobServlet (JobServlet.java:84)
   at  org.apache.sqoop.server.SqoopProtocolServlet (SqoopProtocolServlet.java:81)
   at  javax.servlet.http.HttpServlet (HttpServlet.java:710)
   at  javax.servlet.http.HttpServlet (HttpServlet.java:790)
   at  org.eclipse.jetty.servlet.ServletHolder (ServletHolder.java:808)
   at  org.eclipse.jetty.servlet.ServletHandler$CachedChain (ServletHandler.java:1669)

我开始导入mysql成功了，后面清空mysql数据库重新演示一遍就报错了。

fylanyu · 发表于 2017-4-6 10:03:36

楼主换个名字
start job --name hdfs3mysql

tanxl · 发表于 2017-4-6 10:04:36

fylanyu 发表于 2017-4-6 10:03
楼主换个名字
start job --name hdfs3mysql

你是说新建一个job再运行吗？

desehawk · 发表于 2017-4-6 10:48:36

另外的job可能没有完全停止，可以尝试另起一个

tanxl · 发表于 2017-4-6 13:01:36

desehawk 发表于 2017-4-6 10:48
另外的job可能没有完全停止，可以尝试另起一个

重新建的job也不能运行，查看了一下sqoop.log 报下面的错误：

ERROR [org.apache.sqoop.driver.JobManager$UpdateThread.run(JobManager.java:835)] Update thread encountered exception
org.apache.sqoop.common.SqoopException: MAPREDUCE_0003:Can't get RunningJob instance
at org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.update(MapreduceSubmissionEngine.java:484)
at org.apache.sqoop.driver.JobManager.updateSubmission(JobManager.java:714)
at org.apache.sqoop.driver.JobManager.access$600(JobManager.java:62)
at org.apache.sqoop.driver.JobManager$UpdateThread.run(JobManager.java:828)
Caused by: java.io.IOException: java.net.ConnectException: Call From master/192.168.177.130 to master:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:343)
at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:428)
at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:572)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:593)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:591)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:591)
at org.apache.hadoop.mapred.JobClient.getJobInner(JobClient.java:601)
at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:631)
at org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.update(MapreduceSubmissionEngine.java:466)
... 3 more
Caused by: java.net.ConnectException: Call From master/192.168.177.130 to master:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
at org.apache.hadoop.ipc.Client.call(Client.java:1479)
at org.apache.hadoop.ipc.Client.call(Client.java:1412)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy21.getJobReport(Unknown Source)
at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)
at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:324)
... 15 more
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:614)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:712)
at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:375)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1528)
at org.apache.hadoop.ipc.Client.call(Client.java:1451)
... 23 more

sstutu · 发表于 2017-4-6 13:17:57

本帖最后由 sstutu 于 2017-4-6 13:22 编辑

先看看job的状态，应该是没有完全停止
比如jid为1
[mw_shl_code=bash,true]status job --jid 1
[/mw_shl_code]
然后停止
[mw_shl_code=bash,true]stop job --jid 1
[/mw_shl_code]
在启动试试
[mw_shl_code=bash,true]start job --jid 1 --synchronous
[/mw_shl_code]

tanxl · 发表于 2017-4-6 13:55:06

本帖最后由 tanxl 于 2017-4-6 17:06 编辑

sstutu 发表于 2017-4-6 13:17
先看看job的状态，应该是没有完全停止
比如jid为1
[mw_shl_code=bash,true]status job --jid 1

我看了一下 http://master:8088/cluster 作业的运行状态：
application_1491449351953_0002       root       Sqoop: job33       MAPREDUCE       default       Wed Apr 5 21:38:26 -0700 2017       Wed Apr 5 21:41:10 -0700 2017       FINISHED       FAILED
      History       N/A

显示作业是结束了的并且失败的。应该不存在是作业没结束的问题。

sqoop.log日志报错:

ERROR [org.apache.sqoop.driver.JobManager$UpdateThread.run(JobManager.java:835)] Update thread encountered exception
org.apache.sqoop.common.SqoopException: MAPREDUCE_0003:Can't get RunningJob instance
      at org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.update(MapreduceSubmissionEngine.java:484)
      at org.apache.sqoop.driver.JobManager.updateSubmission(JobManager.java:714)
      at org.apache.sqoop.driver.JobManager.access$600(JobManager.java:62)
      at org.apache.sqoop.driver.JobManager$UpdateThread.run(JobManager.java:828)
Caused by: java.io.IOException: java.net.ConnectException: Call From master/192.168.177.130 to master:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
      at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:343)
      at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:428)
      at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:572)
      at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:184)
      at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:593)
      at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:591)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:422)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
      at org.apache.hadoop.mapred.JobClient.getJobUsingCluster(JobClient.java:591)
      at org.apache.hadoop.mapred.JobClient.getJobInner(JobClient.java:601)
      at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:631)
      at org.apache.sqoop.submission.mapreduce.MapreduceSubmissionEngine.update(MapreduceSubmissionEngine.java:466)
      ... 3 more
Caused by: java.net.ConnectException: Call From master/192.168.177.130 to master:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
      at sun.reflect.GeneratedConstructorAccessor61.newInstance(Unknown Source)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
      at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
      at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
      at org.apache.hadoop.ipc.Client.call(Client.java:1479)
      at org.apache.hadoop.ipc.Client.call(Client.java:1412)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
      at com.sun.proxy.$Proxy21.getJobReport(Unknown Source)
      at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)
      at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:497)
      at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:324)
      ... 15 more
Caused by: java.net.ConnectException: Connection refused
      at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
      at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
      at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
      at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
      at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:614)
      at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:712)
      at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:375)
      at org.apache.hadoop.ipc.Client.getConnection(Client.java:1528)
      at org.apache.hadoop.ipc.Client.call(Client.java:1451)
      ... 23 more

再请教一下论坛的事：
about云回复问答为啥还要审核？
这些都很麻烦啊。