about云开发-活到老 学到老

 找回密码
 立即注册

QQ登录

只需一步,快速开始

扫一扫,访问微社区

查看: 355|回复: 2

oozie上sparkyarn的任务执行完成后又重新找hdfs://nameservice1/user/hdfs/.sparkS...

[复制链接]

1

主题

2

帖子

16

积分

新手上路

Rank: 1

积分
16
发表于 2018-4-11 12:24:45 | 显示全部楼层 |阅读模式
Application application_1522285686689_0523 failed 2 times due to AM Container for appattempt_1522285686689_0523_000002 exited with exitCode: -1000
For more detailed output, check application tracking page:http://bd003:8088/proxy/application_1522285686689_0523/Then, click on links to logs of each attempt.
Diagnostics: File does not exist: hdfs://nameservice1/user/hdfs/.sparkStaging/application_1522285686689_0523/SparkLearn-1.0-SNAPSHOT.jar
java.io.FileNotFoundException: File does not exist: hdfs://nameservice1/user/hdfs/.sparkStaging/application_1522285686689_0523/SparkLearn-1.0-SNAPSHOT.jar
at org.apache.hadoop.hdfs.DistributedFileSystem$20.doCall(DistributedFileSystem.java:1266)
at org.apache.hadoop.hdfs.DistributedFileSystem$20.doCall(DistributedFileSystem.java:1258)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1258)
at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251)
at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)
at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356)
at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Failing this attempt. Failing the application.

5

主题

37

帖子

198

积分

注册会员

Rank: 2

积分
198
发表于 2018-4-11 15:24:31 | 显示全部楼层
这个肯定跟自己的环境和代码有关系。运行完就是运行完了。
特别代码,仔细核实
回复 支持 1 反对 0

使用道具 举报

1

主题

2

帖子

16

积分

新手上路

Rank: 1

积分
16
 楼主| 发表于 2018-4-11 17:44:59 | 显示全部楼层
已经解决,问题是,执行的main方法必须是scala语言,之前是java语言,改了之后就成功
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

站长推荐上一条 /4 下一条

QQ|小黑屋|about云开发-学问论坛|社区-大数据云技术学习分享平台 ( 京ICP备12023829号

GMT+8, 2018-4-26 15:40 , Processed in 0.436163 second(s), 34 queries , Gzip On.

Powered by Discuz! X3.2 Licensed

© 2001-2013 Comsenz Inc.

快速回复 返回顶部 返回列表