分享

hive 运行select count(*) from xx 出错

fish_tx 发表于 2015-1-23 17:49:47 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 4 60569
出错信息:
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
15/01/23 14:42:11 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
15/01/23 14:42:11 INFO Configuration.deprecation: mapred.system.dir is deprecated. Instead, use mapreduce.jobtracker.system.dir
15/01/23 14:42:11 INFO Configuration.deprecation: mapred.local.dir is deprecated. Instead, use mapreduce.cluster.local.dir
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Execution log at: /tmp/root/root_20150123144242_446444a6-1bf4-447a-8740-844ef924eb43.log
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/hive/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Job running in-process (local Hadoop)
Hadoop job information for null: number of mappers: 1; number of reducers: 1
2015-01-23 14:42:46,133 null map = 0%,  reduce = 0%
2015-01-23 14:43:35,007 null map = 100%,  reduce = 0%
2015-01-23 14:43:36,285 null map = 0%,  reduce = 0%
2015-01-23 14:44:04,475 null map = 100%,  reduce = 100%
Ended Job = job_1421996717736_0001 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1421996717736_0001_m_000000 (and more) from job job_1421996717736_0001
Exception in thread "Thread-17" java.lang.IllegalArgumentException: Does not contain a valid host:port authority: local
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:211)
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:163)
        at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:152)
        at org.apache.hadoop.hive.ql.exec.JobTrackerURLResolver.getURL(JobTrackerURLResolver.java:42)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:209)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:92)
        at java.lang.Thread.run(Thread.java:745)
Execution failed with exit status: 2
Obtaining error information

Task failed!
Task ID:
  Stage-1

Logs:

/tmp/root/hive.log
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask

select * from xx; 运行正常
select count(*) from xx;运行出错

请问,有遇到过这个问题吗?

已有(4)人评论

跳转到指定楼层
pengsuyun 发表于 2015-1-23 19:26:06
本帖最后由 pengsuyun 于 2015-1-23 19:27 编辑

检查mapreduce启动没有同时检查hive中的mapreduce配置,哪里用到了host:port,检查host:port的连通性。
回复

使用道具 举报

muyannian 发表于 2015-1-23 19:02:29
hive中
  1. select * from tables
复制代码
是不走mapreduce的。
加上count就不一样了,所以楼主还是从安装配置入手解决问题。
例如:
注意:
hive.aux.jars.path切忌配置正确
  1. <property>
  2.   <name>hive.aux.jars.path</name>
  3.   <value>file:///usr/hive/lib/hive-hbase-handler-0.13.0-SNAPSHOT.jar,file:///usr/hive/lib/protobuf-java-2.5.0.jar,file:///usr/hive/lib/hbase-client-0.96.0-hadoop2.jar,file:///usr/hive/lib/hbase-common-0.96.0-hadoop2.jar,file:///usr/hive/lib/zookeeper-3.4.5.jar,file:///usr/hive/lib/guava-11.0.2.jar</value>
  4. </property>
复制代码
更多参考:
hbase0.96与hive0.12整合高可靠文档及问题总结



回复

使用道具 举报

stark_summer 发表于 2015-1-24 10:01:27
回复

使用道具 举报

zhujun182104906 发表于 2015-5-20 13:40:36
最近报了一个相同的错,先试试吧
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条