用户轨迹工程脚本最近经常报错,报错如下:
搜了下发现,这是hive 0.10.0, 0.11.0, 0.12.0版本的bug,描述如下:
https://issues.apache.org/jira/browse/HIVE-4996
查看了下集群上hive的配置文件,发现有这些:
我习惯都是直接使用如下命令启动hive的
hive在这么多的hive目录里面,如何知道这个命令启动的是以上哪个目录的hive呢?直接打印$HIVE_HOME就知道了
$echo $HIVE_HOME /usr/local/cloud/hive$HIVE_HOME是配置在/etc/profile里面的
$cat /etc/profile
# /etc/profile
# System wide environment and startup programs, for login setup
# Functions and aliases go in /etc/bashrc
pathmunge () {
if ! echo $PATH | /bin/egrep -q "(^|:)$1($|:)" ; then
if [ "$2" = "after" ] ; then
PATH=$PATH:$1
else
PATH=$1:$PATH
fi
fi
}
# ksh workaround
if [ -z "$EUID" -a -x /usr/bin/id ]; then
EUID=`id -u`
UID=`id -ru`
fi
# Path manipulation
if [ "$EUID" = "0" ]; then
pathmunge /sbin
pathmunge /usr/sbin
pathmunge /usr/local/sbin
fi
# No core files by default
ulimit -S -c 0 > /dev/null 2>&1
ulimit -n 102400
if [ -x /usr/bin/id ]; then
USER="`id -un`"
LOGNAME=$USER
MAIL="/var/spool/mail/$USER"
fi
HOSTNAME=`/bin/hostname`
HISTSIZE=2000
HISTTIMEFORMAT="%F %T "
JAVA_HOME=/usr/j2sdk
PATH=$JAVA_HOME/bin:$PATH
if [ -z "$INPUTRC" -a ! -f "$HOME/.inputrc" ]; then
INPUTRC=/etc/inputrc
fi
export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE INPUTRC JAVA_HOME
for i in /etc/profile.d/*.sh ; do
if [ -r "$i" ]; then
. $i
fi
done
unset i
unset pathmung
HIVE_CONFIG_HOME=/usr/local/cloud/hive/conf
HIVE_HOME=/usr/local/cloud/hive
PATH=$HIVE_HOME/bin:$PATH:/opt2/bi_project/software/bin
export FLOW_HOME=/opt2/bi_project/software/conf/flow
export HIVE_CONFIG_HOME=/opt2/bi_project/software/conf/hive
所以直接使用hive命令的话,启动的是/usr/local/cloud/hive目录下的hive,而这个hive的版本是hive 0.10.0,如下:
[pms@yhd-jqhadoop39 /usr/local/cloud/hive/lib]
$ll
total 52480
....
-rw-r--r-- 1 search search 93402 Apr 23 2013 hive-beeline-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 3922 Apr 23 2013 hive-builtins-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 32642 Apr 23 2013 hive-cli-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 89389 Apr 23 2013 hive-common-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 112792 Apr 23 2013 hive-contrib-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 4596605 Apr 23 2013 hive-exec-0.10.0-cdh4.2.1.jar
-rw-r--r-x 1 root root 55055 Oct 20 2014 hive-hbase-handler-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 23884 Apr 23 2013 hive-hwi-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 55005 Apr 23 2013 hive-hwi-0.10.0-cdh4.2.1.war
-rw-r--r-- 1 search search 116852 Apr 23 2013 hive-jdbc-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 3083290 Apr 23 2013 hive-metastore-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 14433 Apr 23 2013 hive-pdk-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 654201 Apr 23 2013 hive-serde-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 1490629 Apr 23 2013 hive-service-0.10.0-cdh4.2.1.jar
-rw-r--r-- 1 search search 98271 Apr 23 2013 hive-shims-0.10.0-cdh4.2.1.jar
....
从上面locate hive-site.xml命令所列出的hive目录中,有hive 0.13版本的,这个版本已经修复了hive 0.10.0的bug,现在要启动这个0.13版本的hive,而不是0.10.0的hive,命令如下:
/home/pms/db_engine/apache-hive-0.13.1-bin/bin/hive