立即注册 登录
About云-梭伦科技 返回首页

bob007的个人空间 https://www.aboutyun.com/?1328 [收藏] [复制] [分享] [RSS]

日志

[Hive]关于Hive的启动问题

已有 1627 次阅读2015-5-12 23:37

业务背景

用户轨迹工程脚本最近经常报错,报错如下:

SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
FAILED: Error in metadata: MetaException(message:java.lang.RuntimeException: commitTransaction was called but openTransactionCalls = 0. This probably indicates that there are unbalanced calls to openTransaction/commitTransaction)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

搜了下发现,这是hive 0.10.0, 0.11.0, 0.12.0版本的bug,描述如下: 
https://issues.apache.org/jira/browse/HIVE-4996

分析

查看了下集群上hive的配置文件,发现有这些:

$locate hive-site.xml
/etc/hive/conf.cloudera.hive/hive-site.xml
/etc/hive/conf.dist/hive-site.xml
/etc/hive/conf.dist/hive-site.xml_back
/etc/hive-hcatalog/conf.dist/proto-hive-site.xml
/home/pms/db_engine/apache-hive-0.13.1-bin/conf/hive-site.xml
/home/pms/db_engine/apache-hive-0.13.1-bin/hcatalog/etc/hcatalog/proto-hive-site.xml
/home/pms/workspace/kangshantong/spark/source/spark/sql/hive/src/test/resources/data/conf/hive-site.xml
/home/pms/workspace/kangshantong/spark/source/spark/sql/hive/target/scala-2.10/test-classes/data/conf/hive-site.xml
/home/pms/workspace/tangning/software/spark-1.3.1/conf/hive-site.xml
/home/pms/workspace/tangning/software/spark-1.3.1/sql/hive/src/test/resources/data/conf/hive-site.xml
/home/pms/workspace/tangning/software/spark-1.3.1/sql/hive/target/scala-2.10/test-classes/data/conf/hive-site.xml
/home/zhoushugang/apache-hive-0.13.1-bin/conf/hive-site.xml
/home/zhoushugang/apache-hive-0.13.1-bin/hcatalog/etc/hcatalog/proto-hive-site.xml
/usr/local/biserver-ce/pentaho-solutions/system/kettle/plugins/pentaho-big-data-plugin/hadoop-configurations/cdh51/hive-site.xml
/usr/local/biserver-ce/pentaho-solutions/system/kettle/plugins/pentaho-big-data-plugin/hadoop-configurations/hadoop-20/hive-site.xml
/usr/local/cloud/hive/conf/hive-site.xml
/usr/local/cloud/hive/scripts/pdk/conf/hive-site.xml
/usr/local/cloud/hive/src/cloudera/patches/0054-CDH-10280-Change-default-value-for-hive-site.xml-in-.patch
/usr/local/cloud/hive/src/common/src/test/resources/hive-site.xml
/usr/local/cloud/hive/src/conf/hive-site.xml
/usr/local/cloud/hive/src/data/conf/hive-site.xml
/usr/local/cloud/hive/src/pdk/scripts/conf/hive-site.xml

我习惯都是直接使用如下命令启动hive的

hive

在这么多的hive目录里面,如何知道这个命令启动的是以上哪个目录的hive呢?直接打印$HIVE_HOME就知道了

$echo $HIVE_HOME /usr/local/cloud/hive

$HIVE_HOME是配置在/etc/profile里面的

$cat /etc/profile

# /etc/profile


# System wide environment and startup programs, for login setup

# Functions and aliases go in /etc/bashrc


pathmunge () {

        if ! echo $PATH | /bin/egrep -q "(^|:)$1($|:)" ; then

           if [ "$2" = "after" ] ; then

              PATH=$PATH:$1

           else

              PATH=$1:$PATH

           fi

        fi

}


# ksh workaround

if [ -z "$EUID" -a -x /usr/bin/id ]; then 

        EUID=`id -u`

        UID=`id -ru`

fi


# Path manipulation

if [ "$EUID" = "0" ]; then

        pathmunge /sbin

        pathmunge /usr/sbin

        pathmunge /usr/local/sbin

fi


# No core files by default

ulimit -S -c 0 > /dev/null 2>&1

ulimit -n 102400


if [ -x /usr/bin/id ]; then

        USER="`id -un`"

        LOGNAME=$USER

        MAIL="/var/spool/mail/$USER"

fi


HOSTNAME=`/bin/hostname`

HISTSIZE=2000

HISTTIMEFORMAT="%F %T "

JAVA_HOME=/usr/j2sdk

PATH=$JAVA_HOME/bin:$PATH

if [ -z "$INPUTRC" -a ! -f "$HOME/.inputrc" ]; then

    INPUTRC=/etc/inputrc

fi


export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE INPUTRC JAVA_HOME


for i in /etc/profile.d/*.sh ; do

    if [ -r "$i" ]; then

        . $i

    fi

done

unset i

unset pathmung


HIVE_CONFIG_HOME=/usr/local/cloud/hive/conf

HIVE_HOME=/usr/local/cloud/hive

PATH=$HIVE_HOME/bin:$PATH:/opt2/bi_project/software/bin

export FLOW_HOME=/opt2/bi_project/software/conf/flow

export HIVE_CONFIG_HOME=/opt2/bi_project/software/conf/hive


所以直接使用hive命令的话,启动的是/usr/local/cloud/hive目录下的hive,而这个hive的版本是hive 0.10.0,如下:

[pms@yhd-jqhadoop39 /usr/local/cloud/hive/lib]

$ll

total 52480

....

-rw-r--r--  1 search search   93402 Apr 23  2013 hive-beeline-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search    3922 Apr 23  2013 hive-builtins-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search   32642 Apr 23  2013 hive-cli-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search   89389 Apr 23  2013 hive-common-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search  112792 Apr 23  2013 hive-contrib-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search 4596605 Apr 23  2013 hive-exec-0.10.0-cdh4.2.1.jar

-rw-r--r-x  1 root   root     55055 Oct 20  2014 hive-hbase-handler-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search   23884 Apr 23  2013 hive-hwi-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search   55005 Apr 23  2013 hive-hwi-0.10.0-cdh4.2.1.war

-rw-r--r--  1 search search  116852 Apr 23  2013 hive-jdbc-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search 3083290 Apr 23  2013 hive-metastore-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search   14433 Apr 23  2013 hive-pdk-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search  654201 Apr 23  2013 hive-serde-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search 1490629 Apr 23  2013 hive-service-0.10.0-cdh4.2.1.jar

-rw-r--r--  1 search search   98271 Apr 23  2013 hive-shims-0.10.0-cdh4.2.1.jar

....


选择hive版本

从上面locate hive-site.xml命令所列出的hive目录中,有hive 0.13版本的,这个版本已经修复了hive 0.10.0的bug,现在要启动这个0.13版本的hive,而不是0.10.0的hive,命令如下:

/home/pms/db_engine/apache-hive-0.13.1-bin/bin/hive

    路过

    雷人

    握手

    鲜花

    鸡蛋

    评论 (0 个评论)

    facelist doodle 涂鸦板

    您需要登录后才可以评论 登录 | 立即注册

    关闭

    推荐上一条 /2 下一条