分享

使用sqoop从mysql向hive导入数据报错

quenlang 2014-9-26 12:01:38 发表于 异常错误 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 9 60637
使用sqoop从mysql向hive导数据时出现出错:Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
[grid@rhel0 ~]$ sqoop import --connect jdbc:mysql://192.168.122.100:3306/sqoop --username sqoop -P --table demo_blog --hive-import

14/09/26 19:47:25 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/opt/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
14/09/26 19:47:25 INFO hive.HiveImport: SLF4J: Found binding in [jar:file:/opt/hbase-0.94.22/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
14/09/26 19:47:25 INFO hive.HiveImport: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
14/09/26 19:47:25 INFO hive.HiveImport: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
14/09/26 19:47:31 INFO hive.HiveImport: Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:346)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
14/09/26 19:47:31 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
14/09/26 19:47:31 INFO hive.HiveImport:     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
14/09/26 19:47:31 INFO hive.HiveImport:     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
14/09/26 19:47:31 INFO hive.HiveImport:     at java.lang.reflect.Method.invoke(Method.java:606)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
14/09/26 19:47:31 INFO hive.HiveImport: Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1412)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)
14/09/26 19:47:31 INFO hive.HiveImport:     ... 7 more
14/09/26 19:47:31 INFO hive.HiveImport: Caused by: java.lang.reflect.InvocationTargetException
14/09/26 19:47:31 INFO hive.HiveImport:     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
14/09/26 19:47:31 INFO hive.HiveImport:     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
14/09/26 19:47:31 INFO hive.HiveImport:     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
14/09/26 19:47:31 INFO hive.HiveImport:     at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410)
14/09/26 19:47:31 INFO hive.HiveImport:     ... 12 more
14/09/26 19:47:31 INFO hive.HiveImport: Caused by: java.lang.NoSuchMethodError: org.apache.thrift.EncodingUtils.setBit(BIZ)B
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.api.PrivilegeGrantInfo.setCreateTimeIsSet(PrivilegeGrantInfo.java:245)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.api.PrivilegeGrantInfo.<init>(PrivilegeGrantInfo.java:163)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles(HiveMetaStore.java:563)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:398)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:356)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:54)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4944)
14/09/26 19:47:31 INFO hive.HiveImport:     at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:171)
14/09/26 19:47:31 INFO hive.HiveImport:     ... 17 more
14/09/26 19:47:31 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Hive exited with status 1
    at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:385)
    at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:335)
    at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:239)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:425)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:502)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)

单独登录hive客户端,使用show  tables正常
[grid@rhel0 ~]$ hive
Logging initialized using configuration in jar:file:/opt/hive-0.13.1/lib/hive-common-0.13.1.jar!/hive-log4j.properties
hive> show tables;
OK
test
Time taken: 0.542 seconds, Fetched: 1 row(s)

使用hive调试模式,其中有错误信息:
[grid@rhel0 ~]$ hive -hiveconf hive.root.logger=DEBUG,console

14/09/26 19:51:58 [main]: DEBUG metastore.HiveMetaStore: admin role already exists
InvalidObjectException(message:Role admin already exists.)
    at org.apache.hadoop.hive.metastore.ObjectStore.addRole(ObjectStore.java:3020)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
    at com.sun.proxy.$Proxy9.addRole(Unknown Source)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles(HiveMetaStore.java:544)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:398)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:356)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:54)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4944)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:171)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
    at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)
    at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
14/09/26 19:51:58 [main]: INFO metastore.HiveMetaStore: Added admin role in metastore
14/09/26 19:51:58 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at:
    org.apache.hadoop.hive.metastore.ObjectStore.addRole(ObjectStore.java:3017)
14/09/26 19:51:58 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at:
    org.apache.hadoop.hive.metastore.ObjectStore.getMRole(ObjectStore.java:3296)
14/09/26 19:51:58 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at:
    org.apache.hadoop.hive.metastore.ObjectStore.getMRole(ObjectStore.java:3302)
14/09/26 19:51:58 [main]: DEBUG metastore.ObjectStore: Rollback transaction, isActive: true at:
    org.apache.hadoop.hive.metastore.ObjectStore.addRole(ObjectStore.java:3030)
14/09/26 19:51:58 [main]: DEBUG metastore.HiveMetaStore: public role already exists
InvalidObjectException(message:Role public already exists.)
    at org.apache.hadoop.hive.metastore.ObjectStore.addRole(ObjectStore.java:3020)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
    at com.sun.proxy.$Proxy9.addRole(Unknown Source)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles(HiveMetaStore.java:553)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:398)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:356)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:54)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4944)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:171)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
    at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)
    at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
14/09/26 19:51:58 [main]: INFO metastore.HiveMetaStore: Added public role in metastore
14/09/26 19:51:58 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at:
    org.apache.hadoop.hive.metastore.ObjectStore.grantPrivileges(ObjectStore.java:3681)
14/09/26 19:51:58 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 2, isActive = true at:
    org.apache.hadoop.hive.metastore.ObjectStore.listPrincipalGlobalGrants(ObjectStore.java:4109)
14/09/26 19:51:58 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 1, isactive true at:
    org.apache.hadoop.hive.metastore.ObjectStore.listPrincipalGlobalGrants(ObjectStore.java:4119)
14/09/26 19:51:58 [main]: DEBUG metastore.ObjectStore: Rollback transaction, isActive: true at:
    org.apache.hadoop.hive.metastore.ObjectStore.grantPrivileges(ObjectStore.java:3880)
14/09/26 19:51:58 [main]: DEBUG metastore.HiveMetaStore: Failed while granting global privs to admin
InvalidObjectException(message:All is already granted by admin)
    at org.apache.hadoop.hive.metastore.ObjectStore.grantPrivileges(ObjectStore.java:3713)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:108)
    at com.sun.proxy.$Proxy9.grantPrivileges(Unknown Source)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultRoles(HiveMetaStore.java:567)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:398)
    at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:356)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:54)
    at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59)
    at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4944)
    at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:171)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
    at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
    at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2453)
    at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2465)
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:340)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
14/09/26 19:51:58 [main]: INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty
14/09/26 19:51:58 [main]: DEBUG security.UserGroupInformation: hadoop login
14/09/26 19:51:58 [main]: DEBUG security.UserGroupInformation: hadoop login commit
14/09/26 19:51:58 [main]: DEBUG security.UserGroupInformation: using local user:UnixPrincipal: grid
14/09/26 19:51:58 [main]: DEBUG security.UserGroupInformation: UGI loginUser:grid (auth:SIMPLE)
14/09/26 19:51:58 [main]: DEBUG hdfs.BlockReaderLocal: dfs.client.use.legacy.blockreader.local = false
14/09/26 19:51:58 [main]: DEBUG hdfs.BlockReaderLocal: dfs.client.read.shortcircuit = false
14/09/26 19:51:58 [main]: DEBUG hdfs.BlockReaderLocal: dfs.client.domain.socket.data.traffic = false
14/09/26 19:51:58 [main]: DEBUG hdfs.BlockReaderLocal: dfs.domain.socket.path =
14/09/26 19:51:58 [main]: DEBUG retry.RetryUtils: multipleLinearRandomRetry = null
14/09/26 19:51:58 [main]: DEBUG ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@6a4a1b05
14/09/26 19:51:58 [main]: DEBUG ipc.Client: getting client out of cache: org.apache.hadoop.ipc.Client@17cd8bab
14/09/26 19:51:59 [Thread-11]: DEBUG unix.DomainSocketWatcher: org.apache.hadoop.net.unix.DomainSocketWatcher$1@3f5fc55e: starting with interruptCheckPeriodMs = 60000
14/09/26 19:51:59 [main]: DEBUG hdfs.BlockReaderLocal: Both short-circuit local reads and UNIX domain socket are disabled.
14/09/26 19:51:59 [main]: INFO session.SessionState: No Tez session required at this point. hive.execution.engine=mr.
14/09/26 19:51:59 [main]: INFO metastore.HiveMetaStore: 0: get_all_databases
14/09/26 19:51:59 [main]: INFO HiveMetaStore.audit: ugi=grid    ip=unknown-ip-addr    cmd=get_all_databases   
14/09/26 19:51:59 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at:
    org.apache.hadoop.hive.metastore.ObjectStore.getDatabases(ObjectStore.java:582)
14/09/26 19:51:59 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at:
    org.apache.hadoop.hive.metastore.ObjectStore.getDatabases(ObjectStore.java:606)
14/09/26 19:51:59 [main]: INFO metastore.HiveMetaStore: 0: get_functions: db=default pat=*
14/09/26 19:51:59 [main]: INFO HiveMetaStore.audit: ugi=grid    ip=unknown-ip-addr    cmd=get_functions: db=default pat=*   
14/09/26 19:51:59 [main]: DEBUG metastore.ObjectStore: Open transaction: count = 1, isActive = true at:
    org.apache.hadoop.hive.metastore.ObjectStore.getFunctions(ObjectStore.java:6629)
14/09/26 19:51:59 [main]: DEBUG metastore.ObjectStore: Commit transaction: count = 0, isactive true at:
    org.apache.hadoop.hive.metastore.ObjectStore.getFunctions(ObjectStore.java:6657)

我的hive-site.xml的配置信息:
<configuration>
<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://192.168.122.100/hive?createDatabaseIfNotExist=true</value>
</property>

<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
</property>

<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
</property>

<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>123456</value>
</property>
</configuration>

hive的元数据是配在mysql中,如下:
mysql> show tables;
+---------------------------+
| Tables_in_hive            |
+---------------------------+
| BUCKETING_COLS            |
| CDS                       |
| COLUMNS_V2                |
| DATABASE_PARAMS           |
| DBS                       |
| FUNCS                     |
| FUNC_RU                   |
| GLOBAL_PRIVS              |
| PARTITIONS                |
| PARTITION_KEYS            |
| PARTITION_KEY_VALS        |
| PARTITION_PARAMS          |
| PART_COL_STATS            |
| ROLES                     |
| SDS                       |
| SD_PARAMS                 |
| SEQUENCE_TABLE            |
| SERDES                    |
| SERDE_PARAMS              |
| SKEWED_COL_NAMES          |
| SKEWED_COL_VALUE_LOC_MAP  |
| SKEWED_STRING_LIST        |
| SKEWED_STRING_LIST_VALUES |
| SKEWED_VALUES             |
| SORT_COLS                 |
| TABLE_PARAMS              |
| TAB_COL_STATS             |
| TBLS                      |
| VERSION                   |
+---------------------------+
29 rows in set (0.01 sec)

网上看了好多报同样错误的,但是解决办法都不能破我遇到的这个问题,社区里面哪位大大遇到过,还望指点一下小弟,跪谢!








已有(9)人评论

跳转到指定楼层
sstutu 发表于 2014-9-27 00:59:19
sqoop import --connect jdbc:mysql://192.168.122.100:3306/sqoop --username sqoop -P --table demo_blog --hive-import
改成下面形式

sqoop import --connect jdbc:mysql://192.168.122.100:3306/sqoop --username sqoop --password --table demo_blog --hive-import --hive-table demo_blog-m 2 --fields-terminated-by "\0001";



回复

使用道具 举报

quenlang 发表于 2014-9-27 09:35:18
sstutu 发表于 2014-9-27 00:59
sqoop import --connect jdbc:mysql://192.168.122.100:3306/sqoop --username sqoop -P --table demo_blog ...

您说的这种形式的我试了一下,还是不好使,请问这个错误是什么原因引起的?谢谢
回复

使用道具 举报

sstutu 发表于 2014-9-27 11:39:39
quenlang 发表于 2014-9-27 09:35
您说的这种形式的我试了一下,还是不好使,请问这个错误是什么原因引起的?谢谢
把hive进程都杀掉,然后重启,在试试
回复

使用道具 举报

desehawk 发表于 2014-9-28 12:49:19
首先查看hadoop进程是否正常

hive-site.xml中用户访问mysql权限问题,
如使用hadoop用户,则需要在mysql数据库中进行授权:
GRANT ALL PRIVILEGES ON *.* TO 'hadoop'@'localhost' WITH GRANT OPTION;

回复

使用道具 举报

sstutu 发表于 2014-9-28 12:59:49
猜测原因是这样的。
由于你在使用sqoop的时候,需要创建表,但是这个涉及到mysql.所以这个问题,你可以尝试在hive中,创建demo_blog 表,这个问题就应该重现了。所以归根结底应该是你的mysql的问题,90%是mysql权限问题。
回复

使用道具 举报

arsenduan 发表于 2015-10-28 19:56:21
楼主这个问题解决了吗
回复

使用道具 举报

credit 发表于 2016-3-17 15:22:00
出现了楼主的问题,请问楼主解决没,急求
回复

使用道具 举报

astuid 发表于 2016-5-3 23:08:19
hive的服务器端有起来么?
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条