分享

Cloudera 5.10.0安装遇到的问题和解决方案

醉半城 发表于 2017-4-19 17:46:27 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 3 26659
本帖最后由 醉半城 于 2017-4-19 17:51 编辑

1. Unable to retrieve non-local non-loopback IP address. Seeing address: cm/127.0.0.1
[mw_shl_code=shell,true]2017-04-18 09:40:29,308 ERROR ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive: Unable to retrieve non-local non-loopback IP address. Seeing address: cm/127.0.0.1. 2017-04-18 09:40:29,308 ERROR ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive failed. Bootup = false2017-04-18 09:40:29,308 ERROR ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive was not able to access CM identity to validate it.2017-04-18 09:40:29,308 ERROR ScmActive-0:com.cloudera.server.cmf.components.ScmActive: ScmActive is deferring the validation to the next run in 15 seconds.2017-04-18 09:40:29,308 WARN ScmActive-0:com.cloudera.enterprise.AbstractWrappedEntityManager: Rolling back transaction that wasn't marked for rollback-only.java.lang.Exception: Non-thrown exception for stack trace.        
at com.cloudera.enterprise.AbstractWrappedEntityManager.close(AbstractWrappedEntityManager.java:161)        
at com.cloudera.cmf.persist.CmfEntityManager.close(CmfEntityManager.java:356)        
at com.cloudera.server.cmf.components.ScmActive.markScmActive(ScmActive.java:224)        
at com.cloudera.server.cmf.components.ScmActive.run(ScmActive.java:87)        
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)        
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)        
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)        
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)        
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)        
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)        
at java.lang.Thread.run(Thread.java:745)[/mw_shl_code]
解决方案:
是由于host文件导致,我的host文件多了一行127.0.0.1   cm,删除这行就OK了

2. Unable to create new directory at /var/lib/cloudera-host-monitor/ts/ts_entity_metadata
[mw_shl_code=shell,true]
2017-04-19 09:44:37,403 INFO com.cloudera.cmon.tstore.leveldb.LDBUtils: Creating directory /var/lib/cloudera-host-monitor/ts/ts_entity_metadata
2017-04-19 09:44:37,404 ERROR com.cloudera.cmon.firehose.Main: Error creating LevelDB timeseries store in directory /var/lib/cloudera-host-monitor/ts
java.io.IOException: Unable to create new directory at /var/lib/cloudera-host-monitor/ts/ts_entity_metadata
        at com.cloudera.cmon.tstore.leveldb.LDBUtils.openVersionedDB(LDBUtils.java:239)
        at com.cloudera.cmon.tstore.leveldb.LDBTimeSeriesMetadataStore.openMetadataDB(LDBTimeSeriesMetadataStore.java:148)
        at com.cloudera.cmon.tstore.leveldb.LDBTimeSeriesMetadataStore.<init>(LDBTimeSeriesMetadataStore.java:138)
        at com.cloudera.cmon.firehose.Main.main(Main.java:452)[/mw_shl_code]
解决方案:
是由于文件夹权限导致,修改文件夹拥有人chown cloudera-scm:cloudera-scm /var/lib/cloudera-host-monitor和/var/lib/cloudera-service-monitor

3. ValueError: too many values to unpack
出现这个问题会导致parcel一直处于分发阶段
[mw_shl_code=applescript,true][19/Apr/2017 14:23:20 +0000] 25170 MainThread agent        INFO     Using parcels directory from server provided value: /opt/cloudera/parcels
[19/Apr/2017 14:23:20 +0000] 25170 MainThread parcel       INFO     Agent does create users/groups and apply file permissions
[19/Apr/2017 14:23:20 +0000] 25170 MainThread parcel_cache INFO     Using /opt/cloudera/parcel-cache for parcel cache
[19/Apr/2017 14:23:20 +0000] 25170 MainThread agent        ERROR    Caught unexpected exception in main loop.
Traceback (most recent call last):
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/agent.py", line 710, in __issue_heartbeat
    self._init_after_first_heartbeat_response(resp_data)
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/agent.py", line 947, in _init_after_first_heartbeat_response
    self.client_configs.load()
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py", line 682, in load
    new_deployed.update(self._lookup_alternatives(fname))
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py", line 432, in _lookup_alternatives
    return self._parse_alternatives(alt_name, out)
  File "/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py", line 444, in _parse_alternatives
    path, _, _, priority_str = line.rstrip().split(" ")
ValueError: too many values to unpack[/mw_shl_code]

解决方案:
修改/usr/lib64/cmf/agent/build/env/lib/python2.7/site-packages/cmf-5.10.0-py2.7.egg/cmf/client_configs.py脚本的第444行代码.
修改为:
[mw_shl_code=java,true]for line in output.splitlines():
      if line.startswith("/"):
        if len(line.rstrip().split(" "))<=4:
          path, _, _, priority_str = line.rstrip().split(" ")

          # Ignore the alternative if it's not managed by CM.
          if CM_MAGIC_PREFIX not in os.path.basename(path):
            continue

          try:
            priority = int(priority_str)
          except ValueError:
            THROTTLED_LOG.info("Failed to parse %s: %s", name, line)

          key = ClientConfigKey(name, path)
          value = ClientConfigValue(priority, self._read_generation(path))
          ret[key] = value

        else:
          pass
    return ret[/mw_shl_code]
具体可以参考:http://blog.csdn.net/qq_23660243/article/details/60870527   感谢这位同学的解决方案.

4. Error, CM server guid updated
[mw_shl_code=shell,true][19/Apr/2017 14:43:41 +0000] 3700 MainThread agent        INFO     Using parcels directory from server provided value: /opt/cloudera/parcels
[19/Apr/2017 14:43:41 +0000] 3700 MainThread agent        INFO     Using parcels directory from server provided value: /opt/cloudera/parcels
[19/Apr/2017 14:43:41 +0000] 3700 MainThread agent        WARNING  Expected user root for /opt/cloudera/parcels but was cloudera-scm
[19/Apr/2017 14:43:41 +0000] 3700 MainThread agent        WARNING  Expected group root for /opt/cloudera/parcels but was cloudera-scm
[19/Apr/2017 14:43:41 +0000] 3700 MainThread parcel       INFO     Agent does create users/groups and apply file permissions
[19/Apr/2017 14:43:41 +0000] 3700 MainThread parcel_cache INFO     Using /opt/cloudera/parcel-cache for parcel cache
[19/Apr/2017 14:43:41 +0000] 3700 MainThread agent        ERROR    Error, CM server guid updated, expected 0fd6eac4-d1dc-4b46-90bb-b58c87fa3d1f, received c9bb909f-e7c0-4d56-b33e-26e3764adae8[/mw_shl_code]
解决方案:
rm -f /var/lib/cloudera-scm-agent/cm_guid 然后 service cloudera-scm-agent restart

5. Failed request to SCM: 302
[mw_shl_code=shell,true]2017-04-19 15:01:36,610 INFO com.cloudera.cmf.BasicScmProxy: Failed request to SCM: 302
2017-04-19 15:01:37,610 INFO com.cloudera.cmf.BasicScmProxy: Authentication to SCM required.
2017-04-19 15:01:37,650 INFO com.cloudera.cmf.BasicScmProxy: Using encrypted credentials for SCM
2017-04-19 15:01:37,653 INFO com.cloudera.cmf.BasicScmProxy: Authenticated to SCM.[/mw_shl_code]

解决方案:
官方回复是
[mw_shl_code=shell,true]The "Failed request to SCM: 302" message occurs when the Host Monitor attempts to communicate to Cloudera Manager but the session has expired.  The Host Monitor acts as a client that authenticates to SCM, so it is subject to session restrictions.
The error can be ignored as the Host Monitor will re-authenticate and we do see that in your case that occurs.
In fact, a code change to make the message an INFO message rather than ERROR is slated for Cloudera Manager 5.7.2 and 5.8.2.[/mw_shl_code]

没啥大的影响,但我用的是5.10.0  ......

6. Unable to find the JDBC database jar on host
这个属于安装oozie和hive时用mysql作为元数据库时遇到的
解决方案:将mysql的连接驱动放到 /usr/share/java/mysql-connector-java.jar

7. 安装Kafka时java.lang.OutOfMemoryError: Java heap space
[mw_shl_code=shell,true]# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="/usr/lib64/cmf/service/common/killparent.sh"
#   Executing /bin/sh -c "/usr/lib64/cmf/service/common/killparent.sh"...
Wed Apr 19 16:15:33 CST 2017
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
Using -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/kafka_kafka-KAFKA_BROKER-c07d94c57862124dad7ce85095d99c9f_pid22663.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh as CSD_JAVA_OPTS
Using /run/cloudera-scm-agent/process/114-kafka-KAFKA_BROKER as conf dir
Using scripts/control.sh as process script
CONF_DIR=/run/cloudera-scm-agent/process/114-kafka-KAFKA_BROKER
CMF_CONF_DIR=/etc/cloudera-scm-agent[/mw_shl_code]

解决方案: 修改Kafka的Java Heap Size of Broker 设置为broker_max_heap_size=256

已有(3)人评论

跳转到指定楼层
醉半城 发表于 2017-4-20 10:33:21

相互学习。
回复

使用道具 举报

lester 发表于 2018-11-27 09:39:20
谢谢分享!按照你的方法果然分发通过了,谢谢。
回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条