分享

ambari kafka无法启动KeyError: 'listeners'

grinsky 发表于 2016-8-31 18:08:30 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 3 23325
完整的报错如下:
[mw_shl_code=python,true]2016-08-31 05:52:18,834 - Could not determine component version because of the parameters is empty. stack_name: ADH, component_name: None
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/ADH/3.1/services/KAFKA/package/scripts/kafka_broker.py", line 109, in <module>
    KafkaBroker().execute()
  File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/ADH/3.1/services/KAFKA/package/scripts/kafka_broker.py", line 77, in start
    self.configure(env, upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/stacks/ADH/3.1/services/KAFKA/package/scripts/kafka_broker.py", line 46, in configure
    kafka(upgrade_type=upgrade_type)
  File "/var/lib/ambari-agent/cache/stacks/ADH/3.1/services/KAFKA/package/scripts/kafka.py", line 53, in kafka
    listeners = kafka_server_config['listeners'].replace("localhost", params.hostname)
KeyError: 'listeners'[/mw_shl_code]

我修改了下kafka.py
#listeners = kafka_server_config['listeners'].replace("localhost", params.hostname)      原来的配置被我注释掉了
listeners = os.popen('hostname').read().rstrip()


然后启动kafka,启动时是启动成功了,但马上就死掉了。
报错日志如下:
[mw_shl_code=applescript,true][2016-08-31 05:51:18,382] FATAL  (kafka.Kafka$)
java.lang.IllegalArgumentException: Error creating broker listeners from 'hadoop1.com': Unable to parse hadoop1.com to a broker endpoint
        at kafka.server.KafkaConfig.validateUniquePortAndProtocol(KafkaConfig.scala:954)
        at kafka.server.KafkaConfig.getListeners(KafkaConfig.scala:973)
        at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:926)
        at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:743)
        at kafka.server.KafkaConfig$.fromProps(KafkaConfig.scala:740)
        at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:28)
        at kafka.Kafka$.main(Kafka.scala:58)
        at kafka.Kafka.main(Kafka.scala)
[/mw_shl_code]


google到的链接:ambari bug
虽然找到了问题,但是看不懂啊,要怎么修复这个BUG呢?
谁能教一下如何阅读使用链接的讯息,及怎么修复BUG?谢谢!!!


已有(3)人评论

跳转到指定楼层
nextuser 发表于 2016-8-31 19:08:57
hadoop1.com可能会被误解析,最好换成hadoop1
回复

使用道具 举报

einhep 发表于 2016-8-31 19:11:24
{ "version":1,
    "jmx_port":9999,
    "timestamp":2233345666,
    "host":"localhost",
   “port”:9092,
    "endpoints": [
            {"plaintext://localhost:9092"},
            {“ssl://localhost:9093”},
            {“kerberos://localhost:9094”}   
]   
}

楼主可以看下endpoints,hosts在这里转换失败了

回复

使用道具 举报

grinsky 发表于 2016-9-1 10:57:31
通过添加print语句,发现在kafka.py中effective_version 是3.1.0.0
所以执行了下面的语句:
[mw_shl_code=python,true]# listeners and advertised.listeners are only added in 2.3.0.0 onwards.
    if effective_version is not None and effective_version != "" and compare_versions(effective_version, '2.3.0.0') >= 0:
      print "AAAA: effective_version: ",effective_version
      print "AAAA: kafka_server_config", kafka_server_config

      listeners = kafka_server_config['listeners'].replace("localhost", params.hostname)
[/mw_shl_code]
effective_version 的定义如下:
effective_version = params.hdp_stack_version if upgrade_type is None else format_hdp_stack_version(params.version)
事实上我并没有使用HDP的stack,而是通过复制hdp的stack(2.0.6)及common service来自定了stack及service用于接管已存在的hadoop平台。
通过修改effective_version,将其写死 effective_version = '2.2.0.0' (小于2.3.0.0即可)
然后覆盖掉ambari agent cache中对应的脚本,即可正常启动kafka了。


额,本人还是个python还没怎么入门的小菜鸟,至于更深层的细节原因,等看完学习手册,有空在查查吧。




回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条