分享

kafka Received -1 when reading from channel, socket has likely been closed.

grinsky 发表于 2016-5-27 18:11:41 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 2 11393
想通过logstash将日志直接发送到kafka报错如下:
log4j, [2016-05-27T17:34:41.268] ERROR: kafka.producer.async.DefaultEventHandler: Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(spark_252)] from broker [ArrayBuffer(id:1,host:101.204.243.113,port:10001, id:2,host:101.204.243.112,port:10001, id:0,host:101.204.243.114,port:10001)] failed
log4j, [2016-05-27T17:34:41.454]  WARN: kafka.client.ClientUtils$: Fetching topic metadata with correlation id 76 for topics [Set(spark_252)] from broker [id:1,host:101.204.243.113,port:10001] failed
java.io.EOFException: Received -1 when reading from channel, socket has likely been closed.
        at kafka.utils.Utils$.read(Utils.scala:381)
        at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
        at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
        at kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)

………………………………………………

已有(2)人评论

跳转到指定楼层
nextuser 发表于 2016-5-27 19:52:27
的log.dir指向的什么目录,是否做过迁移或则跟以前不一样

回复

使用道具 举报

grinsky 发表于 2016-5-30 10:04:39
nextuser 发表于 2016-5-27 19:52
的log.dir指向的什么目录,是否做过迁移或则跟以前不一样

log 目录是/var/log/kafka
并没有改变,启用kerberos后出现这中情况的,
现在我执行
kafka-console-consumer   --topic test --zookeeper hadoop2:2181,hadoop3:2181,hadoop4:2181

报错如下:
[2016-05-30 10:00:18,857] WARN [console-consumer-50488_hadoop3-1464573617238-7028bfa9-leader-finder-thread], Failed to find leader for Set([test,0]) (kafka.consumer.ConsumerFetcherManager$LeaderFinderThread)
kafka.common.BrokerEndPointNotAvailableException: End point PLAINTEXT not found for broker 94
        at kafka.cluster.Broker.getBrokerEndPoint(Broker.scala:141)


回复

使用道具 举报

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条