flume 往 hdfs 中传日志的问题

生产者节点已经显示：
2016-09-07 22:42:07,142 (pool-4-thread-1) [INFO - org.apache.flume.client.avro.ReliableSpoolingFileEventReader.rollCurrentFile(ReliableSpoolingFileEventReader.java:348)] Preparing to move file /usr/flume/flume-1.6.0/logs1/test_spool_015.dat to /usr/flume/flume-1.6.0/logs1/test_spool_015.dat.COMPLETED
2016-09-07 22:42:07,142 (pool-4-thread-1) [DEBUG - org.apache.flume.client.avro.ReliableSpoolingFileEventReader.rollCurrentFile(ReliableSpoolingFileEventReader.java:384)] Successfully rolled file /usr/flume/flume-1.6.0/logs1/test_spool_015.dat to /usr/flume/flume-1.6.0/logs1/test_spool_015.dat.COMPLETED


消费者节点已经显示：
2016-09-07 22:41:18,959 (New I/O server boss #1 ([id: 0x2acdbd94, /192.168.0.116:4141])) [INFO - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:171)] [id: 0x3764d92d, /192.168.0.118:39393 => /192.168.0.116:4141] OPEN
2016-09-07 22:41:18,960 (New I/O  worker #1) [INFO - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:171)] [id: 0x3764d92d, /192.168.0.118:39393 => /192.168.0.116:4141] BOUND: /192.168.0.116:4141
2016-09-07 22:41:18,960 (New I/O  worker #1) [INFO - org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:171)] [id: 0x3764d92d, /192.168.0.118:39393 => /192.168.0.116:4141] CONNECTED: /192.168.0.118:39393


但是。。。

查看hdfs文件系统中的文件，却怎么也没有刚传的日志。

？？？

重启flume也无效。
补充：
配置文件——
a1.sources = r1
a1.sinks = k1
a1.channels = c1

# Describe/configure the source
a1.sources.r1.type = avro
a1.sources.r1.bind = slave3
a1.sources.r1.port= 4141
a1.sources.r1.channels = c1
#a1.sources.r1.interceptors=i1
#a1.sources.r1.interceptors.i1.type=static
#a1.sources.r1.interceptors.i1.key=Collector
#a1.sources.r1.interceptors.i1.value=slave1

# Describe the sink
a1.sinks.k1.type = hdfs
a1.sinks.k1.hdfs.path=hdfs://192.168.0.112:9000/test/%Y-%m-%d
a1.sinks.k1.hdfs.fileType=DataStream
a1.sinks.k1.hdfs.writeFormat=Text
a1.sinks.k1.hdfs.batchSize = 2
a1.sinks.k1.hdfs.rollInterval = 10
a1.sinks.k1.hdfs.rollcount = 1
a1.sinks.k1.hdfs.rollsize = 0
a1.sinks.k1.hdfs.filePrefix = logFile.%Y-%m-%d
a1.sinks.k1.hdfs.fileSuffix = .log
a1.sinks.k1.hdfs.useLocalTimeStamp = true
a1.sinks.k1.hdfs.idleTimeout = 30
a1.sinks.k1.channel=c1


# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.keep-alive=1000
a1.channels.c1.transactionCapacity = 100

ccliu1593571 发表于 2016-9-7 22:54
重启flume也无效。
补充：
配置文件——

看看文件是否变大，系统时间跟当前文件获取日期是否同步

arsenduan 发表于 2016-9-8 06:17
看看文件是否变大，系统时间跟当前文件获取日期是否同步

不同步的话会有什么后果？

造成时间不对了。而且如果集群间时间不一致，都会宕机。还需注意这方面

已解决。
解决方案：
修改生产者的配置文件
修改前：a1.sinks.k1.batch-size = 100
修改后：a1.sinks.k1.batch-size = 1