flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From George Blazer <gbla...@gmail.com>
Subject Unable to deliver event. Exception follows.
Date Sun, 09 Aug 2015 15:36:04 GMT
Hello there,

My flume seems to be getting stuck in the weird state and I'm not sure what
to do about it.

Metrics endpoint looks like Flume is relatively happy, but I'm seeing the
error below

curl localhost:5653/metrics
{"SINK.s3-sink":{"BatchCompleteCount":"0","ConnectionFailedCount":"0","EventDrainAttemptCount":"0","ConnectionCreatedCount":"0","Type":"SINK","BatchEmptyCount":"0","ConnectionClosedCount":"0","EventDrainSuccessCount":"0","StopTime":"0","StartTime":"1439133984193","BatchUnderflowCount":"0"},"SOURCE.spooling-directory":{"OpenConnectionCount":"0","Type":"SOURCE","AppendBatchAcceptedCount":"0","AppendBatchReceivedCount":"1","EventAcceptedCount":"0","AppendReceivedCount":"0"}

typekit@prod-polka-51145e82:/var/log/flume-ng$ ls -la
/mnt/flume/data/log-1955.meta
-rw-r--r-- 1 root root 47 Jul 26 15:57 /mnt/flume/data/log-1955.meta

The error I see is

2015-08-09 15:31:24,236 (SinkRunner-PollingRunner-DefaultSinkProcessor)
[ERROR -
org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable
to deliver event. Exception follows.
java.lang.IllegalStateException: Channel closed [channel=fileChannel]. Due
to java.io.FileNotFoundException: /mnt/flume/data/log-1955.meta (Too many
open files)
at
org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:340)
at
org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122)
at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:368)
at
org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.FileNotFoundException: /mnt/flume/data/log-1955.meta
(Too many open files)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at
org.apache.flume.channel.file.LogFileV3$SequentialReader.<init>(LogFileV3.java:318)
at
org.apache.flume.channel.file.LogFileFactory.getSequentialReader(LogFileFactory.java:165)
at
org.apache.flume.channel.file.ReplayHandler.replayLog(ReplayHandler.java:264)
at org.apache.flume.channel.file.Log.doReplay(Log.java:519)
at org.apache.flume.channel.file.Log.replay(Log.java:445)
at org.apache.flume.channel.file.FileChannel.start(FileChannel.java:290)
at
org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

Is /metrics basically useless to look at the health of Flume?

Please help.

-George

Mime
View raw message