flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Snehal Nagmote <nagmote.sne...@gmail.com>
Subject Spooling Directory Source Stuck in Exception [Serializer has been closed]
Date Mon, 28 Oct 2013 19:05:35 GMT
Hi All,

I am using spooling directory source With Flume 1.4.0 (HDP) . Nginx is set
up in a way that it rotates logs into spool directory every 15 minutes.

I am seeing this exception quite often with flume agents

(org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run:173)
 - Uncaught exception in Runnable
java.lang.IllegalStateException: Serializer has been closed
at
org.apache.flume.serialization.LineDeserializer.ensureOpen(LineDeserializer.java:124)
at
org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:88)
at
org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:221)
at
org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:160)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)


and it just get stuck and needs Flume restart . I found these two tickets
describing same issue,

https://issues.apache.org/jira/browse/FLUME-2160

https://issues.apache.org/jira/browse/FLUME-1887

Has anyone faced similar issues and is there any workaround to avoid this ?



Thanks,
Snehal

Mime
View raw message