flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From terreyshih <terreys...@gmail.com>
Subject Re: flume agent / JVM crashes periodically
Date Sun, 21 Sep 2014 02:20:42 GMT
Yes, I have increased to 1G.  I have also started the jmx remote monitoring on the jvm which
I don’t see any issue.

Do you know when the shut down hook kicks in ?  (under what conditions ?)

thx

On Sep 20, 2014, at 7:01 PM, Sharninder <sharninder@gmail.com> wrote:

> Have you increased the JVM heap size or running with the default 20M only?
> 
> --
> Sharninder
> 
> On 21-Sep-2014, at 3:08 am, terreyshih <terreyshih@gmail.com> wrote:
> 
>> BTW, no exceptions are thrown.  It just started the agent-shutdown-hook.
>> 
>> thx
>> 
>> On Sep 20, 2014, at 2:33 PM, terreyshih <terreyshih@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> I am having a lot of difficulty trying to debug an issue.  The flume server stops
working after a few hours and the JVM shuts down.  The snippet of the message I got in the
log starts with following:
>>> 
>>> 
>>> 20 Sep 2014 12:24:10,899 DEBUG [New I/O  worker #9] (org.hbase.async.RegionClient.decode:1349)
 - ------------------<< LEAVING  DECODE <<------------------ time elapsed: 75us
>>> 20 Sep 2014 12:24:19,942 INFO  [agent-shutdown-hook] (org.apache.flume.lifecycle.LifecycleSupervisor.stop:79)
 - Stopping lifecycle supervisor 13
>>> 20 Sep 2014 12:24:19,943 INFO  [agent-shutdown-hook] (org.apache.flume.instrumentation.MonitoredCounterGroup.stop:149)
 - Component type: CHANNEL, name: myChannelC stopped
>>> 
>>> 
>>> From there it starts to shut down all the channels and eventually the JVM shuts
down.
>>> 
>>> I am running the agent in debug mode and I can see my data coming in correctly
for a couple of hours and the snippet shows, flume enters the “LEAVING DECODE” section
of the flume debug engine.  I am writing the data to HBase sinks by using AsycnHBase.
>>> 
>>> I would appreciate it if someone can help me with this.  This is reproducible
around every 2 hours.
>>> 
>>> When does the agent-shutdown-hook gets triggered ?
>>> 
>>> thanks
>> 


Mime
View raw message