flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hsieh <...@cloudera.com>
Subject Re: java.nio.BufferUnderflowException
Date Mon, 27 Jun 2011 04:40:50 GMT
Hi Eloise,

I'm not familiar with the code your stack trace is showing us.   Based on
some of the namespaces in the code, I think this may be an error in the
Google-internal FlumeJava System.  (There was an publication about this
system a few years back).

This is a mailing list for Apache Flume (an open source log collection
system initially created by Cloudera) for Apache Hadoop (an open source
MapReduce implementation that Yahoo! has contributed to significantly).

Unfortunately, I didn't read the message very carefully when I asked you to
forward it to this list.   Sorry to forward you again, but my guess is that
you may want to send this message to a Google internal mailing list.

Jon

On Sun, Jun 26, 2011 at 7:06 PM, Eloise Carlton <ecarlton@google.com> wrote:

> The following java.nio.BufferUnderflowException happens at the final
>> write to output file stage of a flume pipeline:
>>
>> Jun 25, 2011 12:19:26 PM com.google.mapreduce.ReduceDispatcher
>> reduceCall
>> SEVERE: java.nio.BufferUnderflowException
>>        at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:145)
>>        at
>> com.google.io.file.jni.GoogleChannel.write(GoogleChannel.java:230)
>>        at
>> com.google.pipeline.flume.rt.TextSink.writeToOutputChannel(TextSink.java:
>> 135)
>>        at com.google.pipeline.flume.rt.TextSink.emit(TextSink.java:112)
>>        at com.google.pipeline.flume.rt.TextSink.emit(TextSink.java:23)
>>        at com.google.pipeline.flume.rt.JailableMRSinks
>> $3.emit(JailableMRSinks.java:126)
>>        at com.google.pipeline.flume.rt.JailableMRSinks
>> $3.emit(JailableMRSinks.java:101)
>>        at com.google.pipeline.flume.rt.JailableMSCRReducers
>> $2.reduce(JailableMSCRReducers.java:190)
>>        at
>> com.google.pipeline.flume.rt.MSCRReducer.reduce(MSCRReducer.java:
>> 435)
>>        at
>>
>> com.google.mapreduce.ReduceDispatcher.reduceCallPropagatingExns(ReduceDispatcher.java:
>> 138)
>>        at
>> com.google.mapreduce.ReduceDispatcher.reduceCall(ReduceDispatcher.java:
>> 109)
>>        at
>>
>> com.google.mapreduce.NativeReduceDispatcher.reduceCall(NativeReduceDispatcher.java:
>> 107)
>>
>> F0625 12:19:36.007751 20648 reducerproxyjni.cc:121] Unexpected
>> exception from invocation of Reducer
>> *** Check failure stack trace: ***
>>    @     0x2b66edc41bc6  LogMessage::SendToLog()
>>    @     0x2b66edc3e4b0  LogMessage::Flush()
>>    @     0x2b66edc3e589  LogMessageFatal::~LogMessageFatal()
>>    @     0x2b66eb7ab622  ReducerProxyJNI::Reduce()
>>    @     0x2b66ebae2092  mapreduce::ReduceOutputState::Reduce()
>>    @     0x2b66ebaedfe2  mapreduce::ReduceImpl::DoReduce()
>>    @     0x2b66ebb2ed4a  mapreduce::MR_Worker::Reduce()
>>    @     0x2b66ebb1edb9  _MemberResultCallback_5_0<>::Run()
>>    @     0x2b66ebcff64d
>> mapreduce::EventDispatcher::RunAndDecrQcount()
>>    @     0x2b66ebcfe49d  _MemberResultCallback_2_0<>::Run()
>>    @     0x2b66edb87706  ThreadPoolWorker::Run()
>>    @     0x2b66edb833c2  Thread::ThreadBody()
>>    @     0x2b657596ab9a  start_thread
>> *** SIGABRT received by PID 20445 (TID 20648) from PID 20445; ***
>> F0625 12:19:36.007751 20648 reducerproxyjni.cc:121] Unexpected
>> exception from invocation of Reducer
>>
>>
>>
>> What would cause java.nio.BufferUnderflowException? The output file is a
>> text file on cns.
>>
>> Thanks,
>> Eloise
>>
>


-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

Mime
View raw message