drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deneche A. Hakim (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3317) when ProtobufLengthDecoder couldn't allocate a new DrillBuf, this error is just logged and nothing else is done
Date Thu, 18 Jun 2015 21:09:00 GMT
Deneche A. Hakim created DRILL-3317:
---------------------------------------

             Summary: when ProtobufLengthDecoder couldn't allocate a new DrillBuf, this error
is just logged and nothing else is done
                 Key: DRILL-3317
                 URL: https://issues.apache.org/jira/browse/DRILL-3317
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - RPC
            Reporter: Deneche A. Hakim
            Assignee: Jacques Nadeau
             Fix For: 1.2.0


Trying to reproduce DRILL-3241 I sometimes get the following error in the logs:
{noformat}
ERROR: Out of memory outside any particular fragment.
	at org.apache.drill.exec.rpc.data.DataResponseHandlerImpl.informOutOfMemory(DataResponseHandlerImpl.java:40)
	at org.apache.drill.exec.rpc.data.DataServer$2.handle(DataServer.java:227)
	at org.apache.drill.exec.rpc.ProtobufLengthDecoder.decode(ProtobufLengthDecoder.java:87)
	at org.apache.drill.exec.rpc.data.DataProtobufLengthDecoder$Server.decode(DataProtobufLengthDecoder.java:52)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:315)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:229)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)

WARN: Failure allocating buffer on incoming stream due to memory limits.  Current Allocation:
1372678764.
	at org.apache.drill.exec.rpc.ProtobufLengthDecoder.decode(ProtobufLengthDecoder.java:85)
	at org.apache.drill.exec.rpc.data.DataProtobufLengthDecoder$Server.decode(DataProtobufLengthDecoder.java:52)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:315)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:229)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
{noformat}

ProtobufLengthDecoder.decode() does call OutOfMemoryHandler.handle() which calls DataResponseHandlerImpl.informOutOfMemory()
which just logs the error in the logs.

If we have fragments waiting for data they will be stuck waiting forever, and the query will
hang (behavior observed in DRILL-3241



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message