Return-Path: X-Original-To: apmail-drill-issues-archive@minotaur.apache.org Delivered-To: apmail-drill-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 566FF18A22 for ; Sat, 23 Apr 2016 01:50:13 +0000 (UTC) Received: (qmail 54142 invoked by uid 500); 23 Apr 2016 01:50:13 -0000 Delivered-To: apmail-drill-issues-archive@drill.apache.org Received: (qmail 54113 invoked by uid 500); 23 Apr 2016 01:50:13 -0000 Mailing-List: contact issues-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list issues@drill.apache.org Received: (qmail 54087 invoked by uid 99); 23 Apr 2016 01:50:13 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 23 Apr 2016 01:50:13 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id C81662C14F7 for ; Sat, 23 Apr 2016 01:50:12 +0000 (UTC) Date: Sat, 23 Apr 2016 01:50:12 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: issues@drill.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (DRILL-3317) when ProtobufLengthDecoder couldn't allocate a new DrillBuf, this error is just logged and nothing else is done MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/DRILL-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15255022#comment-15255022 ] ASF GitHub Bot commented on DRILL-3317: --------------------------------------- Github user jacques-n commented on the pull request: https://github.com/apache/drill/pull/446#issuecomment-213647541 Does this get propated as a connection level failure or an RpcRemoteException. (E.g. do we break connection once this happens?) Seems like it would be better to generate a RpcRemoteException on the sending side to ensure that the connection is maintained. > when ProtobufLengthDecoder couldn't allocate a new DrillBuf, this error is just logged and nothing else is done > --------------------------------------------------------------------------------------------------------------- > > Key: DRILL-3317 > URL: https://issues.apache.org/jira/browse/DRILL-3317 > Project: Apache Drill > Issue Type: Bug > Components: Execution - RPC > Reporter: Deneche A. Hakim > Assignee: Jacques Nadeau > Fix For: 1.7.0 > > > Trying to reproduce DRILL-3241 I sometimes get the following error in the logs: > {noformat} > ERROR: Out of memory outside any particular fragment. > at org.apache.drill.exec.rpc.data.DataResponseHandlerImpl.informOutOfMemory(DataResponseHandlerImpl.java:40) > at org.apache.drill.exec.rpc.data.DataServer$2.handle(DataServer.java:227) > at org.apache.drill.exec.rpc.ProtobufLengthDecoder.decode(ProtobufLengthDecoder.java:87) > at org.apache.drill.exec.rpc.data.DataProtobufLengthDecoder$Server.decode(DataProtobufLengthDecoder.java:52) > at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:315) > at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:229) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) > WARN: Failure allocating buffer on incoming stream due to memory limits. Current Allocation: 1372678764. > at org.apache.drill.exec.rpc.ProtobufLengthDecoder.decode(ProtobufLengthDecoder.java:85) > at org.apache.drill.exec.rpc.data.DataProtobufLengthDecoder$Server.decode(DataProtobufLengthDecoder.java:52) > at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:315) > at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:229) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) > at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324) > at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) > at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339) > {noformat} > ProtobufLengthDecoder.decode() does call OutOfMemoryHandler.handle() which calls DataResponseHandlerImpl.informOutOfMemory() which just logs the error in the logs. > If we have fragments waiting for data they will be stuck waiting forever, and the query will hang (behavior observed in DRILL-3241 -- This message was sent by Atlassian JIRA (v6.3.4#6332)