Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Wed, 16 Sep 2015 23:50:46 +0000 (UTC)
From: "Rahul Challapalli (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.12873138.1442447439000.361030.1442447446379@Atlassian.JIRA>
In-Reply-To: <JIRA.12873138.1442447439000@Atlassian.JIRA>
References: <JIRA.12873138.1442447439000@Atlassian.JIRA>
 <JIRA.12873138.1442447439054@arcas>
Subject: [jira] [Created] (DRILL-3794) Fragments outlive query execution in
 certain out of memory cases
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Rahul Challapalli created DRILL-3794:
----------------------------------------

             Summary: Fragments outlive query execution in certain out of memory cases
                 Key: DRILL-3794
                 URL: https://issues.apache.org/jira/browse/DRILL-3794
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
            Reporter: Rahul Challapalli
            Assignee: Chris Westin
             Fix For: 1.2.0


git.commit.id.abbrev=240a455

I created the data set by duplicating the attached lineitem.parquet (tpch0.01) file 5000 times into the lineitem folder.

The below query fails with an out of memory error
{code}
0: jdbc:drill:zk=10.10.100.190:5181> select
. . . . . . . . . . . . . . . . . .>   n.n_name,
. . . . . . . . . . . . . . . . . .>   sum(l.l_extendedprice * (1 - l.l_discount)) as revenue
. . . . . . . . . . . . . . . . . .> from
. . . . . . . . . . . . . . . . . .>   customer c,
. . . . . . . . . . . . . . . . . .>   orders o,
. . . . . . . . . . . . . . . . . .>   lineitem l,
. . . . . . . . . . . . . . . . . .>   supplier s,
. . . . . . . . . . . . . . . . . .>   nation n,
. . . . . . . . . . . . . . . . . .>   region r
. . . . . . . . . . . . . . . . . .> where
. . . . . . . . . . . . . . . . . .>   c.c_custkey = o.o_custkey
. . . . . . . . . . . . . . . . . .>   and l.l_orderkey = o.o_orderkey
. . . . . . . . . . . . . . . . . .>   and l.l_suppkey = s.s_suppkey
. . . . . . . . . . . . . . . . . .>   and c.c_nationkey = s.s_nationkey
. . . . . . . . . . . . . . . . . .>   and s.s_nationkey = n.n_nationkey
. . . . . . . . . . . . . . . . . .>   and n.n_regionkey = r.r_regionkey
. . . . . . . . . . . . . . . . . .>   and r.r_name = 'EUROPE'
. . . . . . . . . . . . . . . . . .>   and o.o_orderdate >= date '1997-01-01'
. . . . . . . . . . . . . . . . . .>   and o.o_orderdate < date '1997-01-01' + interval '1' year
. . . . . . . . . . . . . . . . . .> group by
. . . . . . . . . . . . . . . . . .>   n.n_name
. . . . . . . . . . . . . . . . . .> order by
. . . . . . . . . . . . . . . . . .>   revenue desc;
java.lang.RuntimeException: java.sql.SQLException: RESOURCE ERROR: One or more nodes ran out of memory while executing the query.

Fragment 5:16

[Error Id: 73a0cd7e-507c-488d-8126-3caa1849a633 on qa-node191.qa.lab:31010]
	at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
	at sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
	at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
	at sqlline.SqlLine.print(SqlLine.java:1583)
	at sqlline.Commands.execute(Commands.java:852)
	at sqlline.Commands.sql(Commands.java:751)
	at sqlline.SqlLine.dispatch(SqlLine.java:738)
	at sqlline.SqlLine.begin(SqlLine.java:612)
	at sqlline.SqlLine.start(SqlLine.java:366)
	at sqlline.SqlLine.main(SqlLine.java:259)
{code}

Now when I ran a jstack on the drillbit I still see that there are fragments in the waiting state. I attached the jstack output as well.

My cluster config
{code}
No Of Nodes : 2
DRILL_MAX_DIRECT_MEMORY="32G"
DRILL_MAX_HEAP="4G"
{code}

Let me know if you need anything


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)