drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Challapalli (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3794) Fragments outlive query execution in certain out of memory cases
Date Wed, 16 Sep 2015 23:50:46 GMT
Rahul Challapalli created DRILL-3794:
----------------------------------------

             Summary: Fragments outlive query execution in certain out of memory cases
                 Key: DRILL-3794
                 URL: https://issues.apache.org/jira/browse/DRILL-3794
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
            Reporter: Rahul Challapalli
            Assignee: Chris Westin
             Fix For: 1.2.0


git.commit.id.abbrev=240a455

I created the data set by duplicating the attached lineitem.parquet (tpch0.01) file 5000 times
into the lineitem folder.

The below query fails with an out of memory error
{code}
0: jdbc:drill:zk=10.10.100.190:5181> select
. . . . . . . . . . . . . . . . . .>   n.n_name,
. . . . . . . . . . . . . . . . . .>   sum(l.l_extendedprice * (1 - l.l_discount)) as revenue
. . . . . . . . . . . . . . . . . .> from
. . . . . . . . . . . . . . . . . .>   customer c,
. . . . . . . . . . . . . . . . . .>   orders o,
. . . . . . . . . . . . . . . . . .>   lineitem l,
. . . . . . . . . . . . . . . . . .>   supplier s,
. . . . . . . . . . . . . . . . . .>   nation n,
. . . . . . . . . . . . . . . . . .>   region r
. . . . . . . . . . . . . . . . . .> where
. . . . . . . . . . . . . . . . . .>   c.c_custkey = o.o_custkey
. . . . . . . . . . . . . . . . . .>   and l.l_orderkey = o.o_orderkey
. . . . . . . . . . . . . . . . . .>   and l.l_suppkey = s.s_suppkey
. . . . . . . . . . . . . . . . . .>   and c.c_nationkey = s.s_nationkey
. . . . . . . . . . . . . . . . . .>   and s.s_nationkey = n.n_nationkey
. . . . . . . . . . . . . . . . . .>   and n.n_regionkey = r.r_regionkey
. . . . . . . . . . . . . . . . . .>   and r.r_name = 'EUROPE'
. . . . . . . . . . . . . . . . . .>   and o.o_orderdate >= date '1997-01-01'
. . . . . . . . . . . . . . . . . .>   and o.o_orderdate < date '1997-01-01' + interval
'1' year
. . . . . . . . . . . . . . . . . .> group by
. . . . . . . . . . . . . . . . . .>   n.n_name
. . . . . . . . . . . . . . . . . .> order by
. . . . . . . . . . . . . . . . . .>   revenue desc;
java.lang.RuntimeException: java.sql.SQLException: RESOURCE ERROR: One or more nodes ran out
of memory while executing the query.

Fragment 5:16

[Error Id: 73a0cd7e-507c-488d-8126-3caa1849a633 on qa-node191.qa.lab:31010]
	at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
	at sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
	at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
	at sqlline.SqlLine.print(SqlLine.java:1583)
	at sqlline.Commands.execute(Commands.java:852)
	at sqlline.Commands.sql(Commands.java:751)
	at sqlline.SqlLine.dispatch(SqlLine.java:738)
	at sqlline.SqlLine.begin(SqlLine.java:612)
	at sqlline.SqlLine.start(SqlLine.java:366)
	at sqlline.SqlLine.main(SqlLine.java:259)
{code}

Now when I ran a jstack on the drillbit I still see that there are fragments in the waiting
state. I attached the jstack output as well.

My cluster config
{code}
No Of Nodes : 2
DRILL_MAX_DIRECT_MEMORY="32G"
DRILL_MAX_HEAP="4G"
{code}

Let me know if you need anything



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message