drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kunal Khatua (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-3794) Fragments outlive query execution in certain out of memory cases
Date Wed, 02 Aug 2017 00:48:00 GMT

     [ https://issues.apache.org/jira/browse/DRILL-3794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kunal Khatua updated DRILL-3794:
--------------------------------
    Reviewer: Khurram Faraaz

[~khfaraaz] Can you verify if this issue is resolved with DRILL-5599 (Drill 1.11.0)?

> Fragments outlive query execution in certain out of memory cases
> ----------------------------------------------------------------
>
>                 Key: DRILL-3794
>                 URL: https://issues.apache.org/jira/browse/DRILL-3794
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>            Reporter: Rahul Challapalli
>             Fix For: Future
>
>         Attachments: error.log, jstack.log, lineitem.parquet
>
>
> git.commit.id.abbrev=240a455
> I created the data set by duplicating the attached lineitem.parquet (tpch0.01) file 5000
times into the lineitem folder.
> The below query fails with an out of memory error
> {code}
> 0: jdbc:drill:zk=10.10.100.190:5181> select
> . . . . . . . . . . . . . . . . . .>   n.n_name,
> . . . . . . . . . . . . . . . . . .>   sum(l.l_extendedprice * (1 - l.l_discount))
as revenue
> . . . . . . . . . . . . . . . . . .> from
> . . . . . . . . . . . . . . . . . .>   customer c,
> . . . . . . . . . . . . . . . . . .>   orders o,
> . . . . . . . . . . . . . . . . . .>   lineitem l,
> . . . . . . . . . . . . . . . . . .>   supplier s,
> . . . . . . . . . . . . . . . . . .>   nation n,
> . . . . . . . . . . . . . . . . . .>   region r
> . . . . . . . . . . . . . . . . . .> where
> . . . . . . . . . . . . . . . . . .>   c.c_custkey = o.o_custkey
> . . . . . . . . . . . . . . . . . .>   and l.l_orderkey = o.o_orderkey
> . . . . . . . . . . . . . . . . . .>   and l.l_suppkey = s.s_suppkey
> . . . . . . . . . . . . . . . . . .>   and c.c_nationkey = s.s_nationkey
> . . . . . . . . . . . . . . . . . .>   and s.s_nationkey = n.n_nationkey
> . . . . . . . . . . . . . . . . . .>   and n.n_regionkey = r.r_regionkey
> . . . . . . . . . . . . . . . . . .>   and r.r_name = 'EUROPE'
> . . . . . . . . . . . . . . . . . .>   and o.o_orderdate >= date '1997-01-01'
> . . . . . . . . . . . . . . . . . .>   and o.o_orderdate < date '1997-01-01' +
interval '1' year
> . . . . . . . . . . . . . . . . . .> group by
> . . . . . . . . . . . . . . . . . .>   n.n_name
> . . . . . . . . . . . . . . . . . .> order by
> . . . . . . . . . . . . . . . . . .>   revenue desc;
> java.lang.RuntimeException: java.sql.SQLException: RESOURCE ERROR: One or more nodes
ran out of memory while executing the query.
> Fragment 5:16
> [Error Id: 73a0cd7e-507c-488d-8126-3caa1849a633 on qa-node191.qa.lab:31010]
> 	at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> 	at sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> 	at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> 	at sqlline.SqlLine.print(SqlLine.java:1583)
> 	at sqlline.Commands.execute(Commands.java:852)
> 	at sqlline.Commands.sql(Commands.java:751)
> 	at sqlline.SqlLine.dispatch(SqlLine.java:738)
> 	at sqlline.SqlLine.begin(SqlLine.java:612)
> 	at sqlline.SqlLine.start(SqlLine.java:366)
> 	at sqlline.SqlLine.main(SqlLine.java:259)
> {code}
> Now when I ran a jstack on the drillbit I still see that there are fragments in the waiting
state. I attached the jstack output as well.
> My cluster config
> {code}
> No Of Nodes : 2
> DRILL_MAX_DIRECT_MEMORY="32G"
> DRILL_MAX_HEAP="4G"
> {code}
> Let me know if you need anything



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message