drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhishek Girish (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3913) Possible memory leak during CTAS using 30 TB TPC-H dataset
Date Wed, 07 Oct 2015 23:40:27 GMT
Abhishek Girish created DRILL-3913:
--------------------------------------

             Summary: Possible memory leak during CTAS using 30 TB TPC-H dataset
                 Key: DRILL-3913
                 URL: https://issues.apache.org/jira/browse/DRILL-3913
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 1.2.0
         Environment: 47 nodes configured with 32 GB Drill Direct memory
            Reporter: Abhishek Girish


8 CTAS queries were executed sequentially to write TPC-H text data into Parquet. After successfully
writing a few tables, CTAS failed with OOM.

Restarting Drillbits fixed the problem and re-run of pending CTAS queries completed. This
process was done twice in-order to complete all 8 tables to be written. Overall source was
30TB in size. 

Queries attached. Query profile for one of the CTAS which failed is attached. Logs indicated
that the Drillbit was out of Direct Memory. 

Can share more details as required.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message