Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Tue, 13 Oct 2015 19:33:05 +0000 (UTC)
From: "Deneche A. Hakim (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.12903184.1444261222000.27678.1444764785540@Atlassian.JIRA>
In-Reply-To: <JIRA.12903184.1444261222000@Atlassian.JIRA>
References: <JIRA.12903184.1444261222000@Atlassian.JIRA>
 <JIRA.12903184.1444261222652@arcas>
Subject: [jira] [Updated] (DRILL-3913) Possible memory leak during CTAS
 using 30 TB TPC-H dataset
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/DRILL-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Deneche A. Hakim updated DRILL-3913:
------------------------------------
    Fix Version/s: 1.3.0

> Possible memory leak during CTAS using 30 TB TPC-H dataset
> ----------------------------------------------------------
>
>                 Key: DRILL-3913
>                 URL: https://issues.apache.org/jira/browse/DRILL-3913
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>    Affects Versions: 1.2.0
>         Environment: 47 nodes configured with 32 GB Drill Direct memory
>            Reporter: Abhishek Girish
>            Assignee: Deneche A. Hakim
>            Priority: Critical
>             Fix For: 1.3.0
>
>         Attachments: create_table_sf30000.txt, drillbit.log.txt, drillbit_attempt3.log.txt, queryProfile_attempt2.json, query_profile.json, sqlline_verbose_error.txt, sysmemory.txt
>
>
> 8 CTAS queries were executed sequentially to write TPC-H text data into Parquet. After successfully writing a few tables, CTAS failed with OOM.
> Restarting Drillbits fixed the problem and re-run of pending CTAS queries completed. This process was done twice in-order to complete all 8 tables to be written. Overall source was 30TB in size. 
> Queries attached. Query profile for one of the CTAS which failed is attached. Logs indicated that the Drillbit was out of Direct Memory. 
> Can share more details as required.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)