Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Wed, 7 Oct 2015 16:59:26 +0000 (UTC)
From: "Mehant Baid (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.12902770.1444156161000.47896.1444237166977@Atlassian.JIRA>
In-Reply-To: <JIRA.12902770.1444156161000@Atlassian.JIRA>
References: <JIRA.12902770.1444156161000@Atlassian.JIRA>
 <JIRA.12902770.1444156161998@arcas>
Subject: [jira] [Commented] (DRILL-3901) Performance regression with doing
 Explain of COUNT(*) over 100K files
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/DRILL-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947212#comment-14947212 ] 

Mehant Baid commented on DRILL-3901:
------------------------------------

+1.
Aman, I agree I think its important to have this fix as part of 1.2 since the degradation is close to 50% and is a regression.Performance issue with caching enabled can be addressed later separately.

> Performance regression with doing Explain of COUNT(*) over 100K files
> ---------------------------------------------------------------------
>
>                 Key: DRILL-3901
>                 URL: https://issues.apache.org/jira/browse/DRILL-3901
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.1.0
>            Reporter: Aman Sinha
>            Assignee: Mehant Baid
>         Attachments: 0001-DRILL-3901-Don-t-do-early-expansion-of-directory-in-.patch, 0001-DRILL-3901-Don-t-do-early-expansion-of-directory-in-.patch
>
>
> We are seeing a performance regression when doing an Explain of SELECT COUNT(*) over 100K files in a flat directory (no subdirectories) on latest master branch compared to a run that was done on Sept 26.   Some initial details (I will have more later): 
> {code}
> master branch on Sept 26
>    No metadata cache: 71.452 secs
>    With metadata cache: 15.804 secs
> Latest master branch 
>    No metadata cache: 110 secs
>    With metadata cache: 32 secs
> {code}
> So, both cases show regression.  
> [~mehant] and I took an initial look at this and it appears we might be doing the directory expansion twice.  
>    


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)