drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aman Sinha (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3901) Performance regression with doing Explain of COUNT(*) over 100K files
Date Tue, 06 Oct 2015 18:29:26 GMT
Aman Sinha created DRILL-3901:
---------------------------------

             Summary: Performance regression with doing Explain of COUNT(*) over 100K files
                 Key: DRILL-3901
                 URL: https://issues.apache.org/jira/browse/DRILL-3901
             Project: Apache Drill
          Issue Type: Bug
    Affects Versions: 1.1.0
            Reporter: Aman Sinha


We are seeing a performance regression when doing an Explain of SELECT COUNT(*) over 100K
files in a flat directory (no subdirectories) on latest master branch compared to a run that
was done on Sept 26.   Some initial details (I will have more later): 

{code}
master branch on Sept 26
   No metadata cache: 71.452 secs
   With metadata cache: 15.804 secs

Latest master branch 
   No metadata cache: 110 secs
   With metadata cache: 32 secs
{code}

So, both cases show regression.  

[~mehant] and I took an initial look at this and it appears we might be doing the directory
expansion twice.  
   



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message