drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4857) When no partition pruning occurs with metadata caching there's a performance regression
Date Wed, 24 Aug 2016 00:43:21 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15433956#comment-15433956
] 

ASF GitHub Bot commented on DRILL-4857:
---------------------------------------

Github user amansinha100 commented on a diff in the pull request:

    https://github.com/apache/drill/pull/575#discussion_r75978179
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/partition/PruneScanRule.java
---
    @@ -520,4 +531,10 @@ private static boolean isQualifiedDirPruning(final TableScan scan)
{
         return false;
       }
     
    +  private static void setNotPruned(MetadataContext metaContext) {
    --- End diff --
    
    yes, I will create the common method. 


> When no partition pruning occurs with metadata caching there's a performance regression
> ---------------------------------------------------------------------------------------
>
>                 Key: DRILL-4857
>                 URL: https://issues.apache.org/jira/browse/DRILL-4857
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Metadata, Query Planning & Optimization
>    Affects Versions: 1.7.0
>            Reporter: Aman Sinha
>            Assignee: Aman Sinha
>             Fix For: 1.8.0
>
>
> After DRILL-4530, we see the (expected) performance improvements in planning time with
metadata cache for cases where partition pruning got applied.  However, in cases where it
did not get applied and for sufficiently large number of files (tested with up to 400K files),
 there's performance regression.  Part of this was addressed by DRILL-4846.   This JIRA is
to track some remaining fixes to address the regression.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message