drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3921) Hive LIMIT 1 queries take too long
Date Mon, 12 Oct 2015 18:10:05 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14953499#comment-14953499
] 

ASF GitHub Bot commented on DRILL-3921:
---------------------------------------

Github user jacques-n commented on a diff in the pull request:

    https://github.com/apache/drill/pull/197#discussion_r41784124
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorContextImpl.java
---
    @@ -130,6 +134,60 @@ public OperatorStats getStats() {
         return stats;
       }
     
    +
    --- End diff --
    
    Can we move create a pool for these delegate threads? The default can be empty but we'll
expand as needed. We also need to think about interruptions and errors. It might be better
if the actual method interface is Future<Result> and we allow the implementor pick when
to block. Guava has a abstract checked future that would probably solve most of these needs.


> Hive LIMIT 1 queries take too long
> ----------------------------------
>
>                 Key: DRILL-3921
>                 URL: https://issues.apache.org/jira/browse/DRILL-3921
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>            Reporter: Sudheesh Katkam
>            Assignee: Sudheesh Katkam
>
> Fragment initialization on a Hive table (that is backed by a directory of many files)
can take really long. This is evident through LIMIT 1 queries. The root cause is that the
underlying reader in the HiveRecordReader is initialized when the ctor is called, rather than
when setup is called.
> Two changes need to be made:
> 1) lazily initialize the underlying record reader in HiveRecordReader
> 2) allow for running a callable as a proxy user within an operator (through OperatorContext).
This is required as initialization of the underlying record reader needs to be done as a proxy
user (proxy for owner of the file). Previously, this was handled while creating the record
batch tree.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message