drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "kevin zou (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-5738) Drill query takes 10+ minutes before start executing, excessive Hive metastore queries
Date Wed, 23 Aug 2017 21:45:00 GMT
kevin zou created DRILL-5738:
--------------------------------

             Summary: Drill query takes 10+ minutes before start executing, excessive Hive
metastore queries
                 Key: DRILL-5738
                 URL: https://issues.apache.org/jira/browse/DRILL-5738
             Project: Apache Drill
          Issue Type: Bug
          Components: Query Planning & Optimization
    Affects Versions: 1.6.0
         Environment: mapr 5.2
            Reporter: kevin zou
            Priority: Critical


I ve a Drill query on 14 tables in Hive. The query took a few seconds to execute. However,
the query would stay in "Starting" state for 10+ minutes before execution. 
 
I set up the log to "Debug" mode to figure out what Drill had been doing during the 10+ minutes,
only to find out Drill generated excessive number of meta data queries to hive meta store.
 
 
Although each query took a few micro seconds (meta data cached in memory), the number of queries
was 3438793.
drillbit.log:2017-06-05 18:50:57,201 [26ca5bda-5e87-475a-cd93-17c6957cc3ee:foreman] DEBUG
o.a.d.e.s.hive.HiveMetadataProvider
 - Took 4 ┬Ás to get stats from idm_intel_1x.lu_jde_emp_directory
drillbit.log:2017-06-05 18:50:57,201 [26ca5bda-5e87-475a-cd93-17c6957cc3ee:foreman] DEBUG
o.a.drill.exec.store.hive.HiveScan
- HiveStats: numRows: 15, sizeInBytes: 15
........



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message