hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-11675) make use of file footer PPD API in ETL strategy or separate strategy
Date Fri, 28 Aug 2015 00:51:49 GMT
Sergey Shelukhin created HIVE-11675:
---------------------------------------

             Summary: make use of file footer PPD API in ETL strategy or separate strategy
                 Key: HIVE-11675
                 URL: https://issues.apache.org/jira/browse/HIVE-11675
             Project: Hive
          Issue Type: Bug
            Reporter: Sergey Shelukhin


Need to take a look at the best flow. It won't be much different if we do filtering metastore
call for each partition. So perhaps we'd need the custom sync point/batching after all.
Or we can make it opportunistic and not fetch any footers unless it can be pushed down to
metastore or fetched from local cache, that way the only slow threaded op is directory listings



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message