falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkatesh Seetharam (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FALCON-143) Enable Late data handling for hive tables
Date Tue, 08 Oct 2013 18:40:42 GMT
Venkatesh Seetharam created FALCON-143:

             Summary: Enable Late data handling for hive tables
                 Key: FALCON-143
                 URL: https://issues.apache.org/jira/browse/FALCON-143
             Project: Falcon
          Issue Type: Sub-task
    Affects Versions: 0.3
            Reporter: Venkatesh Seetharam

HCat nor Hive APIs expose internal stats about a given partition. The only way to get the
partition size is to get the location of the partition on HDFS and then use globStatus and
contentSummary APIs. 

With the addition of HIVE-5317, this is going to get more complicated with deltas and minor
and major compactions with no locking.

Need to work with hive to see if there will be an API or Falcon needs to understand the structure
of the layout of the data on the file system.

This message was sent by Atlassian JIRA

View raw message