chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <>
Subject [jira] Created: (CHUKWA-286) Retroactive table creation for loading late data arrival
Date Fri, 12 Jun 2009 00:18:07 GMT
Retroactive table creation for loading late data arrival

                 Key: CHUKWA-286
             Project: Hadoop Chukwa
          Issue Type: Improvement
          Components: Data Processors
         Environment: Redhat EL 5.1, Java 6
            Reporter: Eric Yang

The current MDL drops data on the floor if the data lands outside of existing partition. 
Ideally, Metric Data Loader should retroactive create table for late data arrival.  However,
this can only be done if aggregation is moved outside of database.  This is because late data
arrival may have partition already expired.  The newly create partition may not reflect the
complete data for the partition, hence re-aggreagte again with the late arrival data would
be inaccurate.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message