chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-278) Improve post process manager and metric data loader to support data loading from pig aggregation
Date Tue, 09 Jun 2009 16:45:07 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12717743#action_12717743
] 

Jerome Boulon commented on CHUKWA-278:
--------------------------------------

-1 
Could you use standard JDBC to get metadata information,  MetricDataLoader should be compatible
with all database vendor. This will simplify the testing side (could use Derby instead of
Mysql) and open some new opportunities like Oracle support for example.
- things like "String query = "select * from " + table + "_template limit 1" " prevent usage
of any other database vendor. 


> Improve post process manager and metric data loader to support data loading from pig
aggregation
> ------------------------------------------------------------------------------------------------
>
>                 Key: CHUKWA-278
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-278
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>          Components: Data Processors
>         Environment: Redhat EL 5.1, Java 6
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>             Fix For: 0.2.0
>
>         Attachments: CHUKWA-278-1.patch, CHUKWA-278.patch
>
>
> The current post process manager loads sequence file to database only.  Ideally, the
post process manager should be able to load data into different data loader.  Hence, DataLoaderFactory
should be defined as an interface or abstract class for third party to implement their own
data loader.  The current implementation of pig aggregator generates down sampled data as
different record type.  There should be a new capability for Metric Data Loader to identify
table partition base on record type.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message