chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <>
Subject [jira] Commented: (CHUKWA-422) Rewrite Demux for Hadoop0.20+
Date Mon, 07 Dec 2009 17:13:18 GMT


Jerome Boulon commented on CHUKWA-422:

Since there's a lot of code working on ChukwaRecord, I will keep it during the M/R. My goal
is to have my own OutputFormat that will deal with the output at the last minute. I'll make
the outputFormat a config option. I'm pretty sure that this will be compatible with Tfile
but I'm not so sure about compatibility with Zebra like code since Zebra control his own output

> Rewrite Demux for Hadoop0.20+
> -----------------------------
>                 Key: CHUKWA-422
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>          Components: MR Data Processors
>            Reporter: Jerome Boulon
>            Assignee: Jerome Boulon
> Demux is using the old M/R packages and need to be moved to the new MR API.
> Since I'm now using Hive by default, I will output to Hive table format.
> I will match the RT to a Hive schema and generate an output inline with that schema.
> I will make all of this pluggable but 
> Is there a need to still support the current Chukwa Output format?
> Also since I'm will no longer use the current Chukwa Output format, I no longer need
all Rolling classes?
> Is there a need to still support them?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message