chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] Created: (CHUKWA-497) Separating hadoop metrics log types at source for improving primary key lookup
Date Sun, 18 Jul 2010 22:29:50 GMT
Separating hadoop metrics log types at source for improving primary key lookup
------------------------------------------------------------------------------

                 Key: CHUKWA-497
                 URL: https://issues.apache.org/jira/browse/CHUKWA-497
             Project: Chukwa
          Issue Type: Improvement
          Components: data collection
    Affects Versions: 0.4.0, 0.3.0, 0.1.2
         Environment: Java 6, RHEL 5.x
            Reporter: Eric Yang


Chukwa Hadoop metrics log4j appender is using HadoopLog as a generic type to capture all metrics
from various metrics sources. i.e. JobTracker, TaskTracker, Shuffle, Map, Reduce, NameNode,
and DataNode, etc.  It is best to make separation between those types, hence each data type
can have it's own properly assigned key in demux phase by annotation.  

- Shuffle, Map, Reduce, have key associated with task attempts or Job.
- JobTracker or TaskTracker have key associated with a host.
- NameNode or DataNode have key associated with a  host.

HadoopMetricsProcessor should be deprecated in favor of separated demux classes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message