chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mac Yang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-131) Collect finished_maps, finished_reduces, failed_maps, failed_reduces in mr_job table
Date Mon, 20 Apr 2009 02:49:47 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12700639#action_12700639
] 

Mac Yang commented on CHUKWA-131:
---------------------------------


All fields should be loaded. Two that I'm not sure about are http_port and state_string.

Possible fields Map Reduce Fields:
FAILED_MAPS 
FAILED_REDUCES 
FINISHED_MAPS 
FINISHED_REDUCES 
FINISH_TIME 
JOBCONF 
JOBID 
JOBNAME 
JOB_PRIORITY 
JOB_STATUS 
LAUNCH_TIME 
SUBMIT_TIME 
TOTAL_MAPS 
TOTAL_REDUCES 
USER 
Counter:FileSystemCounters:FILE_BYTES_READ
Counter:FileSystemCounters:FILE_BYTES_WRITTEN
Counter:FileSystemCounters:HDFS_BYTES_READ
Counter:FileSystemCounters:HDFS_BYTES_WRITTEN
Counter:org.apache.hadoop.mapred.JobInProgress$Counter:TOTAL_LAUNCHED_MAPS
Counter:org.apache.hadoop.mapred.JobInProgress$Counter:TOTAL_LAUNCHED_REDUCES
Counter:org.apache.hadoop.mapred.Task$Counter:COMBINE_INPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:COMBINE_OUTPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:MAP_INPUT_BYTES
Counter:org.apache.hadoop.mapred.Task$Counter:MAP_INPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:MAP_OUTPUT_BYTES
Counter:org.apache.hadoop.mapred.Task$Counter:MAP_OUTPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:REDUCE_INPUT_GROUPS
Counter:org.apache.hadoop.mapred.Task$Counter:REDUCE_INPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:REDUCE_OUTPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:REDUCE_SHUFFLE_BYTES
Counter:org.apache.hadoop.mapred.Task$Counter:SPILLED_RECORDS

Possible Task Fields:

Counter:FileSystemCounters:FILE_BYTES_WRITTEN
Counter:FileSystemCounters:HDFS_BYTES_READ
Counter:org.apache.hadoop.mapred.Task$Counter:COMBINE_INPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:COMBINE_OUTPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:MAP_INPUT_BYTES
Counter:org.apache.hadoop.mapred.Task$Counter:MAP_INPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:MAP_OUTPUT_BYTES
Counter:org.apache.hadoop.mapred.Task$Counter:MAP_OUTPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:REDUCE_INPUT_GROUPS
Counter:org.apache.hadoop.mapred.Task$Counter:REDUCE_INPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:REDUCE_OUTPUT_RECORDS
Counter:org.apache.hadoop.mapred.Task$Counter:REDUCE_SHUFFLE_BYTES
Counter:org.apache.hadoop.mapred.Task$Counter:SPILLED_RECORDS
ERROR 
FINISH_TIME 
HOSTNAME 
HTTP_PORT 
JOBID 
SHUFFLE_FINISHED 
SORT_FINISHED 
SPLITS 
START_TIME 
STATE_STRING 
TASKID 
TASK_ATTEMPT_ID 
TASK_ATTEMPT_TIMES 
TASK_STATUS 
TASK_TYPE 
TRACKER_NAME

> Collect finished_maps, finished_reduces, failed_maps, failed_reduces in mr_job table
> ------------------------------------------------------------------------------------
>
>                 Key: CHUKWA-131
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-131
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>          Components: Data Processors
>    Affects Versions: 0.1.2
>         Environment: Redhat 5.1, Java 6
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>            Priority: Blocker
>             Fix For: 0.1.2
>
>         Attachments: CHUKWA-131.patch
>
>
> The current metrics from map reduce job is missing the following important columns:
> finished_maps
> finished_reduces
> failed_maps
> failed_reduces

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message