chukwa-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ey...@apache.org
Subject svn commit: r770351 [1/4] - in /hadoop/chukwa: branches/chukwa-0.1/ branches/chukwa-0.1/conf/ branches/chukwa-0.1/src/docs/src/documentation/content/xdocs/ branches/chukwa-0.1/src/java/org/apache/hadoop/chukwa/util/ branches/chukwa-0.1/src/test/org/apa...
Date Thu, 30 Apr 2009 17:45:08 GMT
Author: eyang
Date: Thu Apr 30 17:45:08 2009
New Revision: 770351

URL: http://svn.apache.org/viewvc?rev=770351&view=rev
Log:
CHUKWA-206. Removed hard coded path from configuration files, and sample data. (Eric Yang)

Added:
    hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestJobLogEntry.java   (contents, props changed)
      - copied, changed from r770317, hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLogEntryTest.java
    hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestPsOutput.java   (contents, props changed)
      - copied, changed from r770317, hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/PsOutputTest.java
    hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestJobLogEntry.java
    hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestPsOutput.java
Removed:
    hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLogEntryTest.java
    hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/PsOutputTest.java
    hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLogEntryTest.java
    hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/PsOutputTest.java
Modified:
    hadoop/chukwa/branches/chukwa-0.1/build.xml
    hadoop/chukwa/branches/chukwa-0.1/conf/nagios.properties.template
    hadoop/chukwa/branches/chukwa-0.1/src/docs/src/documentation/content/xdocs/admin.xml
    hadoop/chukwa/branches/chukwa-0.1/src/java/org/apache/hadoop/chukwa/util/DatabaseWriter.java
    hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt
    hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestJobLog.txt
    hadoop/chukwa/trunk/build.xml
    hadoop/chukwa/trunk/conf/hadoop-metrics-log4j.properties
    hadoop/chukwa/trunk/conf/nagios.properties.template
    hadoop/chukwa/trunk/src/docs/src/documentation/content/xdocs/admin.xml
    hadoop/chukwa/trunk/src/java/org/apache/hadoop/chukwa/util/DatabaseWriter.java
    hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt
    hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestJobLog.txt

Modified: hadoop/chukwa/branches/chukwa-0.1/build.xml
URL: http://svn.apache.org/viewvc/hadoop/chukwa/branches/chukwa-0.1/build.xml?rev=770351&r1=770350&r2=770351&view=diff
==============================================================================
--- hadoop/chukwa/branches/chukwa-0.1/build.xml (original)
+++ hadoop/chukwa/branches/chukwa-0.1/build.xml Thu Apr 30 17:45:08 2009
@@ -450,6 +450,11 @@
 				<include name="late-log4j.properties" />
 			</fileset>
 		</copy>
+		<copy todir="${test.build.dir}/classes">
+			<fileset dir="${basedir}/src/test">
+				<include name="**/*.txt" />
+			</fileset>
+		</copy>
 		<delete dir="${test.build.dir}/conf" />
 		<mkdir dir="${test.build.dir}/conf" />
 		<copy file="${conf.dir}/jdbc.conf.template" tofile="${test.build.dir}/conf/jdbc.conf" />
@@ -918,6 +923,7 @@
 			<filterchain>
 				<replacetokens>
 					<token key="TODO-NAGIOS-SERVER" value="${TODO-NAGIOS-SERVER}" />
+					<token key="TODO_CHUKWA_CONF_DIR" value="${rpm.conf.dir}" />
 				</replacetokens>
 			</filterchain>
 		</copy>

Modified: hadoop/chukwa/branches/chukwa-0.1/conf/nagios.properties.template
URL: http://svn.apache.org/viewvc/hadoop/chukwa/branches/chukwa-0.1/conf/nagios.properties.template?rev=770351&r1=770350&r2=770351&view=diff
==============================================================================
--- hadoop/chukwa/branches/chukwa-0.1/conf/nagios.properties.template (original)
+++ hadoop/chukwa/branches/chukwa-0.1/conf/nagios.properties.template Thu Apr 30 17:45:08 2009
@@ -49,7 +49,7 @@
 # If send_nsca was installed on this system, maybe we just want to point to it ....
 #log4j.appender.NAGIOS.ConfigFile=/usr/local/nagios/nsca-2.4/send_nsca.cfg
 # or, we can just include our own version with our application
-log4j.appender.NAGIOS.ConfigFile=/grid/0/gs/conf/chukwa-current/nsca_send_clear.cfg
+log4j.appender.NAGIOS.ConfigFile=@TODO_CHUKWA_CONF_DIR@/nsca_send_clear.cfg
 
 # -------------------------------------------------------------------------------------------
 # Nagios SERVICE_NAME

Modified: hadoop/chukwa/branches/chukwa-0.1/src/docs/src/documentation/content/xdocs/admin.xml
URL: http://svn.apache.org/viewvc/hadoop/chukwa/branches/chukwa-0.1/src/docs/src/documentation/content/xdocs/admin.xml?rev=770351&r1=770350&r2=770351&view=diff
==============================================================================
--- hadoop/chukwa/branches/chukwa-0.1/src/docs/src/documentation/content/xdocs/admin.xml (original)
+++ hadoop/chukwa/branches/chukwa-0.1/src/docs/src/documentation/content/xdocs/admin.xml Thu Apr 30 17:45:08 2009
@@ -240,12 +240,8 @@
 </li></ul> 
 
 <source>
-export JAVA&#95;HOME&#61;/grid/0/java/jdk
-export HADOOP&#95;HOME&#61;/grid/0/hadoop/current
-export nodeActivityCmde&#61;&#34;/grid/0/torque/current/bin/pbsnodes &#34;
-export TORQUE&#95;HOME&#61;/grid/0/torque
-export TORQUE&#95;SERVER&#61;gs302291
-export DOMAIN&#61;inktomisearch.com
+export JAVA&#95;HOME&#61;/path/to/java
+export HADOOP&#95;HOME&#61;/path/to/hadoop
 export chuwaRecordsRepository&#61;&#34;/chukwa/repos/&#34;
 export JDBC&#95;DRIVER&#61;com.mysql.jdbc.Driver
 export JDBC&#95;URL&#95;PREFIX&#61;jdbc:mysql://
@@ -358,8 +354,6 @@
 </li> <li> org.apache.hadoop.chukwa.inputtools.plugin.metrics.Exec top -b -n 1 -c
 </li> <li> org.apache.hadoop.chukwa.inputtools.plugin.metrics.Exec df -l
 </li> <li> org.apache.hadoop.chukwa.inputtools.plugin.metrics.Exec CHUKWA_HOME/bin/../bin/netstat.sh
-</li> <li> org.apache.hadoop.chukwa.inputtools.mdl.torqueDataLoader - (Only exist in the designated data loading node)
-</li> <li> org.apache.hadoop.chukwa.inputtools.plugin.metrics.Exec /grid/0/torque/current/bin/pbsnodes - (Only exist in the designated data loading node)
 </li></ul> 
 <p>The Chukwa agent process name is identified by:</p>
 <ul>
@@ -426,7 +420,7 @@
 Look at agent.log and collector.log file to determine the problems. </p> 
 <p>The most common problem is the log files are growing unbounded. Set up a cron job to remove old log files:  </p>
 <source>
- 0 12 &#42; &#42; &#42; /grid/0/chukwa/tools/expiration.sh 10 !CHUKWA&#95;HOME/var/log nowait
+ 0 12 &#42; &#42; &#42; CHUKWA&#95;HOME/tools/expiration.sh 10 !CHUKWA&#95;HOME/var/log nowait
 </source>     
 <p>This will setup log file expiration for CHUKWA_HOME/var/log for log files older than 10 days.</p>
 </section>

Modified: hadoop/chukwa/branches/chukwa-0.1/src/java/org/apache/hadoop/chukwa/util/DatabaseWriter.java
URL: http://svn.apache.org/viewvc/hadoop/chukwa/branches/chukwa-0.1/src/java/org/apache/hadoop/chukwa/util/DatabaseWriter.java?rev=770351&r1=770350&r2=770351&view=diff
==============================================================================
--- hadoop/chukwa/branches/chukwa-0.1/src/java/org/apache/hadoop/chukwa/util/DatabaseWriter.java (original)
+++ hadoop/chukwa/branches/chukwa-0.1/src/java/org/apache/hadoop/chukwa/util/DatabaseWriter.java Thu Apr 30 17:45:08 2009
@@ -79,32 +79,6 @@
     }
   }
 
-  public DatabaseWriter() {
-    DataConfig mdlConfig = new DataConfig();
-    String jdbc_url = "jdbc:mysql://" + mdlConfig.get("jdbc.host") + "/"
-        + mdlConfig.get("jdbc.db");
-    if (mdlConfig.get("jdbc.user") != null) {
-      jdbc_url = jdbc_url + "?user=" + mdlConfig.get("jdbc.user");
-      if (mdlConfig.get("jdbc.password") != null) {
-        jdbc_url = jdbc_url + "&password=" + mdlConfig.get("jdbc.password");
-      }
-    }
-    try {
-      // The newInstance() call is a work around for some
-      // broken Java implementations
-      DriverManagerUtil.loadDriver().newInstance();
-    } catch (Exception ex) {
-      // handle the error
-      log.error(ex, ex);
-    }
-    try {
-      conn = org.apache.hadoop.chukwa.util.DriverManagerUtil.getConnection(jdbc_url);
-      log.debug("Initialized JDBC URL: " + jdbc_url);
-    } catch (SQLException ex) {
-      log.error(ex, ex);
-    }
-  }
-
   public void execute(String query) {
     try {
       stmt = conn.createStatement();

Modified: hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt
URL: http://svn.apache.org/viewvc/hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt?rev=770351&r1=770350&r2=770351&view=diff
==============================================================================
--- hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt (original)
+++ hadoop/chukwa/branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt Thu Apr 30 17:45:08 2009
@@ -1,71 +1,71 @@
-Job JOBID="job_200903042324_10816" JOBNAME="Chukwa-Demux_20090309_14_59" USER="gmon" SUBMIT_TIME="1236610746839" JOBCONF="hdfs://gs301017/grid/0/tmp/hadoop-gmon/mapred/system/job_200903042324_10816/job.xml"
+Job JOBID="job_200903042324_10816" JOBNAME="Chukwa-Demux_20090309_14_59" USER="gmon" SUBMIT_TIME="1236610746839" JOBCONF="hdfs://test17/tmp/hadoop-gmon/mapred/system/job_200903042324_10816/job.xml"
 Job JOBID="job_200903042324_10816" LAUNCH_TIME="1236610746959" TOTAL_MAPS="9" TOTAL_REDUCES="8"
-Task TASKID="task_200903042324_10816_m_000000" TASK_TYPE="MAP" START_TIME="1236610747184" SPLITS="/default-rack/gs301022.inktomisearch.com,/default-rack/gs301023.inktomisearch.com,/default-rack/gs301024.inktomisearch.com"
-Task TASKID="task_200903042324_10816_m_000001" TASK_TYPE="MAP" START_TIME="1236610747413" SPLITS="/default-rack/gs301022.inktomisearch.com,/default-rack/gs301023.inktomisearch.com,/default-rack/gs301025.inktomisearch.com"
-Task TASKID="task_200903042324_10816_m_000004" TASK_TYPE="MAP" START_TIME="1236610748232" SPLITS="/default-rack/gs301022.inktomisearch.com,/default-rack/gs301024.inktomisearch.com,/default-rack/gs301023.inktomisearch.com"
-Task TASKID="task_200903042324_10816_m_000002" TASK_TYPE="MAP" START_TIME="1236610748565" SPLITS="/default-rack/gs301023.inktomisearch.com,/default-rack/gs301024.inktomisearch.com,/default-rack/gs301019.inktomisearch.com"
-Task TASKID="task_200903042324_10816_m_000005" TASK_TYPE="MAP" START_TIME="1236610748743" SPLITS="/default-rack/gs301024.inktomisearch.com,/default-rack/gs301023.inktomisearch.com,/default-rack/gs301022.inktomisearch.com"
-Task TASKID="task_200903042324_10816_m_000003" TASK_TYPE="MAP" START_TIME="1236610749344" SPLITS="/default-rack/gs301023.inktomisearch.com,/default-rack/gs301024.inktomisearch.com,/default-rack/gs301025.inktomisearch.com"
-Task TASKID="task_200903042324_10816_m_000006" TASK_TYPE="MAP" START_TIME="1236610749514" SPLITS="/default-rack/gs301023.inktomisearch.com,/default-rack/gs301024.inktomisearch.com,/default-rack/gs301022.inktomisearch.com"
-Task TASKID="task_200903042324_10816_m_000008" TASK_TYPE="MAP" START_TIME="1236610750562" SPLITS="/default-rack/gs301023.inktomisearch.com,/default-rack/gs301024.inktomisearch.com,/default-rack/gs301019.inktomisearch.com"
-Task TASKID="task_200903042324_10816_m_000007" TASK_TYPE="MAP" START_TIME="1236610750629" SPLITS="/default-rack/gs301024.inktomisearch.com,/default-rack/gs301022.inktomisearch.com,/default-rack/gs301023.inktomisearch.com"
+Task TASKID="task_200903042324_10816_m_000000" TASK_TYPE="MAP" START_TIME="1236610747184" SPLITS="/default-rack/test22.chukwa.example.com,/default-rack/test23.chukwa.example.com,/default-rack/test24.chukwa.example.com"
+Task TASKID="task_200903042324_10816_m_000001" TASK_TYPE="MAP" START_TIME="1236610747413" SPLITS="/default-rack/test22.chukwa.example.com,/default-rack/test23.chukwa.example.com,/default-rack/test25.chukwa.example.com"
+Task TASKID="task_200903042324_10816_m_000004" TASK_TYPE="MAP" START_TIME="1236610748232" SPLITS="/default-rack/test22.chukwa.example.com,/default-rack/test24.chukwa.example.com,/default-rack/test23.chukwa.example.com"
+Task TASKID="task_200903042324_10816_m_000002" TASK_TYPE="MAP" START_TIME="1236610748565" SPLITS="/default-rack/test23.chukwa.example.com,/default-rack/test24.chukwa.example.com,/default-rack/test19.chukwa.example.com"
+Task TASKID="task_200903042324_10816_m_000005" TASK_TYPE="MAP" START_TIME="1236610748743" SPLITS="/default-rack/test24.chukwa.example.com,/default-rack/test23.chukwa.example.com,/default-rack/test22.chukwa.example.com"
+Task TASKID="task_200903042324_10816_m_000003" TASK_TYPE="MAP" START_TIME="1236610749344" SPLITS="/default-rack/test23.chukwa.example.com,/default-rack/test24.chukwa.example.com,/default-rack/test25.chukwa.example.com"
+Task TASKID="task_200903042324_10816_m_000006" TASK_TYPE="MAP" START_TIME="1236610749514" SPLITS="/default-rack/test23.chukwa.example.com,/default-rack/test24.chukwa.example.com,/default-rack/test22.chukwa.example.com"
+Task TASKID="task_200903042324_10816_m_000008" TASK_TYPE="MAP" START_TIME="1236610750562" SPLITS="/default-rack/test23.chukwa.example.com,/default-rack/test24.chukwa.example.com,/default-rack/test19.chukwa.example.com"
+Task TASKID="task_200903042324_10816_m_000007" TASK_TYPE="MAP" START_TIME="1236610750629" SPLITS="/default-rack/test24.chukwa.example.com,/default-rack/test22.chukwa.example.com,/default-rack/test23.chukwa.example.com"
 Task TASKID="task_200903042324_10816_r_000000" TASK_TYPE="REDUCE" START_TIME="1236610751724" SPLITS=""
 Task TASKID="task_200903042324_10816_r_000001" TASK_TYPE="REDUCE" START_TIME="1236610752508" SPLITS=""
 Task TASKID="task_200903042324_10816_r_000002" TASK_TYPE="REDUCE" START_TIME="1236610752926" SPLITS=""
 Task TASKID="task_200903042324_10816_r_000003" TASK_TYPE="REDUCE" START_TIME="1236610753582" SPLITS=""
 Task TASKID="task_200903042324_10816_r_000004" TASK_TYPE="REDUCE" START_TIME="1236610753892" SPLITS=""
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000000_0" START_TIME="1236610746708" HOSTNAME="/default-rack/gs301024.inktomisearch.com"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000000_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610753383" HOSTNAME="/default-rack/gs301024.inktomisearch.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000000_0" START_TIME="1236610746708" HOSTNAME="/default-rack/test24.chukwa.example.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000000_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610753383" HOSTNAME="/default-rack/test24.chukwa.example.com"
 Task TASKID="task_200903042324_10816_m_000000" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610753383" COUNTERS="File Systems.HDFS bytes read:148926,File Systems.Local bytes written:483456,DemuxError.count:46,DemuxError.JobHistoryCount:46,DemuxMapOutput.SystemMetrics records:1,DemuxMapOutput.Hadoop_mapred_tasktracker records:3,DemuxMapOutput.Hadoop_rpc_metrics records:7,DemuxMapOutput.total records:4530,DemuxMapOutput.Netstat records:1,DemuxMapOutput.JobLogHistoryReduceProcessor records:3376,DemuxMapOutput.JobHistoryInError records:46,DemuxMapOutput.Hadoop_jvm_metrics records:7,DemuxMapOutput.MRJobReduceProcessor records:46,DemuxMapOutput.Hadoop_dfs_datanode records:4,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:3,DemuxMapOutput.SizeVsFinishTime records:1036,DemuxMapInput.Netstat chunks:1,DemuxMapInput.JobHistory chunks:46,DemuxMapInput.total chunks:58,DemuxMapInput.Top chunks:1,DemuxMapInput.HadoopMetricsProcessor chunks:10,Map-Reduce Framework.Combine outpu
 t records:0,Map-Reduce Framework.Map input records:58,Map-Reduce Framework.Spilled Records:4530,Map-Reduce Framework.Map output bytes:3257991,Map-Reduce Framework.Map input bytes:148780,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:4530"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000002_0" START_TIME="1236610749437" HOSTNAME="/default-rack/gs301019.inktomisearch.com"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000002_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610756006" HOSTNAME="/default-rack/gs301019.inktomisearch.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000002_0" START_TIME="1236610749437" HOSTNAME="/default-rack/test19.chukwa.example.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000002_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610756006" HOSTNAME="/default-rack/test19.chukwa.example.com"
 Task TASKID="task_200903042324_10816_m_000002" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610756006" COUNTERS="File Systems.HDFS bytes read:157644,File Systems.Local bytes written:518039,DemuxError.count:46,DemuxError.JobHistoryCount:46,DemuxMapOutput.SystemMetrics records:3,DemuxMapOutput.total records:4792,DemuxMapOutput.Hadoop_rpc_metrics records:9,DemuxMapOutput.JobHistoryInError records:46,DemuxMapOutput.Hadoop_jvm_metrics records:9,DemuxMapOutput.Hadoop_dfs_datanode records:5,DemuxMapOutput.SizeVsFinishTime records:1030,DemuxMapOutput.Hadoop_mapred_tasktracker records:4,DemuxMapOutput.Netstat records:1,DemuxMapOutput.Ps records:187,DemuxMapOutput.JobLogHistoryReduceProcessor records:3448,DemuxMapOutput.MRJobReduceProcessor records:46,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:4,DemuxMapInput.Netstat chunks:1,DemuxMapInput.JobHistory chunks:46,DemuxMapInput.total chunks:64,DemuxMapInput.Top chunks:3,DemuxMapInput.Ps chunks:1,DemuxMapInput.HadoopMetri
 csProcessor chunks:13,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:64,Map-Reduce Framework.Spilled Records:4792,Map-Reduce Framework.Map output bytes:3413801,Map-Reduce Framework.Map input bytes:157498,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:4792"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000008" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000008_0" START_TIME="1236610751206" HOSTNAME="/default-rack/gs301019.inktomisearch.com"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000008" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000008_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610755155" HOSTNAME="/default-rack/gs301019.inktomisearch.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000008" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000008_0" START_TIME="1236610751206" HOSTNAME="/default-rack/test19.chukwa.example.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000008" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000008_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610755155" HOSTNAME="/default-rack/test19.chukwa.example.com"
 Task TASKID="task_200903042324_10816_m_000008" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610755155" COUNTERS="File Systems.HDFS bytes read:16838,File Systems.Local bytes written:328,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:0,Map-Reduce Framework.Spilled Records:0,Map-Reduce Framework.Map output bytes:0,Map-Reduce Framework.Map input bytes:0,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:0"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000001_0" START_TIME="1236610748631" HOSTNAME="/default-rack/gs301023.inktomisearch.com"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000001_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610757038" HOSTNAME="/default-rack/gs301023.inktomisearch.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000001_0" START_TIME="1236610748631" HOSTNAME="/default-rack/test23.chukwa.example.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000001_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610757038" HOSTNAME="/default-rack/test23.chukwa.example.com"
 Task TASKID="task_200903042324_10816_m_000001" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610757038" COUNTERS="File Systems.HDFS bytes read:145463,File Systems.Local bytes written:466054,DemuxError.count:43,DemuxError.JobHistoryCount:43,DemuxMapOutput.SystemMetrics records:14,DemuxMapOutput.total records:4799,DemuxMapOutput.Hadoop_rpc_metrics records:7,DemuxMapOutput.Hadoop_dfs_namenode records:1,DemuxMapOutput.Df records:6,DemuxMapOutput.JobHistoryInError records:43,DemuxMapOutput.Hadoop_jvm_metrics records:7,DemuxMapOutput.Hadoop_dfs_datanode records:3,DemuxMapOutput.SizeVsFinishTime records:1171,DemuxMapOutput.Hadoop_mapred_tasktracker records:3,DemuxMapOutput.Netstat records:2,DemuxMapOutput.JobLogHistoryReduceProcessor records:3494,DemuxMapOutput.Hadoop_dfs_FSNamesystem records:1,DemuxMapOutput.MRJobReduceProcessor records:43,DemuxMapOutput.Hadoop_dfs_FSDirectory records:1,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:3,DemuxMapInput.Df chunks:1,DemuxMa
 pInput.Netstat chunks:2,DemuxMapInput.JobHistory chunks:43,DemuxMapInput.total chunks:55,DemuxMapInput.Iostat chunks:1,DemuxMapInput.Sar chunks:1,DemuxMapInput.HadoopMetricsProcessor chunks:7,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:55,Map-Reduce Framework.Spilled Records:4799,Map-Reduce Framework.Map output bytes:3381089,Map-Reduce Framework.Map input bytes:145317,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:4799"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000007_0" START_TIME="1236610752390" HOSTNAME="/default-rack/gs301025.inktomisearch.com"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000007_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610757772" HOSTNAME="/default-rack/gs301025.inktomisearch.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000007_0" START_TIME="1236610752390" HOSTNAME="/default-rack/test25.chukwa.example.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000007_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610757772" HOSTNAME="/default-rack/test25.chukwa.example.com"
 Task TASKID="task_200903042324_10816_m_000007" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610757772" COUNTERS="File Systems.HDFS bytes read:86853,File Systems.Local bytes written:284287,DemuxError.count:26,DemuxError.JobHistoryCount:26,DemuxMapOutput.SystemMetrics records:16,DemuxMapOutput.Hadoop_mapred_tasktracker records:2,DemuxMapOutput.Hadoop_rpc_metrics records:2,DemuxMapOutput.total records:2862,DemuxMapOutput.JobLogHistoryReduceProcessor records:2124,DemuxMapOutput.JobHistoryInError records:26,DemuxMapOutput.Df records:6,DemuxMapOutput.Hadoop_jvm_metrics records:2,DemuxMapOutput.MRJobReduceProcessor records:26,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:2,DemuxMapOutput.SizeVsFinishTime records:656,DemuxMapInput.Df chunks:1,DemuxMapInput.JobHistory chunks:27,DemuxMapInput.total chunks:36,DemuxMapInput.Top chunks:2,DemuxMapInput.Iostat chunks:1,DemuxMapInput.Sar chunks:1,DemuxMapInput.HadoopMetricsProcessor chunks:4,Map-Reduce Framework.Combine outpu
 t records:0,Map-Reduce Framework.Map input records:36,Map-Reduce Framework.Spilled Records:2862,Map-Reduce Framework.Map output bytes:1925922,Map-Reduce Framework.Map input bytes:86707,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:2862"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000004_0" START_TIME="1236610751253" HOSTNAME="/default-rack/gs301022.inktomisearch.com"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000004_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610761490" HOSTNAME="/default-rack/gs301022.inktomisearch.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000004_0" START_TIME="1236610751253" HOSTNAME="/default-rack/test22.chukwa.example.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000004_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610761490" HOSTNAME="/default-rack/test22.chukwa.example.com"
 Task TASKID="task_200903042324_10816_m_000004" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610761490" COUNTERS="File Systems.HDFS bytes read:118020,File Systems.Local bytes written:403758,DemuxError.count:40,DemuxError.JobHistoryCount:40,DemuxMapOutput.SystemMetrics records:24,DemuxMapOutput.total records:3850,DemuxMapOutput.Hadoop_rpc_metrics records:9,DemuxMapOutput.Hadoop_dfs_namenode records:1,DemuxMapOutput.Df records:18,DemuxMapOutput.JobHistoryInError records:40,DemuxMapOutput.Hadoop_jvm_metrics records:9,DemuxMapOutput.Hadoop_dfs_datanode records:4,DemuxMapOutput.SizeVsFinishTime records:831,DemuxMapOutput.Hadoop_mapred_tasktracker records:4,DemuxMapOutput.Netstat records:2,DemuxMapOutput.JobLogHistoryReduceProcessor records:2862,DemuxMapOutput.Hadoop_dfs_FSNamesystem records:1,DemuxMapOutput.MRJobReduceProcessor records:40,DemuxMapOutput.Hadoop_dfs_FSDirectory records:1,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:4,DemuxMapInput.Df chunks:3,DemuxMa
 pInput.Netstat chunks:2,DemuxMapInput.JobHistory chunks:40,DemuxMapInput.total chunks:60,DemuxMapInput.Iostat chunks:3,DemuxMapInput.Sar chunks:1,DemuxMapInput.HadoopMetricsProcessor chunks:11,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:60,Map-Reduce Framework.Spilled Records:3850,Map-Reduce Framework.Map output bytes:2666885,Map-Reduce Framework.Map input bytes:117874,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:3850"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000005_0" START_TIME="1236610751497" HOSTNAME="/default-rack/gs301022.inktomisearch.com"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000005_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610760777" HOSTNAME="/default-rack/gs301022.inktomisearch.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000005_0" START_TIME="1236610751497" HOSTNAME="/default-rack/test22.chukwa.example.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000005_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610760777" HOSTNAME="/default-rack/test22.chukwa.example.com"
 Task TASKID="task_200903042324_10816_m_000005" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610760777" COUNTERS="File Systems.HDFS bytes read:114621,File Systems.Local bytes written:381193,DemuxError.count:38,DemuxError.JobHistoryCount:38,DemuxMapOutput.SystemMetrics records:34,DemuxMapOutput.total records:3954,DemuxMapOutput.Hadoop_rpc_metrics records:9,DemuxMapOutput.Df records:6,DemuxMapOutput.JobHistoryInError records:38,DemuxMapOutput.Hadoop_jvm_metrics records:10,DemuxMapOutput.Hadoop_dfs_datanode records:7,DemuxMapOutput.SizeVsFinishTime records:907,DemuxMapOutput.Hadoop_mapred_tasktracker records:2,DemuxMapOutput.JobLogHistoryReduceProcessor records:2900,DemuxMapOutput.MRJobReduceProcessor records:38,DemuxMapOutput.Hadoop_dfs_FSDirectory records:1,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:2,DemuxMapInput.Df chunks:1,DemuxMapInput.JobHistory chunks:39,DemuxMapInput.total chunks:59,DemuxMapInput.Top chunks:2,DemuxMapInput.Iostat chunks:1,DemuxMapInpu
 t.Sar chunks:3,DemuxMapInput.HadoopMetricsProcessor chunks:13,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:59,Map-Reduce Framework.Spilled Records:3954,Map-Reduce Framework.Map output bytes:2695878,Map-Reduce Framework.Map input bytes:114475,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:3954"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000003_0" START_TIME="1236610748665" HOSTNAME="/default-rack/gs301024.inktomisearch.com"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000003_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610758101" HOSTNAME="/default-rack/gs301024.inktomisearch.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000003_0" START_TIME="1236610748665" HOSTNAME="/default-rack/test24.chukwa.example.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000003_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610758101" HOSTNAME="/default-rack/test24.chukwa.example.com"
 Task TASKID="task_200903042324_10816_m_000003" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610758101" COUNTERS="File Systems.HDFS bytes read:127102,File Systems.Local bytes written:414364,DemuxError.count:38,DemuxError.JobHistoryCount:38,DemuxMapOutput.SystemMetrics records:29,DemuxMapOutput.total records:3959,DemuxMapOutput.Hadoop_rpc_metrics records:8,DemuxMapOutput.Hadoop_dfs_namenode records:1,DemuxMapOutput.Df records:12,DemuxMapOutput.JobHistoryInError records:38,DemuxMapOutput.Hadoop_jvm_metrics records:8,DemuxMapOutput.Hadoop_dfs_datanode records:1,DemuxMapOutput.SizeVsFinishTime records:901,DemuxMapOutput.Hadoop_mapred_tasktracker records:6,DemuxMapOutput.Netstat records:3,DemuxMapOutput.JobLogHistoryReduceProcessor records:2906,DemuxMapOutput.Hadoop_dfs_FSNamesystem records:1,DemuxMapOutput.MRJobReduceProcessor records:38,DemuxMapOutput.Hadoop_dfs_FSDirectory records:1,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:6,DemuxMapInput.Df chunks:2,DemuxMa
 pInput.Netstat chunks:3,DemuxMapInput.JobHistory chunks:38,DemuxMapInput.total chunks:58,DemuxMapInput.Top chunks:1,DemuxMapInput.Iostat chunks:2,DemuxMapInput.Sar chunks:2,DemuxMapInput.HadoopMetricsProcessor chunks:10,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:58,Map-Reduce Framework.Spilled Records:3959,Map-Reduce Framework.Map output bytes:2816137,Map-Reduce Framework.Map input bytes:126956,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:3959"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000006_0" START_TIME="1236610750315" HOSTNAME="/default-rack/gs301023.inktomisearch.com"
-MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000006_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610759648" HOSTNAME="/default-rack/gs301023.inktomisearch.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000006_0" START_TIME="1236610750315" HOSTNAME="/default-rack/test23.chukwa.example.com"
+MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000006_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610759648" HOSTNAME="/default-rack/test23.chukwa.example.com"
 Task TASKID="task_200903042324_10816_m_000006" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610759648" COUNTERS="File Systems.HDFS bytes read:88012,File Systems.Local bytes written:293210,DemuxError.count:31,DemuxError.JobHistoryCount:31,DemuxMapOutput.SystemMetrics records:29,DemuxMapOutput.total records:2899,DemuxMapOutput.Netstat records:1,DemuxMapOutput.JobLogHistoryReduceProcessor records:2193,DemuxMapOutput.JobHistoryInError records:31,DemuxMapOutput.Df records:12,DemuxMapOutput.Hadoop_jvm_metrics records:1,DemuxMapOutput.MRJobReduceProcessor records:31,DemuxMapOutput.Hadoop_dfs_FSDirectory records:1,DemuxMapOutput.SizeVsFinishTime records:600,DemuxMapInput.Df chunks:2,DemuxMapInput.Netstat chunks:1,DemuxMapInput.JobHistory chunks:31,DemuxMapInput.total chunks:41,DemuxMapInput.Top chunks:1,DemuxMapInput.Iostat chunks:2,DemuxMapInput.Sar chunks:2,DemuxMapInput.HadoopMetricsProcessor chunks:2,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Ma
 p input records:41,Map-Reduce Framework.Spilled Records:2899,Map-Reduce Framework.Map output bytes:1990647,Map-Reduce Framework.Map input bytes:87866,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:2899"
 Task TASKID="task_200903042324_10816_r_000005" TASK_TYPE="REDUCE" START_TIME="1236610763030" SPLITS=""
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000000_0" START_TIME="1236610753164" HOSTNAME="/default-rack/gs301025.inktomisearch.com"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000000_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610764078" SORT_FINISHED="1236610764104" FINISH_TIME="1236610764326" HOSTNAME="/default-rack/gs301025.inktomisearch.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000000_0" START_TIME="1236610753164" HOSTNAME="/default-rack/test25.chukwa.example.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000000_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610764078" SORT_FINISHED="1236610764104" FINISH_TIME="1236610764326" HOSTNAME="/default-rack/test25.chukwa.example.com"
 Task TASKID="task_200903042324_10816_r_000000" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610764326" COUNTERS="File Systems.HDFS bytes written:560,File Systems.Local bytes read:343,File Systems.Local bytes written:343,DemuxReduceOutput.total records:5,DemuxReduceOutput.Hadoop_dfs_FSDirectory records:5,DemuxReduceInput.total distinct keys:1,DemuxReduceInput.Hadoop_dfs_FSDirectory total distinct keys:1,Map-Reduce Framework.Reduce input groups:1,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:5,Map-Reduce Framework.Spilled Records:5,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:5"
 Task TASKID="task_200903042324_10816_r_000006" TASK_TYPE="REDUCE" START_TIME="1236610765899" SPLITS=""
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000001_0" START_TIME="1236610751786" HOSTNAME="/default-rack/gs301024.inktomisearch.com"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000001_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610763776" SORT_FINISHED="1236610763789" FINISH_TIME="1236610764135" HOSTNAME="/default-rack/gs301024.inktomisearch.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000001_0" START_TIME="1236610751786" HOSTNAME="/default-rack/test24.chukwa.example.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000001_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610763776" SORT_FINISHED="1236610763789" FINISH_TIME="1236610764135" HOSTNAME="/default-rack/test24.chukwa.example.com"
 Task TASKID="task_200903042324_10816_r_000001" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610764135" COUNTERS="File Systems.HDFS bytes written:642,File Systems.Local bytes read:486,File Systems.Local bytes written:486,DemuxReduceOutput.total records:10,DemuxReduceOutput.Netstat records:10,DemuxReduceInput.total distinct keys:7,DemuxReduceInput.Netstat total distinct keys:7,Map-Reduce Framework.Reduce input groups:7,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:10,Map-Reduce Framework.Spilled Records:10,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:10"
 Task TASKID="task_200903042324_10816_r_000007" TASK_TYPE="REDUCE" START_TIME="1236610766344" SPLITS=""
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000002_0" START_TIME="1236610753719" HOSTNAME="/default-rack/gs301023.inktomisearch.com"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000002_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610766361" SORT_FINISHED="1236610766376" FINISH_TIME="1236610766711" HOSTNAME="/default-rack/gs301023.inktomisearch.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000002_0" START_TIME="1236610753719" HOSTNAME="/default-rack/test23.chukwa.example.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000002_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610766361" SORT_FINISHED="1236610766376" FINISH_TIME="1236610766711" HOSTNAME="/default-rack/test23.chukwa.example.com"
 Task TASKID="task_200903042324_10816_r_000002" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610766711" COUNTERS="File Systems.HDFS bytes written:2445,File Systems.Local bytes read:3018,File Systems.Local bytes written:3018,DemuxReduceOutput.total records:84,DemuxReduceOutput.Df records:60,DemuxReduceOutput.Hadoop_mapred_shuffleOutput records:24,DemuxReduceInput.Df total distinct keys:10,DemuxReduceInput.total distinct keys:18,DemuxReduceInput.Hadoop_mapred_shuffleOutput total distinct keys:8,Map-Reduce Framework.Reduce input groups:18,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:84,Map-Reduce Framework.Spilled Records:84,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:84"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000005_0" START_TIME="1236610764414" HOSTNAME="/default-rack/gs301025.inktomisearch.com"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000005_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610769547" SORT_FINISHED="1236610769562" FINISH_TIME="1236610769795" HOSTNAME="/default-rack/gs301025.inktomisearch.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000005_0" START_TIME="1236610764414" HOSTNAME="/default-rack/test25.chukwa.example.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000005_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610769547" SORT_FINISHED="1236610769562" FINISH_TIME="1236610769795" HOSTNAME="/default-rack/test25.chukwa.example.com"
 Task TASKID="task_200903042324_10816_r_000005" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610769795" COUNTERS="File Systems.HDFS bytes written:716,File Systems.Local bytes read:548,File Systems.Local bytes written:548,DemuxReduceOutput.total records:3,DemuxReduceOutput.Hadoop_dfs_namenode records:3,DemuxReduceInput.total distinct keys:1,DemuxReduceInput.Hadoop_dfs_namenode total distinct keys:1,Map-Reduce Framework.Reduce input groups:1,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:3,Map-Reduce Framework.Spilled Records:3,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:3"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000004_0" START_TIME="1236610754515" HOSTNAME="/default-rack/gs301019.inktomisearch.com"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000004_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610771019" SORT_FINISHED="1236610771034" FINISH_TIME="1236610772680" HOSTNAME="/default-rack/gs301019.inktomisearch.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000004_0" START_TIME="1236610754515" HOSTNAME="/default-rack/test19.chukwa.example.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000004_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610771019" SORT_FINISHED="1236610771034" FINISH_TIME="1236610772680" HOSTNAME="/default-rack/test19.chukwa.example.com"
 Task TASKID="task_200903042324_10816_r_000004" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610772680" COUNTERS="File Systems.HDFS bytes written:992710,File Systems.Local bytes read:2055500,File Systems.Local bytes written:2055500,DemuxReduceOutput.SystemMetrics records:15,DemuxReduceOutput.total records:347,DemuxReduceOutput.JobHistoryInError records:308,DemuxReduceOutput.Hadoop_dfs_datanode records:24,DemuxReduceInput.total distinct keys:581,DemuxReduceInput.MRJobReduceProcessor total distinct keys:256,DemuxReduceInput.JobHistoryInError total distinct keys:302,DemuxReduceInput.SystemMetrics total distinct keys:15,DemuxReduceInput.Hadoop_dfs_datanode total distinct keys:8,Map-Reduce Framework.Reduce input groups:581,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:347,Map-Reduce Framework.Spilled Records:790,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:790"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000003_0" START_TIME="1236610756493" HOSTNAME="/default-rack/gs301022.inktomisearch.com"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000003_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610774361" SORT_FINISHED="1236610774376" FINISH_TIME="1236610776203" HOSTNAME="/default-rack/gs301022.inktomisearch.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000003_0" START_TIME="1236610756493" HOSTNAME="/default-rack/test22.chukwa.example.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000003_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610774361" SORT_FINISHED="1236610774376" FINISH_TIME="1236610776203" HOSTNAME="/default-rack/test22.chukwa.example.com"
 Task TASKID="task_200903042324_10816_r_000003" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610776203" COUNTERS="File Systems.HDFS bytes written:9507,File Systems.Local bytes read:13207,File Systems.Local bytes written:13207,DemuxReduceOutput.Hadoop_mapred_tasktracker records:24,DemuxReduceOutput.total records:264,DemuxReduceOutput.Ps records:187,DemuxReduceOutput.Hadoop_jvm_metrics records:53,DemuxReduceInput.total distinct keys:19,DemuxReduceInput.Ps total distinct keys:1,DemuxReduceInput.Hadoop_mapred_tasktracker total distinct keys:8,DemuxReduceInput.Hadoop_jvm_metrics total distinct keys:10,Map-Reduce Framework.Reduce input groups:19,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:264,Map-Reduce Framework.Spilled Records:264,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:264"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000006_0" START_TIME="1236610765151" HOSTNAME="/default-rack/gs301024.inktomisearch.com"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000006_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610770754" SORT_FINISHED="1236610770774" FINISH_TIME="1236610774258" HOSTNAME="/default-rack/gs301024.inktomisearch.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000006_0" START_TIME="1236610765151" HOSTNAME="/default-rack/test24.chukwa.example.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000006_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610770754" SORT_FINISHED="1236610770774" FINISH_TIME="1236610774258" HOSTNAME="/default-rack/test24.chukwa.example.com"
 Task TASKID="task_200903042324_10816_r_000006" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610774258" COUNTERS="File Systems.HDFS bytes written:411924,File Systems.Local bytes read:709720,File Systems.Local bytes written:709720,DemuxReduceOutput.total records:7132,DemuxReduceOutput.SizeVsFinishTime records:7132,DemuxReduceInput.total distinct keys:5878,DemuxReduceInput.SizeVsFinishTime total distinct keys:5878,Map-Reduce Framework.Reduce input groups:5878,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:7132,Map-Reduce Framework.Spilled Records:7132,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:7132"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000007_0" START_TIME="1236610767102" HOSTNAME="/default-rack/gs301023.inktomisearch.com"
-ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000007_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610772790" SORT_FINISHED="1236610772807" FINISH_TIME="1236610776148" HOSTNAME="/default-rack/gs301023.inktomisearch.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000007_0" START_TIME="1236610767102" HOSTNAME="/default-rack/test23.chukwa.example.com"
+ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000007_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610772790" SORT_FINISHED="1236610772807" FINISH_TIME="1236610776148" HOSTNAME="/default-rack/test23.chukwa.example.com"
 Task TASKID="task_200903042324_10816_r_000007" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610776148" COUNTERS="File Systems.HDFS bytes written:181366,File Systems.Local bytes read:454838,File Systems.Local bytes written:454838,DemuxReduceOutput.Hadoop_rpc_metrics records:51,DemuxReduceOutput.total records:19314,DemuxReduceOutput.Hadoop_dfs_FSNamesystem records:3,DemuxReduceOutput.MSSRGraph records:19260,DemuxReduceInput.Hadoop_dfs_FSNamesystem total distinct keys:1,DemuxReduceInput.total distinct keys:19271,DemuxReduceInput.JobLogHistoryReduceProcessor total distinct keys:19260,DemuxReduceInput.Hadoop_rpc_metrics total distinct keys:10,Map-Reduce Framework.Reduce input groups:19271,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:19314,Map-Reduce Framework.Spilled Records:23357,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:23357"
-Job JOBID="job_200903042324_10816" FINISH_TIME="1236610776480" JOB_STATUS="SUCCESS" FINISHED_MAPS="9" FINISHED_REDUCES="8" FAILED_MAPS="0" FAILED_REDUCES="0" COUNTERS="DemuxReduceOutput.SystemMetrics records:15,DemuxReduceOutput.Hadoop_rpc_metrics records:51,DemuxReduceOutput.total records:27159,DemuxReduceOutput.Hadoop_dfs_namenode records:3,DemuxReduceOutput.Df records:60,DemuxReduceOutput.JobHistoryInError records:308,DemuxReduceOutput.Hadoop_jvm_metrics records:53,DemuxReduceOutput.Hadoop_dfs_datanode records:24,DemuxReduceOutput.SizeVsFinishTime records:7132,DemuxReduceOutput.Hadoop_mapred_tasktracker records:24,DemuxReduceOutput.Netstat records:10,DemuxReduceOutput.Ps records:187,DemuxReduceOutput.Hadoop_dfs_FSNamesystem records:3,DemuxReduceOutput.MSSRGraph records:19260,DemuxReduceOutput.Hadoop_dfs_FSDirectory records:5,DemuxReduceOutput.Hadoop_mapred_shuffleOutput records:24,File Systems.HDFS bytes read:1003479,File Systems.HDFS bytes written:1599870,File Systems.Lo
 cal bytes read:3237660,File Systems.Local bytes written:6482349,DemuxMapOutput.SystemMetrics records:150,DemuxMapOutput.Hadoop_rpc_metrics records:51,DemuxMapOutput.total records:31645,DemuxMapOutput.Hadoop_dfs_namenode records:3,DemuxMapOutput.Df records:60,DemuxMapOutput.JobHistoryInError records:308,DemuxMapOutput.Hadoop_jvm_metrics records:53,DemuxMapOutput.Hadoop_dfs_datanode records:24,DemuxMapOutput.SizeVsFinishTime records:7132,DemuxMapOutput.Hadoop_mapred_tasktracker records:24,DemuxMapOutput.Netstat records:10,DemuxMapOutput.Ps records:187,DemuxMapOutput.JobLogHistoryReduceProcessor records:23303,DemuxMapOutput.Hadoop_dfs_FSNamesystem records:3,DemuxMapOutput.MRJobReduceProcessor records:308,DemuxMapOutput.Hadoop_dfs_FSDirectory records:5,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:24,DemuxError.count:308,DemuxError.JobHistoryCount:308,Job Counters .Launched reduce tasks:8,Job Counters .Rack-local map tasks:1,Job Counters .Launched map tasks:9,Job Counters .
 Data-local map tasks:8,DemuxReduceInput.total distinct keys:25776,DemuxReduceInput.Hadoop_mapred_tasktracker total distinct keys:8,DemuxReduceInput.Hadoop_dfs_namenode total distinct keys:1,DemuxReduceInput.JobLogHistoryReduceProcessor total distinct keys:19260,DemuxReduceInput.Hadoop_jvm_metrics total distinct keys:10,DemuxReduceInput.Hadoop_dfs_FSDirectory total distinct keys:1,DemuxReduceInput.Hadoop_dfs_FSNamesystem total distinct keys:1,DemuxReduceInput.Df total distinct keys:10,DemuxReduceInput.Hadoop_mapred_shuffleOutput total distinct keys:8,DemuxReduceInput.Ps total distinct keys:1,DemuxReduceInput.MRJobReduceProcessor total distinct keys:256,DemuxReduceInput.JobHistoryInError total distinct keys:302,DemuxReduceInput.Netstat total distinct keys:7,DemuxReduceInput.SystemMetrics total distinct keys:15,DemuxReduceInput.Hadoop_rpc_metrics total distinct keys:10,DemuxReduceInput.Hadoop_dfs_datanode total distinct keys:8,DemuxReduceInput.SizeVsFinishTime total distinct ke
 ys:5878,DemuxMapInput.Df chunks:10,DemuxMapInput.Netstat chunks:10,DemuxMapInput.JobHistory chunks:310,DemuxMapInput.total chunks:431,DemuxMapInput.Iostat chunks:10,DemuxMapInput.Top chunks:10,DemuxMapInput.Ps chunks:1,DemuxMapInput.Sar chunks:10,DemuxMapInput.HadoopMetricsProcessor chunks:70,Map-Reduce Framework.Reduce input groups:25776,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:431,Map-Reduce Framework.Reduce output records:27159,Map-Reduce Framework.Spilled Records:63290,Map-Reduce Framework.Map output bytes:22148350,Map-Reduce Framework.Map input bytes:985473,Map-Reduce Framework.Map output records:31645,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:31645"
\ No newline at end of file
+Job JOBID="job_200903042324_10816" FINISH_TIME="1236610776480" JOB_STATUS="SUCCESS" FINISHED_MAPS="9" FINISHED_REDUCES="8" FAILED_MAPS="0" FAILED_REDUCES="0" COUNTERS="DemuxReduceOutput.SystemMetrics records:15,DemuxReduceOutput.Hadoop_rpc_metrics records:51,DemuxReduceOutput.total records:27159,DemuxReduceOutput.Hadoop_dfs_namenode records:3,DemuxReduceOutput.Df records:60,DemuxReduceOutput.JobHistoryInError records:308,DemuxReduceOutput.Hadoop_jvm_metrics records:53,DemuxReduceOutput.Hadoop_dfs_datanode records:24,DemuxReduceOutput.SizeVsFinishTime records:7132,DemuxReduceOutput.Hadoop_mapred_tasktracker records:24,DemuxReduceOutput.Netstat records:10,DemuxReduceOutput.Ps records:187,DemuxReduceOutput.Hadoop_dfs_FSNamesystem records:3,DemuxReduceOutput.MSSRGraph records:19260,DemuxReduceOutput.Hadoop_dfs_FSDirectory records:5,DemuxReduceOutput.Hadoop_mapred_shuffleOutput records:24,File Systems.HDFS bytes read:1003479,File Systems.HDFS bytes written:1599870,File Systems.Lo
 cal bytes read:3237660,File Systems.Local bytes written:6482349,DemuxMapOutput.SystemMetrics records:150,DemuxMapOutput.Hadoop_rpc_metrics records:51,DemuxMapOutput.total records:31645,DemuxMapOutput.Hadoop_dfs_namenode records:3,DemuxMapOutput.Df records:60,DemuxMapOutput.JobHistoryInError records:308,DemuxMapOutput.Hadoop_jvm_metrics records:53,DemuxMapOutput.Hadoop_dfs_datanode records:24,DemuxMapOutput.SizeVsFinishTime records:7132,DemuxMapOutput.Hadoop_mapred_tasktracker records:24,DemuxMapOutput.Netstat records:10,DemuxMapOutput.Ps records:187,DemuxMapOutput.JobLogHistoryReduceProcessor records:23303,DemuxMapOutput.Hadoop_dfs_FSNamesystem records:3,DemuxMapOutput.MRJobReduceProcessor records:308,DemuxMapOutput.Hadoop_dfs_FSDirectory records:5,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:24,DemuxError.count:308,DemuxError.JobHistoryCount:308,Job Counters .Launched reduce tasks:8,Job Counters .Rack-local map tasks:1,Job Counters .Launched map tasks:9,Job Counters .
 Data-local map tasks:8,DemuxReduceInput.total distinct keys:25776,DemuxReduceInput.Hadoop_mapred_tasktracker total distinct keys:8,DemuxReduceInput.Hadoop_dfs_namenode total distinct keys:1,DemuxReduceInput.JobLogHistoryReduceProcessor total distinct keys:19260,DemuxReduceInput.Hadoop_jvm_metrics total distinct keys:10,DemuxReduceInput.Hadoop_dfs_FSDirectory total distinct keys:1,DemuxReduceInput.Hadoop_dfs_FSNamesystem total distinct keys:1,DemuxReduceInput.Df total distinct keys:10,DemuxReduceInput.Hadoop_mapred_shuffleOutput total distinct keys:8,DemuxReduceInput.Ps total distinct keys:1,DemuxReduceInput.MRJobReduceProcessor total distinct keys:256,DemuxReduceInput.JobHistoryInError total distinct keys:302,DemuxReduceInput.Netstat total distinct keys:7,DemuxReduceInput.SystemMetrics total distinct keys:15,DemuxReduceInput.Hadoop_rpc_metrics total distinct keys:10,DemuxReduceInput.Hadoop_dfs_datanode total distinct keys:8,DemuxReduceInput.SizeVsFinishTime total distinct ke
 ys:5878,DemuxMapInput.Df chunks:10,DemuxMapInput.Netstat chunks:10,DemuxMapInput.JobHistory chunks:310,DemuxMapInput.total chunks:431,DemuxMapInput.Iostat chunks:10,DemuxMapInput.Top chunks:10,DemuxMapInput.Ps chunks:1,DemuxMapInput.Sar chunks:10,DemuxMapInput.HadoopMetricsProcessor chunks:70,Map-Reduce Framework.Reduce input groups:25776,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:431,Map-Reduce Framework.Reduce output records:27159,Map-Reduce Framework.Spilled Records:63290,Map-Reduce Framework.Map output bytes:22148350,Map-Reduce Framework.Map input bytes:985473,Map-Reduce Framework.Map output records:31645,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:31645"



Mime
View raw message