Return-Path: Delivered-To: apmail-hadoop-chukwa-commits-archive@minotaur.apache.org Received: (qmail 27875 invoked from network); 15 Apr 2009 17:42:25 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 15 Apr 2009 17:42:25 -0000 Received: (qmail 88825 invoked by uid 500); 15 Apr 2009 17:42:25 -0000 Delivered-To: apmail-hadoop-chukwa-commits-archive@hadoop.apache.org Received: (qmail 88804 invoked by uid 500); 15 Apr 2009 17:42:25 -0000 Mailing-List: contact chukwa-commits-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: chukwa-dev@hadoop.apache.org Delivered-To: mailing list chukwa-commits@hadoop.apache.org Received: (qmail 88782 invoked by uid 99); 15 Apr 2009 17:42:25 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Apr 2009 17:42:25 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 15 Apr 2009 17:42:16 +0000 Received: by eris.apache.org (Postfix, from userid 65534) id C696023889D9; Wed, 15 Apr 2009 17:41:56 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r765282 [2/2] - in /hadoop/chukwa: branches/chukwa-0.1/ branches/chukwa-0.1/src/java/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/ branches/chukwa-0.1/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/ trunk/... Date: Wed, 15 Apr 2009 17:41:55 -0000 To: chukwa-commits@hadoop.apache.org From: eyang@apache.org X-Mailer: svnmailer-1.0.8 Message-Id: <20090415174156.C696023889D9@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Added: hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt URL: http://svn.apache.org/viewvc/hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt?rev=765282&view=auto ============================================================================== --- hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt (added) +++ hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt Wed Apr 15 17:41:55 2009 @@ -0,0 +1,71 @@ +Job JOBID="job_200903042324_10816" JOBNAME="Chukwa-Demux_20090309_14_59" USER="gmon" SUBMIT_TIME="1236610746839" JOBCONF="hdfs://gs301017/grid/0/tmp/hadoop-gmon/mapred/system/job_200903042324_10816/job.xml" +Job JOBID="job_200903042324_10816" LAUNCH_TIME="1236610746959" TOTAL_MAPS="9" TOTAL_REDUCES="8" +Task TASKID="task_200903042324_10816_m_000000" TASK_TYPE="MAP" START_TIME="1236610747184" SPLITS="/default-rack/gs301022.inktomisearch.com,/default-rack/gs301023.inktomisearch.com,/default-rack/gs301024.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000001" TASK_TYPE="MAP" START_TIME="1236610747413" SPLITS="/default-rack/gs301022.inktomisearch.com,/default-rack/gs301023.inktomisearch.com,/default-rack/gs301025.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000004" TASK_TYPE="MAP" START_TIME="1236610748232" SPLITS="/default-rack/gs301022.inktomisearch.com,/default-rack/gs301024.inktomisearch.com,/default-rack/gs301023.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000002" TASK_TYPE="MAP" START_TIME="1236610748565" SPLITS="/default-rack/gs301023.inktomisearch.com,/default-rack/gs301024.inktomisearch.com,/default-rack/gs301019.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000005" TASK_TYPE="MAP" START_TIME="1236610748743" SPLITS="/default-rack/gs301024.inktomisearch.com,/default-rack/gs301023.inktomisearch.com,/default-rack/gs301022.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000003" TASK_TYPE="MAP" START_TIME="1236610749344" SPLITS="/default-rack/gs301023.inktomisearch.com,/default-rack/gs301024.inktomisearch.com,/default-rack/gs301025.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000006" TASK_TYPE="MAP" START_TIME="1236610749514" SPLITS="/default-rack/gs301023.inktomisearch.com,/default-rack/gs301024.inktomisearch.com,/default-rack/gs301022.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000008" TASK_TYPE="MAP" START_TIME="1236610750562" SPLITS="/default-rack/gs301023.inktomisearch.com,/default-rack/gs301024.inktomisearch.com,/default-rack/gs301019.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000007" TASK_TYPE="MAP" START_TIME="1236610750629" SPLITS="/default-rack/gs301024.inktomisearch.com,/default-rack/gs301022.inktomisearch.com,/default-rack/gs301023.inktomisearch.com" +Task TASKID="task_200903042324_10816_r_000000" TASK_TYPE="REDUCE" START_TIME="1236610751724" SPLITS="" +Task TASKID="task_200903042324_10816_r_000001" TASK_TYPE="REDUCE" START_TIME="1236610752508" SPLITS="" +Task TASKID="task_200903042324_10816_r_000002" TASK_TYPE="REDUCE" START_TIME="1236610752926" SPLITS="" +Task TASKID="task_200903042324_10816_r_000003" TASK_TYPE="REDUCE" START_TIME="1236610753582" SPLITS="" +Task TASKID="task_200903042324_10816_r_000004" TASK_TYPE="REDUCE" START_TIME="1236610753892" SPLITS="" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000000_0" START_TIME="1236610746708" HOSTNAME="/default-rack/gs301024.inktomisearch.com" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000000_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610753383" HOSTNAME="/default-rack/gs301024.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000000" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610753383" COUNTERS="File Systems.HDFS bytes read:148926,File Systems.Local bytes written:483456,DemuxError.count:46,DemuxError.JobHistoryCount:46,DemuxMapOutput.SystemMetrics records:1,DemuxMapOutput.Hadoop_mapred_tasktracker records:3,DemuxMapOutput.Hadoop_rpc_metrics records:7,DemuxMapOutput.total records:4530,DemuxMapOutput.Netstat records:1,DemuxMapOutput.JobLogHistoryReduceProcessor records:3376,DemuxMapOutput.JobHistoryInError records:46,DemuxMapOutput.Hadoop_jvm_metrics records:7,DemuxMapOutput.MRJobReduceProcessor records:46,DemuxMapOutput.Hadoop_dfs_datanode records:4,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:3,DemuxMapOutput.SizeVsFinishTime records:1036,DemuxMapInput.Netstat chunks:1,DemuxMapInput.JobHistory chunks:46,DemuxMapInput.total chunks:58,DemuxMapInput.Top chunks:1,DemuxMapInput.HadoopMetricsProcessor chunks:10,Map-Reduce Framework.Combine outpu t records:0,Map-Reduce Framework.Map input records:58,Map-Reduce Framework.Spilled Records:4530,Map-Reduce Framework.Map output bytes:3257991,Map-Reduce Framework.Map input bytes:148780,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:4530" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000002_0" START_TIME="1236610749437" HOSTNAME="/default-rack/gs301019.inktomisearch.com" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000002_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610756006" HOSTNAME="/default-rack/gs301019.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000002" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610756006" COUNTERS="File Systems.HDFS bytes read:157644,File Systems.Local bytes written:518039,DemuxError.count:46,DemuxError.JobHistoryCount:46,DemuxMapOutput.SystemMetrics records:3,DemuxMapOutput.total records:4792,DemuxMapOutput.Hadoop_rpc_metrics records:9,DemuxMapOutput.JobHistoryInError records:46,DemuxMapOutput.Hadoop_jvm_metrics records:9,DemuxMapOutput.Hadoop_dfs_datanode records:5,DemuxMapOutput.SizeVsFinishTime records:1030,DemuxMapOutput.Hadoop_mapred_tasktracker records:4,DemuxMapOutput.Netstat records:1,DemuxMapOutput.Ps records:187,DemuxMapOutput.JobLogHistoryReduceProcessor records:3448,DemuxMapOutput.MRJobReduceProcessor records:46,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:4,DemuxMapInput.Netstat chunks:1,DemuxMapInput.JobHistory chunks:46,DemuxMapInput.total chunks:64,DemuxMapInput.Top chunks:3,DemuxMapInput.Ps chunks:1,DemuxMapInput.HadoopMetri csProcessor chunks:13,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:64,Map-Reduce Framework.Spilled Records:4792,Map-Reduce Framework.Map output bytes:3413801,Map-Reduce Framework.Map input bytes:157498,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:4792" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000008" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000008_0" START_TIME="1236610751206" HOSTNAME="/default-rack/gs301019.inktomisearch.com" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000008" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000008_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610755155" HOSTNAME="/default-rack/gs301019.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000008" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610755155" COUNTERS="File Systems.HDFS bytes read:16838,File Systems.Local bytes written:328,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:0,Map-Reduce Framework.Spilled Records:0,Map-Reduce Framework.Map output bytes:0,Map-Reduce Framework.Map input bytes:0,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:0" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000001_0" START_TIME="1236610748631" HOSTNAME="/default-rack/gs301023.inktomisearch.com" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000001_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610757038" HOSTNAME="/default-rack/gs301023.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000001" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610757038" COUNTERS="File Systems.HDFS bytes read:145463,File Systems.Local bytes written:466054,DemuxError.count:43,DemuxError.JobHistoryCount:43,DemuxMapOutput.SystemMetrics records:14,DemuxMapOutput.total records:4799,DemuxMapOutput.Hadoop_rpc_metrics records:7,DemuxMapOutput.Hadoop_dfs_namenode records:1,DemuxMapOutput.Df records:6,DemuxMapOutput.JobHistoryInError records:43,DemuxMapOutput.Hadoop_jvm_metrics records:7,DemuxMapOutput.Hadoop_dfs_datanode records:3,DemuxMapOutput.SizeVsFinishTime records:1171,DemuxMapOutput.Hadoop_mapred_tasktracker records:3,DemuxMapOutput.Netstat records:2,DemuxMapOutput.JobLogHistoryReduceProcessor records:3494,DemuxMapOutput.Hadoop_dfs_FSNamesystem records:1,DemuxMapOutput.MRJobReduceProcessor records:43,DemuxMapOutput.Hadoop_dfs_FSDirectory records:1,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:3,DemuxMapInput.Df chunks:1,DemuxMa pInput.Netstat chunks:2,DemuxMapInput.JobHistory chunks:43,DemuxMapInput.total chunks:55,DemuxMapInput.Iostat chunks:1,DemuxMapInput.Sar chunks:1,DemuxMapInput.HadoopMetricsProcessor chunks:7,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:55,Map-Reduce Framework.Spilled Records:4799,Map-Reduce Framework.Map output bytes:3381089,Map-Reduce Framework.Map input bytes:145317,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:4799" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000007_0" START_TIME="1236610752390" HOSTNAME="/default-rack/gs301025.inktomisearch.com" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000007_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610757772" HOSTNAME="/default-rack/gs301025.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000007" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610757772" COUNTERS="File Systems.HDFS bytes read:86853,File Systems.Local bytes written:284287,DemuxError.count:26,DemuxError.JobHistoryCount:26,DemuxMapOutput.SystemMetrics records:16,DemuxMapOutput.Hadoop_mapred_tasktracker records:2,DemuxMapOutput.Hadoop_rpc_metrics records:2,DemuxMapOutput.total records:2862,DemuxMapOutput.JobLogHistoryReduceProcessor records:2124,DemuxMapOutput.JobHistoryInError records:26,DemuxMapOutput.Df records:6,DemuxMapOutput.Hadoop_jvm_metrics records:2,DemuxMapOutput.MRJobReduceProcessor records:26,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:2,DemuxMapOutput.SizeVsFinishTime records:656,DemuxMapInput.Df chunks:1,DemuxMapInput.JobHistory chunks:27,DemuxMapInput.total chunks:36,DemuxMapInput.Top chunks:2,DemuxMapInput.Iostat chunks:1,DemuxMapInput.Sar chunks:1,DemuxMapInput.HadoopMetricsProcessor chunks:4,Map-Reduce Framework.Combine outpu t records:0,Map-Reduce Framework.Map input records:36,Map-Reduce Framework.Spilled Records:2862,Map-Reduce Framework.Map output bytes:1925922,Map-Reduce Framework.Map input bytes:86707,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:2862" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000004_0" START_TIME="1236610751253" HOSTNAME="/default-rack/gs301022.inktomisearch.com" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000004_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610761490" HOSTNAME="/default-rack/gs301022.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000004" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610761490" COUNTERS="File Systems.HDFS bytes read:118020,File Systems.Local bytes written:403758,DemuxError.count:40,DemuxError.JobHistoryCount:40,DemuxMapOutput.SystemMetrics records:24,DemuxMapOutput.total records:3850,DemuxMapOutput.Hadoop_rpc_metrics records:9,DemuxMapOutput.Hadoop_dfs_namenode records:1,DemuxMapOutput.Df records:18,DemuxMapOutput.JobHistoryInError records:40,DemuxMapOutput.Hadoop_jvm_metrics records:9,DemuxMapOutput.Hadoop_dfs_datanode records:4,DemuxMapOutput.SizeVsFinishTime records:831,DemuxMapOutput.Hadoop_mapred_tasktracker records:4,DemuxMapOutput.Netstat records:2,DemuxMapOutput.JobLogHistoryReduceProcessor records:2862,DemuxMapOutput.Hadoop_dfs_FSNamesystem records:1,DemuxMapOutput.MRJobReduceProcessor records:40,DemuxMapOutput.Hadoop_dfs_FSDirectory records:1,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:4,DemuxMapInput.Df chunks:3,DemuxMa pInput.Netstat chunks:2,DemuxMapInput.JobHistory chunks:40,DemuxMapInput.total chunks:60,DemuxMapInput.Iostat chunks:3,DemuxMapInput.Sar chunks:1,DemuxMapInput.HadoopMetricsProcessor chunks:11,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:60,Map-Reduce Framework.Spilled Records:3850,Map-Reduce Framework.Map output bytes:2666885,Map-Reduce Framework.Map input bytes:117874,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:3850" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000005_0" START_TIME="1236610751497" HOSTNAME="/default-rack/gs301022.inktomisearch.com" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000005_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610760777" HOSTNAME="/default-rack/gs301022.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000005" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610760777" COUNTERS="File Systems.HDFS bytes read:114621,File Systems.Local bytes written:381193,DemuxError.count:38,DemuxError.JobHistoryCount:38,DemuxMapOutput.SystemMetrics records:34,DemuxMapOutput.total records:3954,DemuxMapOutput.Hadoop_rpc_metrics records:9,DemuxMapOutput.Df records:6,DemuxMapOutput.JobHistoryInError records:38,DemuxMapOutput.Hadoop_jvm_metrics records:10,DemuxMapOutput.Hadoop_dfs_datanode records:7,DemuxMapOutput.SizeVsFinishTime records:907,DemuxMapOutput.Hadoop_mapred_tasktracker records:2,DemuxMapOutput.JobLogHistoryReduceProcessor records:2900,DemuxMapOutput.MRJobReduceProcessor records:38,DemuxMapOutput.Hadoop_dfs_FSDirectory records:1,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:2,DemuxMapInput.Df chunks:1,DemuxMapInput.JobHistory chunks:39,DemuxMapInput.total chunks:59,DemuxMapInput.Top chunks:2,DemuxMapInput.Iostat chunks:1,DemuxMapInpu t.Sar chunks:3,DemuxMapInput.HadoopMetricsProcessor chunks:13,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:59,Map-Reduce Framework.Spilled Records:3954,Map-Reduce Framework.Map output bytes:2695878,Map-Reduce Framework.Map input bytes:114475,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:3954" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000003_0" START_TIME="1236610748665" HOSTNAME="/default-rack/gs301024.inktomisearch.com" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000003_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610758101" HOSTNAME="/default-rack/gs301024.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000003" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610758101" COUNTERS="File Systems.HDFS bytes read:127102,File Systems.Local bytes written:414364,DemuxError.count:38,DemuxError.JobHistoryCount:38,DemuxMapOutput.SystemMetrics records:29,DemuxMapOutput.total records:3959,DemuxMapOutput.Hadoop_rpc_metrics records:8,DemuxMapOutput.Hadoop_dfs_namenode records:1,DemuxMapOutput.Df records:12,DemuxMapOutput.JobHistoryInError records:38,DemuxMapOutput.Hadoop_jvm_metrics records:8,DemuxMapOutput.Hadoop_dfs_datanode records:1,DemuxMapOutput.SizeVsFinishTime records:901,DemuxMapOutput.Hadoop_mapred_tasktracker records:6,DemuxMapOutput.Netstat records:3,DemuxMapOutput.JobLogHistoryReduceProcessor records:2906,DemuxMapOutput.Hadoop_dfs_FSNamesystem records:1,DemuxMapOutput.MRJobReduceProcessor records:38,DemuxMapOutput.Hadoop_dfs_FSDirectory records:1,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:6,DemuxMapInput.Df chunks:2,DemuxMa pInput.Netstat chunks:3,DemuxMapInput.JobHistory chunks:38,DemuxMapInput.total chunks:58,DemuxMapInput.Top chunks:1,DemuxMapInput.Iostat chunks:2,DemuxMapInput.Sar chunks:2,DemuxMapInput.HadoopMetricsProcessor chunks:10,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:58,Map-Reduce Framework.Spilled Records:3959,Map-Reduce Framework.Map output bytes:2816137,Map-Reduce Framework.Map input bytes:126956,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:3959" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000006_0" START_TIME="1236610750315" HOSTNAME="/default-rack/gs301023.inktomisearch.com" +MapAttempt TASK_TYPE="MAP" TASKID="task_200903042324_10816_m_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_m_000006_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236610759648" HOSTNAME="/default-rack/gs301023.inktomisearch.com" +Task TASKID="task_200903042324_10816_m_000006" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236610759648" COUNTERS="File Systems.HDFS bytes read:88012,File Systems.Local bytes written:293210,DemuxError.count:31,DemuxError.JobHistoryCount:31,DemuxMapOutput.SystemMetrics records:29,DemuxMapOutput.total records:2899,DemuxMapOutput.Netstat records:1,DemuxMapOutput.JobLogHistoryReduceProcessor records:2193,DemuxMapOutput.JobHistoryInError records:31,DemuxMapOutput.Df records:12,DemuxMapOutput.Hadoop_jvm_metrics records:1,DemuxMapOutput.MRJobReduceProcessor records:31,DemuxMapOutput.Hadoop_dfs_FSDirectory records:1,DemuxMapOutput.SizeVsFinishTime records:600,DemuxMapInput.Df chunks:2,DemuxMapInput.Netstat chunks:1,DemuxMapInput.JobHistory chunks:31,DemuxMapInput.total chunks:41,DemuxMapInput.Top chunks:1,DemuxMapInput.Iostat chunks:2,DemuxMapInput.Sar chunks:2,DemuxMapInput.HadoopMetricsProcessor chunks:2,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Ma p input records:41,Map-Reduce Framework.Spilled Records:2899,Map-Reduce Framework.Map output bytes:1990647,Map-Reduce Framework.Map input bytes:87866,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Map output records:2899" +Task TASKID="task_200903042324_10816_r_000005" TASK_TYPE="REDUCE" START_TIME="1236610763030" SPLITS="" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000000_0" START_TIME="1236610753164" HOSTNAME="/default-rack/gs301025.inktomisearch.com" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000000" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000000_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610764078" SORT_FINISHED="1236610764104" FINISH_TIME="1236610764326" HOSTNAME="/default-rack/gs301025.inktomisearch.com" +Task TASKID="task_200903042324_10816_r_000000" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610764326" COUNTERS="File Systems.HDFS bytes written:560,File Systems.Local bytes read:343,File Systems.Local bytes written:343,DemuxReduceOutput.total records:5,DemuxReduceOutput.Hadoop_dfs_FSDirectory records:5,DemuxReduceInput.total distinct keys:1,DemuxReduceInput.Hadoop_dfs_FSDirectory total distinct keys:1,Map-Reduce Framework.Reduce input groups:1,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:5,Map-Reduce Framework.Spilled Records:5,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:5" +Task TASKID="task_200903042324_10816_r_000006" TASK_TYPE="REDUCE" START_TIME="1236610765899" SPLITS="" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000001_0" START_TIME="1236610751786" HOSTNAME="/default-rack/gs301024.inktomisearch.com" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000001" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000001_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610763776" SORT_FINISHED="1236610763789" FINISH_TIME="1236610764135" HOSTNAME="/default-rack/gs301024.inktomisearch.com" +Task TASKID="task_200903042324_10816_r_000001" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610764135" COUNTERS="File Systems.HDFS bytes written:642,File Systems.Local bytes read:486,File Systems.Local bytes written:486,DemuxReduceOutput.total records:10,DemuxReduceOutput.Netstat records:10,DemuxReduceInput.total distinct keys:7,DemuxReduceInput.Netstat total distinct keys:7,Map-Reduce Framework.Reduce input groups:7,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:10,Map-Reduce Framework.Spilled Records:10,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:10" +Task TASKID="task_200903042324_10816_r_000007" TASK_TYPE="REDUCE" START_TIME="1236610766344" SPLITS="" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000002_0" START_TIME="1236610753719" HOSTNAME="/default-rack/gs301023.inktomisearch.com" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000002" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000002_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610766361" SORT_FINISHED="1236610766376" FINISH_TIME="1236610766711" HOSTNAME="/default-rack/gs301023.inktomisearch.com" +Task TASKID="task_200903042324_10816_r_000002" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610766711" COUNTERS="File Systems.HDFS bytes written:2445,File Systems.Local bytes read:3018,File Systems.Local bytes written:3018,DemuxReduceOutput.total records:84,DemuxReduceOutput.Df records:60,DemuxReduceOutput.Hadoop_mapred_shuffleOutput records:24,DemuxReduceInput.Df total distinct keys:10,DemuxReduceInput.total distinct keys:18,DemuxReduceInput.Hadoop_mapred_shuffleOutput total distinct keys:8,Map-Reduce Framework.Reduce input groups:18,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:84,Map-Reduce Framework.Spilled Records:84,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:84" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000005_0" START_TIME="1236610764414" HOSTNAME="/default-rack/gs301025.inktomisearch.com" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000005" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000005_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610769547" SORT_FINISHED="1236610769562" FINISH_TIME="1236610769795" HOSTNAME="/default-rack/gs301025.inktomisearch.com" +Task TASKID="task_200903042324_10816_r_000005" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610769795" COUNTERS="File Systems.HDFS bytes written:716,File Systems.Local bytes read:548,File Systems.Local bytes written:548,DemuxReduceOutput.total records:3,DemuxReduceOutput.Hadoop_dfs_namenode records:3,DemuxReduceInput.total distinct keys:1,DemuxReduceInput.Hadoop_dfs_namenode total distinct keys:1,Map-Reduce Framework.Reduce input groups:1,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:3,Map-Reduce Framework.Spilled Records:3,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:3" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000004_0" START_TIME="1236610754515" HOSTNAME="/default-rack/gs301019.inktomisearch.com" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000004" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000004_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610771019" SORT_FINISHED="1236610771034" FINISH_TIME="1236610772680" HOSTNAME="/default-rack/gs301019.inktomisearch.com" +Task TASKID="task_200903042324_10816_r_000004" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610772680" COUNTERS="File Systems.HDFS bytes written:992710,File Systems.Local bytes read:2055500,File Systems.Local bytes written:2055500,DemuxReduceOutput.SystemMetrics records:15,DemuxReduceOutput.total records:347,DemuxReduceOutput.JobHistoryInError records:308,DemuxReduceOutput.Hadoop_dfs_datanode records:24,DemuxReduceInput.total distinct keys:581,DemuxReduceInput.MRJobReduceProcessor total distinct keys:256,DemuxReduceInput.JobHistoryInError total distinct keys:302,DemuxReduceInput.SystemMetrics total distinct keys:15,DemuxReduceInput.Hadoop_dfs_datanode total distinct keys:8,Map-Reduce Framework.Reduce input groups:581,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:347,Map-Reduce Framework.Spilled Records:790,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:790" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000003_0" START_TIME="1236610756493" HOSTNAME="/default-rack/gs301022.inktomisearch.com" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000003" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000003_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610774361" SORT_FINISHED="1236610774376" FINISH_TIME="1236610776203" HOSTNAME="/default-rack/gs301022.inktomisearch.com" +Task TASKID="task_200903042324_10816_r_000003" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610776203" COUNTERS="File Systems.HDFS bytes written:9507,File Systems.Local bytes read:13207,File Systems.Local bytes written:13207,DemuxReduceOutput.Hadoop_mapred_tasktracker records:24,DemuxReduceOutput.total records:264,DemuxReduceOutput.Ps records:187,DemuxReduceOutput.Hadoop_jvm_metrics records:53,DemuxReduceInput.total distinct keys:19,DemuxReduceInput.Ps total distinct keys:1,DemuxReduceInput.Hadoop_mapred_tasktracker total distinct keys:8,DemuxReduceInput.Hadoop_jvm_metrics total distinct keys:10,Map-Reduce Framework.Reduce input groups:19,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:264,Map-Reduce Framework.Spilled Records:264,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:264" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000006_0" START_TIME="1236610765151" HOSTNAME="/default-rack/gs301024.inktomisearch.com" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000006" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000006_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610770754" SORT_FINISHED="1236610770774" FINISH_TIME="1236610774258" HOSTNAME="/default-rack/gs301024.inktomisearch.com" +Task TASKID="task_200903042324_10816_r_000006" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610774258" COUNTERS="File Systems.HDFS bytes written:411924,File Systems.Local bytes read:709720,File Systems.Local bytes written:709720,DemuxReduceOutput.total records:7132,DemuxReduceOutput.SizeVsFinishTime records:7132,DemuxReduceInput.total distinct keys:5878,DemuxReduceInput.SizeVsFinishTime total distinct keys:5878,Map-Reduce Framework.Reduce input groups:5878,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:7132,Map-Reduce Framework.Spilled Records:7132,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:7132" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000007_0" START_TIME="1236610767102" HOSTNAME="/default-rack/gs301023.inktomisearch.com" +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903042324_10816_r_000007" TASK_ATTEMPT_ID="attempt_200903042324_10816_r_000007_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236610772790" SORT_FINISHED="1236610772807" FINISH_TIME="1236610776148" HOSTNAME="/default-rack/gs301023.inktomisearch.com" +Task TASKID="task_200903042324_10816_r_000007" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236610776148" COUNTERS="File Systems.HDFS bytes written:181366,File Systems.Local bytes read:454838,File Systems.Local bytes written:454838,DemuxReduceOutput.Hadoop_rpc_metrics records:51,DemuxReduceOutput.total records:19314,DemuxReduceOutput.Hadoop_dfs_FSNamesystem records:3,DemuxReduceOutput.MSSRGraph records:19260,DemuxReduceInput.Hadoop_dfs_FSNamesystem total distinct keys:1,DemuxReduceInput.total distinct keys:19271,DemuxReduceInput.JobLogHistoryReduceProcessor total distinct keys:19260,DemuxReduceInput.Hadoop_rpc_metrics total distinct keys:10,Map-Reduce Framework.Reduce input groups:19271,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Reduce output records:19314,Map-Reduce Framework.Spilled Records:23357,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:23357" +Job JOBID="job_200903042324_10816" FINISH_TIME="1236610776480" JOB_STATUS="SUCCESS" FINISHED_MAPS="9" FINISHED_REDUCES="8" FAILED_MAPS="0" FAILED_REDUCES="0" COUNTERS="DemuxReduceOutput.SystemMetrics records:15,DemuxReduceOutput.Hadoop_rpc_metrics records:51,DemuxReduceOutput.total records:27159,DemuxReduceOutput.Hadoop_dfs_namenode records:3,DemuxReduceOutput.Df records:60,DemuxReduceOutput.JobHistoryInError records:308,DemuxReduceOutput.Hadoop_jvm_metrics records:53,DemuxReduceOutput.Hadoop_dfs_datanode records:24,DemuxReduceOutput.SizeVsFinishTime records:7132,DemuxReduceOutput.Hadoop_mapred_tasktracker records:24,DemuxReduceOutput.Netstat records:10,DemuxReduceOutput.Ps records:187,DemuxReduceOutput.Hadoop_dfs_FSNamesystem records:3,DemuxReduceOutput.MSSRGraph records:19260,DemuxReduceOutput.Hadoop_dfs_FSDirectory records:5,DemuxReduceOutput.Hadoop_mapred_shuffleOutput records:24,File Systems.HDFS bytes read:1003479,File Systems.HDFS bytes written:1599870,File Systems.Lo cal bytes read:3237660,File Systems.Local bytes written:6482349,DemuxMapOutput.SystemMetrics records:150,DemuxMapOutput.Hadoop_rpc_metrics records:51,DemuxMapOutput.total records:31645,DemuxMapOutput.Hadoop_dfs_namenode records:3,DemuxMapOutput.Df records:60,DemuxMapOutput.JobHistoryInError records:308,DemuxMapOutput.Hadoop_jvm_metrics records:53,DemuxMapOutput.Hadoop_dfs_datanode records:24,DemuxMapOutput.SizeVsFinishTime records:7132,DemuxMapOutput.Hadoop_mapred_tasktracker records:24,DemuxMapOutput.Netstat records:10,DemuxMapOutput.Ps records:187,DemuxMapOutput.JobLogHistoryReduceProcessor records:23303,DemuxMapOutput.Hadoop_dfs_FSNamesystem records:3,DemuxMapOutput.MRJobReduceProcessor records:308,DemuxMapOutput.Hadoop_dfs_FSDirectory records:5,DemuxMapOutput.Hadoop_mapred_shuffleOutput records:24,DemuxError.count:308,DemuxError.JobHistoryCount:308,Job Counters .Launched reduce tasks:8,Job Counters .Rack-local map tasks:1,Job Counters .Launched map tasks:9,Job Counters . Data-local map tasks:8,DemuxReduceInput.total distinct keys:25776,DemuxReduceInput.Hadoop_mapred_tasktracker total distinct keys:8,DemuxReduceInput.Hadoop_dfs_namenode total distinct keys:1,DemuxReduceInput.JobLogHistoryReduceProcessor total distinct keys:19260,DemuxReduceInput.Hadoop_jvm_metrics total distinct keys:10,DemuxReduceInput.Hadoop_dfs_FSDirectory total distinct keys:1,DemuxReduceInput.Hadoop_dfs_FSNamesystem total distinct keys:1,DemuxReduceInput.Df total distinct keys:10,DemuxReduceInput.Hadoop_mapred_shuffleOutput total distinct keys:8,DemuxReduceInput.Ps total distinct keys:1,DemuxReduceInput.MRJobReduceProcessor total distinct keys:256,DemuxReduceInput.JobHistoryInError total distinct keys:302,DemuxReduceInput.Netstat total distinct keys:7,DemuxReduceInput.SystemMetrics total distinct keys:15,DemuxReduceInput.Hadoop_rpc_metrics total distinct keys:10,DemuxReduceInput.Hadoop_dfs_datanode total distinct keys:8,DemuxReduceInput.SizeVsFinishTime total distinct ke ys:5878,DemuxMapInput.Df chunks:10,DemuxMapInput.Netstat chunks:10,DemuxMapInput.JobHistory chunks:310,DemuxMapInput.total chunks:431,DemuxMapInput.Iostat chunks:10,DemuxMapInput.Top chunks:10,DemuxMapInput.Ps chunks:1,DemuxMapInput.Sar chunks:10,DemuxMapInput.HadoopMetricsProcessor chunks:70,Map-Reduce Framework.Reduce input groups:25776,Map-Reduce Framework.Combine output records:0,Map-Reduce Framework.Map input records:431,Map-Reduce Framework.Reduce output records:27159,Map-Reduce Framework.Spilled Records:63290,Map-Reduce Framework.Map output bytes:22148350,Map-Reduce Framework.Map input bytes:985473,Map-Reduce Framework.Map output records:31645,Map-Reduce Framework.Combine input records:0,Map-Reduce Framework.Reduce input records:31645" \ No newline at end of file Added: hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLogEntryTest.java URL: http://svn.apache.org/viewvc/hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLogEntryTest.java?rev=765282&view=auto ============================================================================== --- hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLogEntryTest.java (added) +++ hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/JobLogEntryTest.java Wed Apr 15 17:41:55 2009 @@ -0,0 +1,118 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.chukwa.extraction.demux.processor.mapper; + +import java.io.BufferedReader; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.Map.Entry; + +import org.apache.hadoop.chukwa.extraction.demux.processor.mapper.JobLog.JobLogLine; +import org.apache.hadoop.chukwa.extraction.engine.ChukwaRecord; +import org.apache.hadoop.chukwa.extraction.engine.ChukwaRecordKey; +import org.apache.hadoop.mapred.OutputCollector; +import org.apache.hadoop.mapred.Reporter; + +import junit.framework.TestCase; + +public class JobLogEntryTest extends TestCase { + private ArrayList testLogList = new ArrayList(); + + protected void setUp() throws Exception { + super.setUp(); + InputStream stream = this.getClass().getResourceAsStream("/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestJobLog.txt"); + BufferedReader br = new BufferedReader(new InputStreamReader(stream)); + while(true) { + String line = br.readLine(); + if(line == null) + break; + testLogList.add(line); + } + + stream = this.getClass().getResourceAsStream("/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/Hadoop18JobHistoryLog.txt"); + br = new BufferedReader(new InputStreamReader(stream)); + while(true) { + String line = br.readLine(); + if(line == null) + break; + testLogList.add(line); + } + } + + public void testJobLogEntry() { + JobLog jobLog = new JobLog(); + JobLogLine log = jobLog.getJobLogLine(testLogList.get(0)); + assertEquals("Meta", log.getLogType()); + assertEquals("1", log.get("VERSION")); + assertEquals(0l, log.getTimestamp()); + + log = jobLog.getJobLogLine(testLogList.get(1)); + assertEquals("JobData", log.getLogType()); + assertEquals("hdfs://gs301033/grid/0/tmp/hadoop-gmon/mapred/system/job_200903062215_0577/job\\.xml", log.get("JOBCONF")); + assertEquals("job_200903062215_0577", log.get("JOBID")); + assertEquals("grep-search", log.get("JOBNAME")); + assertEquals("gmon", log.get("USER")); + assertEquals("1236386525570", log.get("SUBMIT_TIME")); + assertEquals(1236386525570l, log.getTimestamp()); + + log = jobLog.getJobLogLine(testLogList.get(2)); + assertEquals(1236386525570l, log.getTimestamp()); + + log = jobLog.getJobLogLine(testLogList.get(4)); + assertEquals("TaskData", log.getLogType()); + assertEquals("", log.get("SPLITS")); + assertEquals(1236386529449l, log.getTimestamp()); + + log = jobLog.getJobLogLine(testLogList.get(72)); + assertEquals("TaskData", log.getLogType()); + assertEquals("{(org\\.apache\\.hadoop\\.mapred\\.Task$Counter)(Map-Reduce Framework)[(REDUCE_INPUT_GROUPS)(Reduce input groups)(0)][(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(REDUCE_SHUFFLE_BYTES)(Reduce shuffle bytes)(0)][(REDUCE_OUTPUT_RECORDS)(Reduce output records)(0)][(SPILLED_RECORDS)(Spilled Records)(0)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(REDUCE_INPUT_RECORDS)(Reduce input records)(0)]}", log.get("COUNTERS")); + + log = jobLog.getJobLogLine(testLogList.get(73)); + HashMap counters = log.getCounterHash().flat(); + assertEquals("1", counters.get("Counter:org.apache.hadoop.mapred.JobInProgress$Counter:TOTAL_LAUNCHED_REDUCES").toString()); + assertEquals("20471", counters.get("Counter:FileSystemCounters:HDFS_BYTES_READ").toString()); + + log = jobLog.getJobLogLine(""); + assertTrue(log==null); + + log = jobLog.getJobLogLine("Job JOBID=\"job_200903042324_8630\" FINISH_TIME=\"1236527538594\" JOB_STATUS=\"SUCCESS\" FINISHED_MAPS=\"10\" FINISHED_REDUCES=\"8\" FAILED_MAPS=\"0\" FAILED_REDUCES=\"0\" COUNTERS=\"input records:0,Map-Reduce Framework.Reduce input records:57038\""); + + // print all key-values + for(String line : testLogList) { + log = jobLog.getJobLogLine(line); + if(log == null) { + continue; + } + System.out.println(log.getLogType()); + for(Entry entry : log.entrySet()) { + System.out.println(entry.getKey() + ": " + entry.getValue()); + } + + // list all counters for this entry + for(Entry entry : log.getCounterHash().flat().entrySet()) { + System.out.println(entry.getKey() + ": " + entry.getValue()); + } + + System.out.println(); + } + } + +} Added: hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestJobLog.txt URL: http://svn.apache.org/viewvc/hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestJobLog.txt?rev=765282&view=auto ============================================================================== --- hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestJobLog.txt (added) +++ hadoop/chukwa/trunk/src/test/org/apache/hadoop/chukwa/extraction/demux/processor/mapper/TestJobLog.txt Wed Apr 15 17:41:55 2009 @@ -0,0 +1,89 @@ +Meta VERSION="1" . +Job JOBID="job_200903062215_0577" JOBNAME="grep-search" USER="gmon" SUBMIT_TIME="1236386525570" JOBCONF="hdfs://gs301033/grid/0/tmp/hadoop-gmon/mapred/system/job_200903062215_0577/job\.xml" . +Job JOBID="job_200903062215_0577" JOB_PRIORITY="NORMAL" . +Job JOBID="job_200903062215_0577" LAUNCH_TIME="1236386526545" TOTAL_MAPS="14" TOTAL_REDUCES="1" JOB_STATUS="PREP" . +Task TASKID="task_200903062215_0577_m_000015" TASK_TYPE="SETUP" START_TIME="1236386529449" SPLITS="" . +MapAttempt TASK_TYPE="SETUP" TASKID="task_200903062215_0577_m_000015" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000015_0" START_TIME="1236386529667" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="SETUP" TASKID="task_200903062215_0577_m_000015" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000015_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386531629" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="setup" COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000015" TASK_TYPE="SETUP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386532461" COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" . +Job JOBID="job_200903062215_0577" JOB_STATUS="RUNNING" . +Task TASKID="task_200903062215_0577_m_000000" TASK_TYPE="MAP" START_TIME="1236386532464" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +Task TASKID="task_200903062215_0577_m_000001" TASK_TYPE="MAP" START_TIME="1236386532466" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000000" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000000_0" START_TIME="1236386532528" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000000" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000000_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386536314" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/capacity-scheduler\.xml:0+6275" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(6275)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(156)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(6275)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000000" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386538505" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(6275)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(156)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(6275)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000001" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000001_0" START_TIME="1236386532592" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000001" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000001_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386536420" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/hadoop-policy\.xml:0+4190" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(4190)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(57)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(1)][(MAP_INPUT_RECORDS)(Map input records)(97)][(SPILLED_RECORDS)(Spilled Records)(1)][(MAP_OUTPUT_BYTES)(Map output bytes)(17)][(MAP_INPUT_BYTES)(Map input bytes)(4190)][(COMBINE_INPUT_RECORDS)(Combine input records)(1)][(MAP_OUTPUT_RECORDS)(Map output records)(1)]}" . +Task TASKID="task_200903062215_0577_m_000001" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386538521" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(4190)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(57)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(1)][(MAP_INPUT_RECORDS)(Map input records)(97)][(SPILLED_RECORDS)(Spilled Records)(1)][(MAP_OUTPUT_BYTES)(Map output bytes)(17)][(MAP_INPUT_BYTES)(Map input bytes)(4190)][(COMBINE_INPUT_RECORDS)(Combine input records)(1)][(MAP_OUTPUT_RECORDS)(Map output records)(1)]}" . +Task TASKID="task_200903062215_0577_m_000002" TASK_TYPE="MAP" START_TIME="1236386538534" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +Task TASKID="task_200903062215_0577_m_000003" TASK_TYPE="MAP" START_TIME="1236386538538" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +Task TASKID="task_200903062215_0577_r_000000" TASK_TYPE="REDUCE" START_TIME="1236386538540" SPLITS="" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000002" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000002_0" START_TIME="1236386538639" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000002" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000002_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386543228" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/log4j\.properties:0+2815" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(2815)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(94)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(2815)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000002" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386544603" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(2815)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(94)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(2815)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000003" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000003_0" START_TIME="1236386538793" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000003" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000003_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386542941" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/hadoop-env\.sh:0+2237" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(2237)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(54)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(2237)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000003" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386544606" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(2237)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(54)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(2237)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000004" TASK_TYPE="MAP" START_TIME="1236386544614" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +Task TASKID="task_200903062215_0577_m_000005" TASK_TYPE="MAP" START_TIME="1236386544615" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000004" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000004_0" START_TIME="1236386544677" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000004" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000004_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386548270" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/hadoop-metrics\.properties:0+1245" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(1245)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(145)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(5)][(MAP_INPUT_RECORDS)(Map input records)(40)][(SPILLED_RECORDS)(Spilled Records)(5)][(MAP_OUTPUT_BYTES)(Map output bytes)(152)][(MAP_INPUT_BYTES)(Map input bytes)(1245)][(COMBINE_INPUT_RECORDS)(Combine input records)(8)][(MAP_OUTPUT_RECORDS)(Map output records)(8)]}" . +Task TASKID="task_200903062215_0577_m_000004" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386550678" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(1245)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(145)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(5)][(MAP_INPUT_RECORDS)(Map input records)(40)][(SPILLED_RECORDS)(Spilled Records)(5)][(MAP_OUTPUT_BYTES)(Map output bytes)(152)][(MAP_INPUT_BYTES)(Map input bytes)(1245)][(COMBINE_INPUT_RECORDS)(Combine input records)(8)][(MAP_OUTPUT_RECORDS)(Map output records)(8)]}" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000005" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000005_0" START_TIME="1236386544756" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000005" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000005_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386548990" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/ssl-client\.xml\.example:0+1243" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(1243)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(57)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(1243)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000005" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386550680" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(1243)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(57)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(1243)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000006" TASK_TYPE="MAP" START_TIME="1236386550684" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +Task TASKID="task_200903062215_0577_m_000007" TASK_TYPE="MAP" START_TIME="1236386550685" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000006" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000006_0" START_TIME="1236386550745" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000006" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000006_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386554427" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/ssl-server\.xml\.example:0+1195" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(1195)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(55)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(1195)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000006" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386556711" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(1195)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(55)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(1195)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000007" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000007_0" START_TIME="1236386550810" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000007" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000007_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386554264" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/configuration\.xsl:0+535" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(535)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(24)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(535)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000007" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386556714" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(535)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(24)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(535)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000008" TASK_TYPE="MAP" START_TIME="1236386556718" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +Task TASKID="task_200903062215_0577_m_000009" TASK_TYPE="MAP" START_TIME="1236386556720" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000008" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000008_0" START_TIME="1236386556772" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000008" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000008_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386560499" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/taskcontroller\.cfg:0+184" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(184)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(3)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(184)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000008" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386562757" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(184)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(3)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(184)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000009" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000009_0" START_TIME="1236386556893" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000009" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000009_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386560131" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/core-site\.xml:0+178" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(178)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(8)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(178)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000009" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386562759" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(178)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(8)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(178)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000010" TASK_TYPE="MAP" START_TIME="1236386562765" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +Task TASKID="task_200903062215_0577_m_000011" TASK_TYPE="MAP" START_TIME="1236386562767" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000010" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000010_0" START_TIME="1236386562818" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000010" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000010_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386565490" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/hdfs-site\.xml:0+178" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(178)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(8)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(178)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000010" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386565779" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(178)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(8)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(178)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000011" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000011_0" START_TIME="1236386562883" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000011" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000011_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386565621" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/mapred-site\.xml:0+178" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(178)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(8)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(178)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000011" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386565780" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(178)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(8)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(178)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000012" TASK_TYPE="MAP" START_TIME="1236386565784" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +Task TASKID="task_200903062215_0577_m_000013" TASK_TYPE="MAP" START_TIME="1236386565785" SPLITS="/default-rack/gs301033\.inktomisearch\.com" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000012" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000012_0" START_TIME="1236386565836" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000012" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000012_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386568540" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/masters:0+9" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(9)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(1)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(9)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000012" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386568798" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(9)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(1)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(9)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000013" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000013_0" START_TIME="1236386565899" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200903062215_0577_m_000013" TASK_ATTEMPT_ID="attempt_200903062215_0577_m_000013_0" TASK_STATUS="SUCCESS" FINISH_TIME="1236386568625" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="hdfs://gs301033/user/gmon/input/slaves:0+9" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(9)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(1)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(9)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +Task TASKID="task_200903062215_0577_m_000013" TASK_TYPE="MAP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386568801" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(HDFS_BYTES_READ)(HDFS_BYTES_READ)(9)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(38)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(MAP_INPUT_RECORDS)(Map input records)(1)][(SPILLED_RECORDS)(Spilled Records)(0)][(MAP_OUTPUT_BYTES)(Map output bytes)(0)][(MAP_INPUT_BYTES)(Map input bytes)(9)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(MAP_OUTPUT_RECORDS)(Map output records)(0)]}" . +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903062215_0577_r_000000" TASK_ATTEMPT_ID="attempt_200903062215_0577_r_000000_0" START_TIME="1236386538714" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +ReduceAttempt TASK_TYPE="REDUCE" TASKID="task_200903062215_0577_r_000000" TASK_ATTEMPT_ID="attempt_200903062215_0577_r_000000_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236386574219" SORT_FINISHED="1236386574283" FINISH_TIME="1236386575661" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="reduce > reduce" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(FILE_BYTES_READ)(FILE_BYTES_READ)(132)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(132)][(HDFS_BYTES_WRITTEN)(HDFS_BYTES_WRITTEN)(248)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(REDUCE_INPUT_GROUPS)(Reduce input groups)(6)][(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(REDUCE_SHUFFLE_BYTES)(Reduce shuffle bytes)(204)][(REDUCE_OUTPUT_RECORDS)(Reduce output records)(6)][(SPILLED_RECORDS)(Spilled Records)(6)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(REDUCE_INPUT_RECORDS)(Reduce input records)(6)]}" . +Task TASKID="task_200903062215_0577_r_000000" TASK_TYPE="REDUCE" TASK_STATUS="SUCCESS" FINISH_TIME="1236386577843" COUNTERS="{(FileSystemCounters)(FileSystemCounters)[(FILE_BYTES_READ)(FILE_BYTES_READ)(132)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(132)][(HDFS_BYTES_WRITTEN)(HDFS_BYTES_WRITTEN)(248)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(REDUCE_INPUT_GROUPS)(Reduce input groups)(6)][(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(REDUCE_SHUFFLE_BYTES)(Reduce shuffle bytes)(204)][(REDUCE_OUTPUT_RECORDS)(Reduce output records)(6)][(SPILLED_RECORDS)(Spilled Records)(6)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(REDUCE_INPUT_RECORDS)(Reduce input records)(6)]}" . +Task TASKID="task_200903062215_0577_r_000001" TASK_TYPE="CLEANUP" START_TIME="1236386577847" SPLITS="" . +ReduceAttempt TASK_TYPE="CLEANUP" TASKID="task_200903062215_0577_r_000001" TASK_ATTEMPT_ID="attempt_200903062215_0577_r_000001_0" START_TIME="1236386577882" TRACKER_NAME="tracker_gs301033\.inktomisearch\.com:localhost/127\.0\.0\.1:57217" HTTP_PORT="50060" . +ReduceAttempt TASK_TYPE="CLEANUP" TASKID="task_200903062215_0577_r_000001" TASK_ATTEMPT_ID="attempt_200903062215_0577_r_000001_0" TASK_STATUS="SUCCESS" SHUFFLE_FINISHED="1236386579532" SORT_FINISHED="1236386579532" FINISH_TIME="1236386579532" HOSTNAME="/default-rack/gs301033\.inktomisearch\.com" STATE_STRING="cleanup" COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(REDUCE_INPUT_GROUPS)(Reduce input groups)(0)][(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(REDUCE_SHUFFLE_BYTES)(Reduce shuffle bytes)(0)][(REDUCE_OUTPUT_RECORDS)(Reduce output records)(0)][(SPILLED_RECORDS)(Spilled Records)(0)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(REDUCE_INPUT_RECORDS)(Reduce input records)(0)]}" . +Task TASKID="task_200903062215_0577_r_000001" TASK_TYPE="CLEANUP" TASK_STATUS="SUCCESS" FINISH_TIME="1236386580865" COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(REDUCE_INPUT_GROUPS)(Reduce input groups)(0)][(COMBINE_OUTPUT_RECORDS)(Combine output records)(0)][(REDUCE_SHUFFLE_BYTES)(Reduce shuffle bytes)(0)][(REDUCE_OUTPUT_RECORDS)(Reduce output records)(0)][(SPILLED_RECORDS)(Spilled Records)(0)][(COMBINE_INPUT_RECORDS)(Combine input records)(0)][(REDUCE_INPUT_RECORDS)(Reduce input records)(0)]}" . +Job JOBID="job_200903062215_0577" FINISH_TIME="1236386580866" JOB_STATUS="SUCCESS" FINISHED_MAPS="14" FINISHED_REDUCES="1" FAILED_MAPS="0" FAILED_REDUCES="0" COUNTERS="{(org\.apache\.hadoop\.mapred\.JobInProgress$Counter)(Job Counters )[(TOTAL_LAUNCHED_REDUCES)(Launched reduce tasks)(1)][(TOTAL_LAUNCHED_MAPS)(Launched map tasks)(14)][(DATA_LOCAL_MAPS)(Data-local map tasks)(14)]}{(FileSystemCounters)(FileSystemCounters)[(FILE_BYTES_READ)(FILE_BYTES_READ)(132)][(HDFS_BYTES_READ)(HDFS_BYTES_READ)(20471)][(FILE_BYTES_WRITTEN)(FILE_BYTES_WRITTEN)(790)][(HDFS_BYTES_WRITTEN)(HDFS_BYTES_WRITTEN)(248)]}{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(REDUCE_INPUT_GROUPS)(Reduce input groups)(6)][(COMBINE_OUTPUT_RECORDS)(Combine output records)(6)][(MAP_INPUT_RECORDS)(Map input records)(606)][(REDUCE_SHUFFLE_BYTES)(Reduce shuffle bytes)(204)][(REDUCE_OUTPUT_RECORDS)(Reduce output records)(6)][(SPILLED_RECORDS)(Spilled Records)(12)][(MAP_OUTPUT_BYTES)(Map output byte s)(169)][(MAP_INPUT_BYTES)(Map input bytes)(20471)][(MAP_OUTPUT_RECORDS)(Map output records)(9)][(COMBINE_INPUT_RECORDS)(Combine input records)(9)][(REDUCE_INPUT_RECORDS)(Reduce input records)(6)]}" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200904060626_2141_m_000108" TASK_ATTEMPT_ID="attempt_200904060626_2141_m_000108_1" START_TIME="1239190934835" TRACKER_NAME="tracker_kry50024\.inktomisearch\.com:localhost/127\.0\.0\.1:39507" HTTP_PORT="50060" . +MapAttempt TASK_TYPE="MAP" TASKID="task_200904060626_2141_m_000108" TASK_ATTEMPT_ID="attempt_200904060626_2141_m_000108_1" TASK_STATUS="FAILED" FINISH_TIME="1239190949062" HOSTNAME="kry50024\.inktomisearch\.com" ERROR="java\.io\.IOException: MROutput/MRErrThread failed:java\.lang\.ArrayIndexOutOfBoundsException: -1 + at org\.apache\.hadoop\.mapred\.lib\.KeyFieldBasedPartitioner\.hashCode(KeyFieldBasedPartitioner\.java:95) + at org\.apache\.hadoop\.mapred\.lib\.KeyFieldBasedPartitioner\.getPartition(KeyFieldBasedPartitioner\.java:87) + at org\.apache\.hadoop\.mapred\.MapTask$MapOutputBuffer\.collect(MapTask\.java:801) + at org\.apache\.hadoop\.streaming\.PipeMapRed$MROutputThread\.run(PipeMapRed\.java:378) + + at org\.apache\.hadoop\.streaming\.PipeMapper\.map(PipeMapper\.java:87) + at org\.apache\.hadoop\.mapred\.MapRunner\.run(MapRunner\.java:50) + at org\.apache\.hadoop\.streaming\.PipeMapRunner\.run(PipeMapRunner\.java:36) + at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:356) + at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:305) + at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:156) +" . +MapAttempt TASK_TYPE="CLEANUP" TASKID="task_200904060626_2141_m_000197" TASK_ATTEMPT_ID="attempt_200904060626_2141_m_000197_0" START_TIME="1239190961843" TRACKER_NAME="tracker_kry3083\.inktomisearch\.com:localhost/127\.0\.0\.1:60970" HTTP_PORT="50060" .