Return-Path: Delivered-To: apmail-hadoop-chukwa-dev-archive@minotaur.apache.org Received: (qmail 69613 invoked from network); 14 Apr 2009 17:29:39 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 14 Apr 2009 17:29:39 -0000 Received: (qmail 45875 invoked by uid 500); 14 Apr 2009 17:29:39 -0000 Delivered-To: apmail-hadoop-chukwa-dev-archive@hadoop.apache.org Received: (qmail 45860 invoked by uid 500); 14 Apr 2009 17:29:39 -0000 Mailing-List: contact chukwa-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: chukwa-dev@hadoop.apache.org Delivered-To: mailing list chukwa-dev@hadoop.apache.org Received: (qmail 45850 invoked by uid 99); 14 Apr 2009 17:29:39 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Apr 2009 17:29:39 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Apr 2009 17:29:36 +0000 Received: from brutus (localhost [127.0.0.1]) by brutus.apache.org (Postfix) with ESMTP id 7A865234C044 for ; Tue, 14 Apr 2009 10:29:15 -0700 (PDT) Message-ID: <1635248119.1239730155500.JavaMail.jira@brutus> Date: Tue, 14 Apr 2009 10:29:15 -0700 (PDT) From: "Eric Yang (JIRA)" To: chukwa-dev@hadoop.apache.org Subject: [jira] Created: (CHUKWA-132) Handle multiline output in Job History file MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org Handle multiline output in Job History file ------------------------------------------- Key: CHUKWA-132 URL: https://issues.apache.org/jira/browse/CHUKWA-132 Project: Hadoop Chukwa Issue Type: Bug Components: Data Processors Environment: Redhat EL 5.1, Java 6 Reporter: Eric Yang Priority: Critical When there are multi line output in the Job History file, the parser fails with exception like this: MapAttempt TASK_TYPE="MAP" TASKID="task_200904060626_2141_m_000108" TASK_ATTEMPT_ID="attempt_200904060626_2141_m_000108_1" START_TIME="1239190934835" TRACKER_NAME="tracker_kry50024\.inktomisearch\.com:localhost/127\.0\.0\.1:39507" HTTP_PORT="50060" . MapAttempt TASK_TYPE="MAP" TASKID="task_200904060626_2141_m_000108" TASK_ATTEMPT_ID="attempt_200904060626_2141_m_000108_1" TASK_STATUS="FAILED" FINISH_TIME="1239190949062" HOSTNAME="kry50024\.inktomisearch\.com" ERROR="java\.io\.IOException: MROutput/MRErrThread failed:java\.lang\.ArrayIndexOutOfBoundsException: -1 at org\.apache\.hadoop\.mapred\.lib\.KeyFieldBasedPartitioner\.hashCode(KeyFieldBasedPartitioner\.java:95) at org\.apache\.hadoop\.mapred\.lib\.KeyFieldBasedPartitioner\.getPartition(KeyFieldBasedPartitioner\.java:87) at org\.apache\.hadoop\.mapred\.MapTask$MapOutputBuffer\.collect(MapTask\.java:801) at org\.apache\.hadoop\.streaming\.PipeMapRed$MROutputThread\.run(PipeMapRed\.java:378) at org\.apache\.hadoop\.streaming\.PipeMapper\.map(PipeMapper\.java:87) at org\.apache\.hadoop\.mapred\.MapRunner\.run(MapRunner\.java:50) at org\.apache\.hadoop\.streaming\.PipeMapRunner\.run(PipeMapRunner\.java:36) at org\.apache\.hadoop\.mapred\.MapTask\.runOldMapper(MapTask\.java:356) at org\.apache\.hadoop\.mapred\.MapTask\.run(MapTask\.java:305) at org\.apache\.hadoop\.mapred\.Child\.main(Child\.java:156) " . MapAttempt TASK_TYPE="CLEANUP" TASKID="task_200904060626_2141_m_000197" TASK_ATTEMPT_ID="attempt_200904060626_2141_m_000197_0" START_TIME="1239190961843" TRACKER_NAME="tracker_kry3083\.inktomisearch\.com:localhost/127\.0\.0\.1:60970" HTTP_PORT="50060" . MapAttempt TASK_TYPE="CLEANUP" TASKID="task_200904060626_2141_m_000197" TASK_ATTEMPT_ID="attempt_200904060626_2141_m_000197_0" TASK_STATUS="SUCCESS" FINISH_TIME="1239190963602" HOSTNAME="/74\.6\.135\.128/kry3083\.inktomisearch\.com" STATE_STRING="cleanup" COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" . Task TASKID="task_200904060626_2141_m_000197" TASK_TYPE="CLEANUP" TASK_STATUS="SUCCESS" FINISH_TIME="1239190963509" COUNTERS="{(org\.apache\.hadoop\.mapred\.Task$Counter)(Map-Reduce Framework)[(SPILLED_RECORDS)(Spilled Records)(0)]}" . Job JOBID="job_200904060626_2141" FINISH_TIME="1239190963510" JOB_STATUS="FAILED" FINISHED_MAPS="0" FINISHED_REDUCES="0" . [cchunkException] :java.lang.StringIndexOutOfBoundsException: String index out of range: -1 at java.lang.String.substring(String.java:1938) at org.apache.hadoop.chukwa.extraction.demux.processor.mapper.JobLog$JobLogLine.(JobLog.java:114) at org.apache.hadoop.chukwa.extraction.demux.processor.mapper.JobLog.parse(JobLog.java:39) at org.apache.hadoop.chukwa.extraction.demux.processor.mapper.AbstractProcessor.process(AbstractProcessor.java:90) at org.apache.hadoop.chukwa.extraction.demux.Demux$MapClass.map(Demux.java:94) at org.apache.hadoop.chukwa.extraction.demux.Demux$MapClass.map(Demux.java:60) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2210) [csource] :kry-jt1.red.ygrid.yahoo.com [ctags] :cluster="kryptonitered" -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.