Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of hadoopnode@gmail.com designates
 209.85.215.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=LNB2pzIuPJQ1znkNPTg8cfuumWGRa//edVDajNrGCkG4yc3etKJvdMBlNdTjPSn0mT
         VRzP0BLfBgWqvDQowvvt//rLywHNc9NGC0tVOkH/LDy5zhdRw9snaEsHYPCMRHvehss1
         KiH9v3MXohyBEe53BcdWsAFOu3dLuwE2u2IvE=
MIME-Version: 1.0
Date: Tue, 19 Oct 2010 18:44:52 -0400
Message-ID: <AANLkTimVACV98y14BKsTcDXYb5XAQBMHw9+tN7zHMq0L@mail.gmail.com>
Subject: How to stop a mapper within a map-reduce job when you detect bad
 input
From: ed <hadoopnode@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0015174c0fc67e89f00493000ad7

--0015174c0fc67e89f00493000ad7
Content-Type: text/plain; charset=ISO-8859-1

Hello,

I have a simple map-reduce job that reads in zipped files and converts them
to lzo compression.  Some of the files are not properly zipped which results
in Hadoop throwing an "java.io.EOFException: Unexpected end of input stream
error" and causes the job to fail.  Is there a way to catch this exception
and tell hadoop to just ignore the file and move on?  I think the exception
is being thrown by the class reading in the Gzip file and not my mapper
class.  Is this correct?  Is there a way to handle this type of error
gracefully?

Thank you!

~Ed

--0015174c0fc67e89f00493000ad7--