Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of qwertymaniac@gmail.com
 designates 209.85.161.48 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :content-type:content-transfer-encoding;
        b=oFEAuSNlxcf0zVzJB62/DvRbAIhQXcBQD3XNn6kHJ3Z5PG0SuVi1k4tSqnyZ0DkgLb
         j6qiVB8ue5PydLvQSdC+MRZ4oHjsSNfqQHNEFWorfq5AiNfty1hhKkcys3zNy08bCZjF
         6gws4IYAhLky2i/AJDezsniHQA8vR8Ix/Tjqw=
MIME-Version: 1.0
In-Reply-To: <9E0073162FD25B489A01AD86D92D983E0449044D@XMBIL132.northgrum.com>
References: <47331.61478.qm@web39708.mail.mud.yahoo.com>
 <AANLkTimmcB0yek5Y6g1D0cnO=7-tGsTiEmYw5LS6XtuH@mail.gmail.com>
 <08C814576B3D664DBE68BE682A57AE990D39327A@USA0300MS03.na.xerox.net>
 <AANLkTik2PH-24khm97XNPMUkvHMwfye=hSdROYCRp=9i@mail.gmail.com>
 <9E0073162FD25B489A01AD86D92D983E04D2EBDB@XMBIL132.northgrum.com>
 <AANLkTi=yT=eDLh96C8BBGCHhqa8J+TeHhLiaGs03=Ntv@mail.gmail.com>
 <9E0073162FD25B489A01AD86D92D983E0449044D@XMBIL132.northgrum.com>
From: Harsh J <qwertymaniac@gmail.com>
Date: Sun, 26 Dec 2010 21:50:41 +0530
Message-ID: <AANLkTi=4PMtOpHUHdhagPDKyuNxqePUz00VM+Rg_kUyP@mail.gmail.com>
Subject: Re: Custom input split
To: common-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hi,

On Sun, Dec 26, 2010 at 6:29 PM, Black, Michael (IS)
<Michael.Black2@ngc.com> wrote:
> I assume there's a way to make a specific # of splits and add each docume=
nt to the separate splits...but I'll be darned if I can find the docs or an=
 example to show this.

Would CombineFileInputFormat and CombineFileSplit be what you're looking fo=
r?

Doc links: http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hado=
op/mapred/lib/CombineFileInputFormat.html
& http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/mapred=
/lib/CombineFileSplit.html

> As I said I'm using hadoop-0.20.2 which I know makes a difference as so m=
any things get deprecated on each release. =A0Old references don't seem to =
work.

The API marked deprecated in 0.20.{0,1,2} has been un-deprecated in
the 0.21.0 release  and is also considered as the "stable" API. You
can continue using it, as it is still supported.

(Maybe 0.20.3 will have them un-deprecated too, I'm not sure what's
the status on that, although doing so would surely help avoid beginner
confusion.)
--=20
Harsh J
www.harshj.com