Mailing-List: contact mapreduce-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates
 209.85.216.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <BANLkTikqObwERG_+ZVr31sqN+2Hf4ojzMg@mail.gmail.com>
References: <BANLkTikqObwERG_+ZVr31sqN+2Hf4ojzMg@mail.gmail.com>
From: Harsh J <harsh@cloudera.com>
Date: Thu, 14 Apr 2011 13:22:38 +0530
Message-ID: <BANLkTin5s7mwp_BhTnMcKRj4E2za1=uvSg@mail.gmail.com>
Subject: Re: Using MultipleTextOutputFormat for map-only jobs
To: mapreduce-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Hello Hari,

On Thu, Apr 14, 2011 at 11:09 AM, Hari Sreekumar
<hsreekumar@clickable.com> wrote:
> Hi,
> I have a map-only mapreduce job where I want to deduce the output filenam=
e
> from the output key/value. I figured=A0MultipleTextOutputFormat=A0is the =
best
> fit for my purpose. But I am unable to use it in map-only jobs. I was abl=
e
> to run it if I add a reduce phase. But when I use map-only jobs, the file
> gets written to the usual part-0000xx files. Also, is there no support fo=
r
> this output format in v0.20.2? I mean, is it necessary to use the depreca=
ted
> classes if I want to use this?
> Thanks,
> Hari

The class MultipleOutputFormat is not available in the Hadoop for the
new, unstable API, as it has been replaced in functionality by the
MultipleOutputs class that does the same very similarly. However, the
new API MultipleOutputs is not part of the Apache's Hadoop 0.20.2
release either [1].

Using the stable API is still recommended (it is no longer marked
deprecated in 0.20.3 and 0.21 also supports the old API)

That said, it should still work for Map-only jobs as described in two
of its usecases [2]. Could you give us some details of your code setup
for using this?

[1] - It is available as part of 0.21.0, though, or in Cloudera's
Distribution including Apache Hadoop 0.20.2.
[2] - http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/ma=
pred/lib/MultipleOutputFormat.html

--=20
Harsh J