Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: local policy)
From: Tony Burton <TBurton@SportingIndex.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Date: Wed, 29 Aug 2012 11:38:03 +0100
Subject: RE: hadoop 1.0.3 equivalent of MultipleTextOutputFormat
Thread-Topic: hadoop 1.0.3 equivalent of MultipleTextOutputFormat
Thread-Index: Ac2FztW0U8yuZjwjSP2hB/rHEYfNywAA3eYQ
Message-ID: 
 <556325346CA26341B6F0530E07F90D96016C64CD95D0@GBGH-EXCH-CMS.sig.ads>
References: <20120828115859.240490@gmx.net>
 <556325346CA26341B6F0530E07F90D96016C64CD95C7@GBGH-EXCH-CMS.sig.ads>
 <CAOcnVr0s9U8NbTuNpkmn1Tf-O5QmeT9YLSVJbVt2ajRg+PebvQ@mail.gmail.com>
 <556325346CA26341B6F0530E07F90D96016C64CD95C9@GBGH-EXCH-CMS.sig.ads>
 <CAOcnVr3C=mD1ooM-HVbUwr0OF-fMbKaSHxBcno2RMppgJAJJ-A@mail.gmail.com>
In-Reply-To: 
 <CAOcnVr3C=mD1ooM-HVbUwr0OF-fMbKaSHxBcno2RMppgJAJJ-A@mail.gmail.com>
Accept-Language: en-US, en-GB
Content-Language: en-US
acceptlanguage: en-US, en-GB
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Thanks Harsh! Will try it out and report back later.


-----Original Message-----
From: Harsh J [mailto:harsh@cloudera.com]=20
Sent: 29 August 2012 11:12
To: user@hadoop.apache.org
Subject: Re: hadoop 1.0.3 equivalent of MultipleTextOutputFormat

Hi Tony,

Seeing your new question, I recalled Tom's post to a user once, here:
https://groups.google.com/a/cloudera.org/d/msg/cdh-user/pdyVyydt5Ys/1CaLukt=
4v1AJ

This specific call allows you to specify / characters in your name,
that gets translated into creation of directories automatically:
http://hadoop.apache.org/common/docs/stable/api/org/apache/hadoop/mapreduce=
/lib/output/MultipleOutputs.html#write(KEYOUT,%20VALUEOUT,%20java.lang.Stri=
ng)
(The last argument is where you will need to specify the path)

Try it out and let us know!

On Tue, Aug 28, 2012 at 7:06 PM, Tony Burton <TBurton@sportingindex.com> wr=
ote:
> Hi Harsh
>
> Thanks for the reply - my understanding is that with MultipleOutputs I ca=
n write differently named files into the same target directory. With Multip=
leTextOutputFormat I was able to override the target directory name to perf=
orm the segmentation, by overriding generateFileNameForKeyValue().
>
> Does the 1.0.3 MultipleOutputs give me the ability to alter the target di=
rectory name as well as the file name?
>
> Thanks,
>
> Tony
>
>
>
> -----Original Message-----
> From: Harsh J [mailto:harsh@cloudera.com]
> Sent: 28 August 2012 13:44
> To: user@hadoop.apache.org
> Subject: Re: hadoop 1.0.3 equivalent of MultipleTextOutputFormat
>
> The Multiple*OutputFormat have been deprecated in favor of the generic
> MultipleOutputs API. Would using that instead work for you?
>
> On Tue, Aug 28, 2012 at 6:05 PM, Tony Burton <TBurton@sportingindex.com> =
wrote:
>> Hi,
>>
>> I've seen that org.apache.hadoop.mapred.lib.MultipleTextOutputFormat is =
good for writing results into (for example) different directories created o=
n the fly. However, now I'm implementing a MapReduce job using Hadoop 1.0.3=
, I see that the new API no longer supports MultipleTextOutputFormat. Is th=
ere an equivalent that I can use, or will it be supported in a future relea=
se?
>>
>> Thanks,
>>
>> Tony
>>
>>
>> **********************************************************************
>> This email and any attachments are confidential, protected by copyright =
and may be legally privileged.  If you are not the intended recipient, then=
 the dissemination or copying of this email is prohibited. If you have rece=
ived this in error, please notify the sender by replying by email and then =
delete the email completely from your system.  Neither Sporting Index nor t=
he sender accepts responsibility for any virus, or any other defect which m=
ight affect any computer or IT system into which the email is received and/=
or opened.  It is the responsibility of the recipient to scan the email and=
 no responsibility is accepted for any loss or damage arising in any way fr=
om receipt or use of this email.  Sporting Index Ltd is a company registere=
d in England and Wales with company number 2636842, whose registered office=
 is at Gateway House, Milverton Street, London, SE11 4AP.  Sporting Index L=
td is authorised and regulated by the UK Financial Services Authority (reg.=
 no. 150404) and Gambling Commission (reg. no. 000-027343-R-308898-001).  A=
ny financial promotion contained herein has been issued
>> and approved by Sporting Index Ltd.
>>
>> Outbound email has been scanned for viruses and SPAM
>>
>
>
>
> --
> Harsh J
> www.sportingindex.com
> Inbound Email has been scanned for viruses and SPAM
> **********************************************************************
> This email and any attachments are confidential, protected by copyright a=
nd may be legally privileged.  If you are not the intended recipient, then =
the dissemination or copying of this email is prohibited. If you have recei=
ved this in error, please notify the sender by replying by email and then d=
elete the email completely from your system.  Neither Sporting Index nor th=
e sender accepts responsibility for any virus, or any other defect which mi=
ght affect any computer or IT system into which the email is received and/o=
r opened.  It is the responsibility of the recipient to scan the email and =
no responsibility is accepted for any loss or damage arising in any way fro=
m receipt or use of this email.  Sporting Index Ltd is a company registered=
 in England and Wales with company number 2636842, whose registered office =
is at Gateway House, Milverton Street, London, SE11 4AP.  Sporting Index Lt=
d is authorised and regulated by the UK Financial Services Authority (reg. =
no. 150404) and Gambling Commission (reg. no. 000-027343-R-308898-001).  An=
y financial promotion contained herein has been issued
> and approved by Sporting Index Ltd.
>
> Outbound email has been scanned for viruses and SPAM


--=20
Harsh J
www.sportingindex.com
Inbound Email has been scanned for viruses and SPAM=20
=
**********************************************************************=0D=
=0AThis email and any attachments are confidential, protected by copyri=
ght and may be legally privileged.  If you are not the intended recipie=
nt, then the dissemination or copying of this email is prohibited. If y=
ou have received this in error, please notify the sender by replying by=
 email and then delete the email completely from your system.  Neither =
Sporting Index nor the sender accepts responsibility for any virus, or =
any other defect which might affect any computer or IT system into whic=
h the email is received and/or opened.  It is the responsibility of the=
 recipient to scan the email and no responsibility is accepted for any =
loss or damage arising in any way from receipt or use of this email.  S=
porting Index Ltd is a company registered in England and Wales with com=
pany number 2636842, whose registered office is at Gateway House, Milve=
rton Street, London, SE11 4AP.  Sporting Index Ltd is authorised and re=
gulated by the UK Financial Services Authority (reg. no. 150404) and Ga=
mbling Commission (reg. no. 000-027343-R-308898-001).  Any financial pr=
omotion contained herein has been issued=20=0D=0Aand approved by Sporti=
ng Index Ltd.=0D=0A=0D=0AOutbound email has been scanned for viruses an=
d SPAM=0D=0A=