Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of shahab.yunus@gmail.com
 designates 209.85.215.46 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <290DD281-B0E7-4AD2-84C3-A981907E2C7B@gmail.com>
References: <E1A57EFB-CD4B-4143-B49C-EAF697F1193B@gmail.com>
	<CACc2-YsXcmSeNSeYB3ohGoxAqPOD4+w8w11iRzVLbZ=8BWqMGA@mail.gmail.com>
	<CAEGXsyCQmD2-fO8pS1VSVGZn1AnoyPbsJiStB250OwELFtHY+g@mail.gmail.com>
	<DE2B0D36-3DEA-479A-B94D-283732DCE367@gmail.com>
	<CAEGXsyDh_qb88hTtsF35D2KO7wMVhVJtzY5mkw-pykk8WAJHTQ@mail.gmail.com>
	<290DD281-B0E7-4AD2-84C3-A981907E2C7B@gmail.com>
Date: Wed, 10 Sep 2014 14:47:41 -0400
Message-ID: 
 <CAEo-6+RPRVA4_RAFu375DZf1tDTpH47UwS3j2y4Tm7V1OjNPgg@mail.gmail.com>
Subject: Re: Writing output from streaming task without dealing with key/value
From: Shahab Yunus <shahab.yunus@gmail.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=089e013d1708a561280502ba7c11

--089e013d1708a561280502ba7c11
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Examples (the top ones are related to streaming jobs):

http://www.infoq.com/articles/HadoopOutputFormat
http://research.neustar.biz/2011/08/30/custom-inputoutput-formats-in-hadoop=
-streaming/
http://stackoverflow.com/questions/12759651/how-to-override-inputformat-and=
-outputformat-in-hadoop-application

Regards,
Shahab

On Wed, Sep 10, 2014 at 2:28 PM, Dmitry Sivachenko <trtrmitya@gmail.com>
wrote:

>
> On 10 =D1=81=D0=B5=D0=BD=D1=82. 2014 =D0=B3., at 22:19, Rich Haase <rdhaa=
se@gmail.com> wrote:
>
> > You can write a custom output format
>
>
> Any clues how can this can be done?
>
>
>
> > , or you can write your mapreduce job in Java and use a NullWritable as
> Susheel recommended.
> >
> > grep (and every other *nix text processing command) I can think of woul=
d
> not be limited by a trailing tab character.  It's even quite easy to stri=
p
> away that tab character if you don't want it during the post processing
> steps you want to perform with *nix commands.
>
>
> Problem is that the line itself contains a TAB in the middle, there will
> not be extra trailing TAB at the end.
> So it is not that simple.
> You never know if it is a TAB from the original line or it is extra TAB
> added by TextOutputFormat.
>
> Thanks!

--089e013d1708a561280502ba7c11
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Examples (the top ones are related to streaming jobs):<div=
><br></div><div><a href=3D"http://www.infoq.com/articles/HadoopOutputFormat=
">http://www.infoq.com/articles/HadoopOutputFormat</a><br></div><div><a hre=
f=3D"http://research.neustar.biz/2011/08/30/custom-inputoutput-formats-in-h=
adoop-streaming/">http://research.neustar.biz/2011/08/30/custom-inputoutput=
-formats-in-hadoop-streaming/</a><br></div><div><a href=3D"http://stackover=
flow.com/questions/12759651/how-to-override-inputformat-and-outputformat-in=
-hadoop-application">http://stackoverflow.com/questions/12759651/how-to-ove=
rride-inputformat-and-outputformat-in-hadoop-application</a><br></div><div>=
<br></div><div>Regards,</div><div>Shahab</div></div><div class=3D"gmail_ext=
ra"><br><div class=3D"gmail_quote">On Wed, Sep 10, 2014 at 2:28 PM, Dmitry =
Sivachenko <span dir=3D"ltr">&lt;<a href=3D"mailto:trtrmitya@gmail.com" tar=
get=3D"_blank">trtrmitya@gmail.com</a>&gt;</span> wrote:<br><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;pa=
dding-left:1ex"><br>
On 10 =D1=81=D0=B5=D0=BD=D1=82. 2014 =D0=B3., at 22:19, Rich Haase &lt;<a h=
ref=3D"mailto:rdhaase@gmail.com">rdhaase@gmail.com</a>&gt; wrote:<br>
<br>
&gt; You can write a custom output format<br>
<br>
<br>
Any clues how can this can be done?<br>
<br>
<br>
<br>
&gt; , or you can write your mapreduce job in Java and use a NullWritable a=
s Susheel recommended.<br>
&gt;<br>
&gt; grep (and every other *nix text processing command) I can think of wou=
ld not be limited by a trailing tab character.=C2=A0 It&#39;s even quite ea=
sy to strip away that tab character if you don&#39;t want it during the pos=
t processing steps you want to perform with *nix commands.<br>
<br>
<br>
Problem is that the line itself contains a TAB in the middle, there will no=
t be extra trailing TAB at the end.<br>
So it is not that simple.<br>
You never know if it is a TAB from the original line or it is extra TAB add=
ed by TextOutputFormat.<br>
<br>
Thanks!</blockquote></div><br></div>

--089e013d1708a561280502ba7c11--