hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma" <jssa...@facebook.com>
Subject RE: problems reading compressed sequencefiles in streaming (0.13.1)
Date Fri, 26 Oct 2007 17:22:03 GMT
Thanks!

A gigantic Duh moment.

-----Original Message-----
From: Runping Qi [mailto:runping@yahoo-inc.com] 
Sent: Friday, October 26, 2007 9:59 AM
To: hadoop-user@lucene.apache.org
Subject: RE: problems reading compressed sequencefiles in streaming (0.13.1)


Try to add the package name too:

o.a.h.m. SequenceFileAsTextInputFormat

Runping


> -----Original Message-----
> From: Joydeep Sen Sarma [mailto:jssarma@facebook.com]
> Sent: Friday, October 26, 2007 12:30 AM
> To: hadoop-user@lucene.apache.org
> Subject: problems reading compressed sequencefiles in streaming (0.13.1)
> 
> I was hoping to use -inputformat SequenceFileAsTextInputFormat to process
> compressed sequencefiles in streaming jobs
> 
> 
> 
> However, using a python mapper that just echoes out each line as it gets,
> and numreducetasks=0 - here's what the streaming job output looks like:
> 
> 
> 
> SEQ^F
> org.apache.hadoop.io.IntWritable^Yorg.apache.hadoop.io.Text^A^A'org.apache
> .hadoop.io.compress.GzipCodec^@^@^@^@Z+r������^F�
> 
> 
> 
> So seems like the input file was not treated as sequencefile.
> 
> 
> 
> I must be missing some args - except don't understand what. Help
> appreciated ..
> 
> 
> 
> Thx,
> 
> 
> 
> Joydeep


Mime
View raw message