hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Upendra Dadi <ud...@gmu.edu>
Subject Pipes with the latest version of mapReduce
Date Sat, 28 Nov 2009 03:04:25 GMT
Hi,
  I am trying to use MapReduce Pipes with version 0.20.1. of hadoop. If I 
used  org.apache.hadoop.mapreduce.lib.input.TextInputFormat as the input 
format with the             -InputFormat option, I am getting the following 
error:
    Exception in thread "main" java.lang.ClassCastException: class 
org.apache.hadoop.mapreduce.lib.input.TextInputFormat
     at java.lang.Class.asSubclass(Class.java:3018)
     at 
org.apache.hadoop.mapred.pipes.Submitter.getClass(Submitter.java:372)
     at org.apache.hadoop.mapred.pipes.Submitter.run(Submitter.java:421)
     at org.apache.hadoop.mapred.pipes.Submitter.main(Submitter.java:494)

I was able to run Pipes without the -InputFormat option, but my requirement 
is to extend TextInputFormat.  I guess Pipes is assuming the inputFormat  to 
be of type org.apache.hadoop.mapred.InputFormat, which is deprecated. So I 
guess 0.20.1 is not fully ported to work with Pipes. Am I right here? Which 
version should I use? Thanks.

Upendra



----- Original Message ----- 
From: "Erez Katz" <erez_katz@yahoo.com>
To: <common-dev@hadoop.apache.org>
Sent: Friday, November 27, 2009 6:59 PM
Subject: Re: mapreduce with non-text data


> you could always base64 encode your binary data...
>
>
>
> --- On Thu, 11/26/09, Upendra Dadi <udadi@gmu.edu> wrote:
>
>> From: Upendra Dadi <udadi@gmu.edu>
>> Subject: mapreduce with non-text data
>> To: common-dev@hadoop.apache.org
>> Date: Thursday, November 26, 2009, 5:07 PM
>> Hi,
>>  Are there any use cases, examples of use of Hadoop
>> MapReduce for non-text data? Only examples that I see on the
>> web are for text data. Any pointers in that direction is
>> greatly appreciated. Thanks.
>>
>> Regards,
>> Upendra
>>
>
>
> 


Mime
View raw message