hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mice <mice1...@gmail.com>
Subject Re: help: InputFormat problem ?
Date Mon, 27 Oct 2008 07:26:08 GMT
I think both of your solution work.
Saving the result into a file is trivial. In related to process the
ResultSet as Input , you need to write your customized inputFormat.

If your web service provides splitable results, it is similiar to read
from a DB;  you can check DBInputFormat in trunk

http://svn.apache.org/viewvc/hadoop/core/trunk/src/mapred/org/apache/hadoop/mapred/lib/db/DBInputFormat.java?view=markup

Regards
Mice

2008/10/27 ZhiHong Fu <ddream84@gmail.com>:
> Hello :
>
>        In hadoop  InputFormat are always  based on the InputFileFormat ,
> But Now I will get data from a web service application. The data will be
> wrapped as ResultSet type. Now  I am wandering " should I write the
> ResultSet to a file And then read out to do mapreduce job. Or How can I
> process the ResultSet as Input of the MapReduce Job, and How should I define
> the InputFormat"  Any suggestions are welcome , thanks!
>

Mime
View raw message