hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kai Voigt...@123.org>
Subject Re: Best way to Merge small XML files
Date Thu, 03 Feb 2011 10:46:48 GMT
Did you look into Hadoop Archives?

http://hadoop.apache.org/mapreduce/docs/r0.21.0/hadoop_archives.html

Kai

Am 03.02.2011 um 11:44 schrieb madhu phatak:

> Hi
> You can write an InputFormat which create input splits from multiple files .
> It will solve your problem.
> 
> On Wed, Feb 2, 2011 at 4:04 PM, Shuja Rehman <shujamughal@gmail.com> wrote:
> 
>> Hi Folks,
>> 
>> I am having hundreds of small xml files coming each hour. The size varies
>> from 5 Mb to 15 Mb. As Hadoop did not work well with small files so i want
>> to merge these small files. So what is the best option to merge these xml
>> files?
>> 
>> 
>> 
>> --
>> Regards
>> Shuja-ur-Rehman Baig
>> <http://pk.linkedin.com/in/shujamughal>
>> 

-- 
Kai Voigt
k@123.org





Mime
View raw message