hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From madhu phatak <phatak....@gmail.com>
Subject Re: Best way to Merge small XML files
Date Thu, 03 Feb 2011 10:44:30 GMT
Hi
You can write an InputFormat which create input splits from multiple files .
It will solve your problem.

On Wed, Feb 2, 2011 at 4:04 PM, Shuja Rehman <shujamughal@gmail.com> wrote:

> Hi Folks,
>
> I am having hundreds of small xml files coming each hour. The size varies
> from 5 Mb to 15 Mb. As Hadoop did not work well with small files so i want
> to merge these small files. So what is the best option to merge these xml
> files?
>
>
>
> --
> Regards
> Shuja-ur-Rehman Baig
> <http://pk.linkedin.com/in/shujamughal>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message