hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shashwat shriparv <dwivedishash...@gmail.com>
Subject Re: queues in haddop
Date Fri, 11 Jan 2013 05:31:54 GMT
The attached screenshot will shows how flume will work, and also you can
consider RabbitMQ, as it persistent too..



∞
Shashwat Shriparv



On Fri, Jan 11, 2013 at 10:24 AM, Mohit Anchlia <mohitanchlia@gmail.com>wrote:

> Have you looked at flume?
>
> Sent from my iPhone
>
> On Jan 10, 2013, at 7:12 PM, Panshul Whisper <ouchwhisper@gmail.com>
> wrote:
>
> > Hello,
> >
> > I have a hadoop cluster setup of 10 nodes and I an in need of
> implementing queues in the cluster for receiving high volumes of data.
> > Please suggest what will be more efficient to use in the case of
> receiving 24 Million Json files.. approx 5 KB each in every 24 hours :
> > 1. Using Capacity Scheduler
> > 2. Implementing RabbitMQ and receive data from them using Spring
> Integration Data pipe lines.
> >
> > I cannot afford to loose any of the JSON files received.
> >
> > Thanking You,
> >
> > --
> > Regards,
> > Ouch Whisper
> > 010101010101
>

Mime
View raw message