hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Yakshin <greycat.na....@gmail.com>
Subject Re: Using Hadoop for near real-time processing of log data
Date Wed, 25 Feb 2009 18:13:45 GMT

> Is anyone using Hadoop as more of a near/almost real-time processing
> of log data for their systems to aggregate stats, etc?

We do, although "near realtime" is pretty relative subject and your
mileage may vary. For example, startups / shutdowns of Hadoop jobs are
pretty expensive and it could take anything from 5-10 seconds up to
several minutes to get the job started and almost same thing goes for
job finalization. Generally, if your "near realtime" would tolerate
3-4-5 minutes lag, it's possible to use Hadoop.

WBR, Mikhail Yakshin

View raw message