incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Rabkin <asrab...@gmail.com>
Subject Re: SocketTeeWriter
Date Mon, 10 May 2010 23:37:57 GMT
That's how we use it at Berkeley, to process metrics from hundreds of
machines; total data rate less than a megabyte per second, though.
What scale of data are you looking at?

The intent of SocketTee was if you need some subset of the data now,
while write-to-HDFS-and-process-with-Hadoop is still the default path.
 What sort of low-latency processing do you need?

--Ari

On Mon, May 10, 2010 at 4:28 PM, Corbin Hoenes <corbin@tynt.com> wrote:
> Has anyone used the "Tee" in a larger scale deployment to try to get real-time/low latency
data?  Interested in how feasible it would be to use it to pipe data into another system
to handle these low latency requests and leave the long term analysis to hadoop.
>
>



-- 
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department

Mime
View raw message