hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yanbo Liang <yanboha...@gmail.com>
Subject Re: Number of concurrent writer to HDFS
Date Mon, 06 Aug 2012 06:14:57 GMT
You can use scribe or flume to collect log data and integrated with hadoop.

2012/8/4 Nguyen Manh Tien <tien.nguyenmanh@gmail.com>

> Hi,
> I plan to streaming logs data HDFS using many writer, each writer write a
> stream of data to a HDFS file (may rotate)
>
> I wonder how many concurrent writer i should use?
> And if you have that experience please share to me : hadoop cluster size,
> number of writer, replication.
>
> Thanks.
> Tien
>

Mime
View raw message