hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manoj Babu <manoj...@gmail.com>
Subject Reg: parsing all files & file append
Date Sun, 09 Sep 2012 15:58:54 GMT
Hi All,

I have two questions, providing info on it will be helpful.

1, I am using hadoop to analyze and to find top n search term metric's from
logs.
If any new log file is added to HDFS then again we are running the job to
find the metrics.
Daily we will be getting log files and we are parsing the whole file and
getting the metric's.
All the log file's are parsed daily to get the latest metric's is there any
way is there any way to avoid this?

2, Does file append is production stable?

Cheers!
Manoj.

Mime
View raw message