hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Loddengaard" <a...@cloudera.com>
Subject Re: Recovery from Failed Jobs
Date Tue, 04 Nov 2008 18:38:22 GMT
With regard to checkpointing, not yet.  This JIRA is a prerequisite:

I'm a little confused about what you're trying to do with log parsing.  You
should consider Scribe or Chukwa, though Chukwa isn't ready to be used yet.
 Learn more here:




On Tue, Nov 4, 2008 at 11:51 AM, shahab mehmandoust <shahab53@gmail.com>wrote:

> Hello,
> I want to parse lines of an access logs, line by line with map/reduce.  I
> want to know, once my access log is in the HDFS, am I guaranteed that every
> line will be processed and results will be in the output dir?  In other
> words, if a job fails, does hadoop know where it failed? and can hadoop
> recover from that point so no data is lost?
> Thanks,
> Shahab

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message