chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ari Rabkin (JIRA)" <>
Subject [jira] Commented: (CHUKWA-369) proposed reliability mechanism
Date Wed, 05 Aug 2009 17:07:14 GMT


Ari Rabkin commented on CHUKWA-369:

I had thought LocalWriter was finished and working. What are the known limitations?

As I see it, there are two problems with LocalWriter.  First, that you can run out of local
disk. Second, if the collector crashes, data will be lost unless and until the collector comes
back up. If the node dies, data is lost.   The approach I'm suggesting is entirely flushless,
and avoids both problems. Also, it can coexist gracefully with the existing localwriter, or
with with base SeqFileWriter.

> proposed reliability mechanism
> ------------------------------
>                 Key: CHUKWA-369
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>          Components: data collection
>    Affects Versions: 0.3.0
>            Reporter: Ari Rabkin
>             Fix For: 0.3.0
> We like to say that Chukwa is a system for reliable log collection. It isn't, quite,
since we don't handle collector crashes.  Here's a proposed reliability mechanism.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message