Mailing-List: contact chukwa-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: chukwa-dev@hadoop.apache.org
Message-ID: <856755208.1244618467380.JavaMail.jira@brutus>
Date: Wed, 10 Jun 2009 00:21:07 -0700 (PDT)
From: "Ari Rabkin (JIRA)" <jira@apache.org>
To: chukwa-dev@hadoop.apache.org
Subject: [jira] Created: (CHUKWA-284) division of responsibility between
 agent and adaptor
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

division of responsibility between agent and adaptor
----------------------------------------------------

                 Key: CHUKWA-284
                 URL: https://issues.apache.org/jira/browse/CHUKWA-284
             Project: Hadoop Chukwa
          Issue Type: Bug
          Components: data collection
    Affects Versions: 0.1.2, 0.2.0
            Reporter: Ari Rabkin
             Fix For: 0.2.0


The division of responsibility between adaptor and agent for managing offsets, datatypes, and such has gotten quite squirrely. And the comments don't match the code.

For instance, the adaptors track last offset.  This means that checkpoint files include the last byte enqueued to be SENT, rather than the last byte ACKNOWLEDGED. This violates both the spec and the comments.

I propose that we make the agent responsible for recording offsets and possibly also datatypes.  Are there cases where an Adaptor decides on its own what the Datatype is for the chunks it outputs?  Or outputs chunks of different Types?


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.