Mailing-List: contact chukwa-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: chukwa-dev@hadoop.apache.org
Message-ID: <854652961.681041270404447337.JavaMail.jira@brutus.apache.org>
Date: Sun, 4 Apr 2010 18:07:27 +0000 (UTC)
From: "Ari Rabkin (JIRA)" <jira@apache.org>
To: chukwa-dev@hadoop.apache.org
Subject: [jira] Commented: (CHUKWA-4) Collectors don't finish writing .done
 datasink from last .chukwa datasink when stopped using bin/stop-collectors
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CHUKWA-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853272#action_12853272 ] 

Ari Rabkin commented on CHUKWA-4:
---------------------------------

I don't think you need to identify the bad chunk. Why not just move the "close new file and rename things" logic into the exception handler? I would split the handler into "ChecksumException" -- where you recover by closing and renaming the file -- and everything else, which you should log the way you do now.

> Collectors don't finish writing .done datasink from last .chukwa datasink when stopped using bin/stop-collectors
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: CHUKWA-4
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-4
>             Project: Hadoop Chukwa
>          Issue Type: Bug
>          Components: data collection
>         Environment: I am running on our local cluster. This is a linux machine that I also run Hadoop cluster from.
>            Reporter: Andy Konwinski
>            Priority: Minor
>
> When I use start-collectors, it creates the datasink as expected, writes to it as per normal, i.e. writes to the .chukwa file, and roll overs work fine when it renames the .chukwa file to .done. However, when I use bin/stop-collectors to shut down the running collector it leaves a .chukwa file in the HDFS file system. Not sure if this is a valid sink or not, but I think that the collector should gracefully clean up the datasink and rename it .done before exiting.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.