chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-313) Remove the 10 sec ack from SeqWriter
Date Fri, 19 Jun 2009 00:03:07 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721569#action_12721569
] 

Jerome Boulon commented on CHUKWA-313:
--------------------------------------

TestChukwaWriters is the test case that I have used to make sure that the dataSink file is
still correct

    [junit] Running org.apache.hadoop.chukwa.datacollection.writer.TestChukwaWriters
    [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 11.458 sec



> Remove the 10 sec ack from SeqWriter
> ------------------------------------
>
>                 Key: CHUKWA-313
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-313
>             Project: Hadoop Chukwa
>          Issue Type: Improvement
>    Affects Versions: 0.2.0, 0.3.0
>            Reporter: Jerome Boulon
>            Assignee: Jerome Boulon
>             Fix For: 0.2.0, 0.3.0
>
>         Attachments: CHUKWA-313.patch
>
>
> SeqWriter.add is waiting 10 sec in a worse case before calling a flush. This was there
to guarantee that the data has been written to HDFS before returning the ack.
> It used to work well with 0.18 but in 0.20 the flush is a nohup so there's no point in
waiting 10 sec for nothing.
> The real flush will give us the ability to write redo logs directly to HDFS so the ack
implementation will have to change anyway.
> So I vote for removing the 10 sec ack from the add method.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message