hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharath Mundlapudi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2123) 1073: Checkpoint interval should be based on txn count, not size
Date Tue, 05 Jul 2011 18:01:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13060046#comment-13060046
] 

Bharath Mundlapudi commented on HDFS-2123:
------------------------------------------

I have reviewed the patch. This change is more meaningful than check pointing based on size.
+1 to the approach. 

There are some minor comments on logging msgs since we are moving to txns, we should reflect
this in log msgs too.

1. Replace in Checkpointer

+    LOG.info("Log Size Trigger  : " + checkpointTxnCount + " txns ");
With
+    LOG.info("Transaction Count Trigger  : " + checkpointTxnCount + " txns ");

2. Replace in SecondaryNameNode
+      + "\nCheckpoint Size      : " + StringUtils.byteDesc(checkpointTxnCount)
+                                    + " (= " + checkpointTxnCount + " bytes)" 
With
+      + "\nTransaction Count      : " + StringUtils.byteDesc(checkpointTxnCount)
+                                    + " (= " + checkpointTxnCount + " txns)" 

3. Replace in SecondaryNamenode
+    LOG.info("Log Size Trigger    :" + checkpointTxnCount + " txns");
with
+    LOG.info("Transaction Count Trigger  : " + checkpointTxnCount + " txns ");


4. Replace in SecondaryNamenode
+          System.err.println("EditLog size " + count + " transactions is " +
                              "smaller than configured checkpoint " +
+                             "interval " + checkpointTxnCount + " transactions.");
with
+          System.err.println("EditLog transactions " + count + " is " +
                              "smaller than configured checkpoint " +
+                             "transactions " + checkpointTxnCount);




> 1073: Checkpoint interval should be based on txn count, not size
> ----------------------------------------------------------------
>
>                 Key: HDFS-2123
>                 URL: https://issues.apache.org/jira/browse/HDFS-2123
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: Edit log branch (HDFS-1073)
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: Edit log branch (HDFS-1073)
>
>         Attachments: hdfs-2123.txt, hdfs-2123.txt
>
>
> Currently, the administrator can configure the secondary namenode to checkpoint either
every N seconds, or every N bytes worth of edit log. It would make more sense to get rid of
the size-based interval and instead allow the administrator to specify checkpoints every N
transactions. This also simplifies the code a little bit.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message