hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jakob Homan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1572) Checkpointer should trigger checkpoint with specified period.
Date Tue, 11 Jan 2011 23:02:46 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980404#action_12980404
] 

Jakob Homan commented on HDFS-1572:
-----------------------------------

The getJournalSize() rpc is only called once every five minutes to check if the journal size
is too big.  The 60 seconds is just to check if that call needs to be made, or if the duration-based
checkpointing should be done.  The 60 seconds improves the granularity of the duration.  Currently
it's duration+5 minutes at a maximum.  This moves it to duration+1 minute at a maximum.  The
same number of RPC calls are being made since the getJournalSize() branch is only traversed
every 5 minutes.  It is true that if the period has expired (once an hour by default, once
a day on all production systems I've seen), we'd do a single extra RPC call.  I think this
is worth it to improve code clarity.

> Checkpointer should trigger checkpoint with specified period.
> -------------------------------------------------------------
>
>                 Key: HDFS-1572
>                 URL: https://issues.apache.org/jira/browse/HDFS-1572
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.22.0
>            Reporter: Liyin Liang
>            Priority: Blocker
>             Fix For: 0.21.0
>
>         Attachments: 1527-1.diff, 1572-2.diff, HDFS-1572-2.patch, HDFS-1572.patch
>
>
> {code:}
>   long now = now();
>   boolean shouldCheckpoint = false;
>   if(now >= lastCheckpointTime + periodMSec) {
>     shouldCheckpoint = true;
>   } else {
>     long size = getJournalSize();
>     if(size >= checkpointSize)
>       shouldCheckpoint = true;
>   }
> {code}
> {dfs.namenode.checkpoint.period} in configuration determines the period of checkpoint.
However, with above code, the Checkpointer triggers a checkpoint every 5 minutes (periodMSec=5*60*1000).
According to SecondaryNameNode.java, the first *if*  statement should be:
>  {code:}
> if(now >= lastCheckpointTime + 1000 * checkpointPeriod) {
>  {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message