hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers" <...@cloudera.com>
Subject Re: Question for edit log retain
Date Mon, 01 Jul 2013 23:02:54 GMT
Hi Azuryy,

On Wed, Jun 26, 2013 at 6:12 PM, Azuryy Yu <azuryyyu@gmail.com> wrote:

> Hi Dear all,
>
> I have some confusion for edit log retain,
>
> NNStorageRetentionManager.java:
>
> 1)
> purgeCheckpointsOlderThan()
>   What's mean of check point here?
>

Here, "checkpoint" is referring to the fsimage files on the NN.


>
> 2)purgeOldStorage()
>   I cannot understand the calculation of minimum txid, I think I can
> understand it if I know these keys indicates.
> DFS_NAMENODE_NUM_CHECKPOINTS_RETAINED_KEY
>

This is the number of old fsimage files to retain on the NN during purging.


> DFS_NAMENODE_NUM_EXTRA_EDITS_RETAINED_KEY
>

This is the number of extra transactions to retain during purging. Here,
"extra" is referring to transactions beyond what is strictly required to
play back all FS changes since the last checkpoint (fsimage.)


> DFS_NAMENODE_MAX_EXTRA_EDITS_SEGMENTS_RETAINED_KEY
>

This is the maximum number of extra edit _segments_ to retain during
purging. Segments refer to finalized edit log files, with a start and end
transaction ID. A single edit file is a "segment."


>
> Can any developer give me a short detail explanation? Thanks much.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message