hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4539) Streaming Edits to a Standby Name-Node.
Date Wed, 29 Oct 2008 22:36:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12643684#action_12643684

Suresh Srinivas commented on HADOOP-4539:

Comments organized by section:
General comments:
- More description on how the NN and SNN startup and communicate with each other would be
good to have.
- Design will only work for one standby namenode. For example if a standby node starts spooling,
followed by another standby node how does flow work?

- Proposal is for SNN to treat each received record as name-node command and to call respective
name-node method to update namespace. This is a not a good idea. For example modification
time etc. which are based on NN will be different on SNN. SNN should get indication of what
changed and apply it to its copy of namespace.
- How is the reliability of edit changes ensured? That is, when NN sends records to SNN how
are the following ensured?
  - Did SNN application really receive the record?
  - Did SNN apply that record to its namespace copy?
  - Did SNN store the the record in secondary storage reliably?

Journal Spool:
- Clarify that JS is stored as a file on the harddisk

Admin command saveCheckpoint
Not very clear on the following sentence. Does NN keep running if all the storage directories
fail? Why should it then enter safe mode?
- NN should keep running even if the last storage directory fails. It should enter safe mode
in this case.

> Streaming Edits to a Standby Name-Node.
> ---------------------------------------
>                 Key: HADOOP-4539
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4539
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: dfs
>            Reporter: Konstantin Shvachko
>             Fix For: 0.20.0
>         Attachments: image001.gif, StreamEditsToSNN.htm
> Currently Secondary name-node acts as mere checkpointer.
> Secondary name-node should be transformed into a standby name-node (SNN). 
> The long term goal is to make it a warm standby. 
> The purpose of this issue is to provide real time streaming of edits to SNN so that it
contained the up-to-date namespace state.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message