hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1508) Ability to do savenamespace without being in safemode
Date Fri, 03 Dec 2010 08:17:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966448#action_12966448
] 

Konstantin Shvachko commented on HDFS-1508:
-------------------------------------------

> I am unable to write a unit test that would trigger all or any of the races

This is exactly my point. There is a whole chess game going on underneath with moving files/directories
and threads writing in parallel. Changing the position of one pawn can change the outcome
of the game.
If saveNamespace() succeeds we are lucky and checkpoint fails. If not then somebody has to
clean up the mess and there is lots of failure scenarios. We with Todd once spent quite some
time sorting out all of them. May be I am paranoid and your change doesn't change the game,
but it needs some convincing argumentation, which is hard.
That is why I was asking alternatively about the use case. I understand setting NN in safe
mode causes jobs failure. But why do you need to call saveNamespace()? What is wrong with
checkpointing?

> Ability to do savenamespace without being in safemode
> -----------------------------------------------------
>
>                 Key: HDFS-1508
>                 URL: https://issues.apache.org/jira/browse/HDFS-1508
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: savenamespaceWithoutSafemode.txt, savenamespaceWithoutSafemode2.txt,
savenamespaceWithoutSafemode3.txt
>
>
> In the current code, the administrator can run savenamespace only after putting the namenode
in safemode. This means that applications that are writing to HDFS encounters errors because
the NN is in safemode. We would like to allow saveNamespace even when the namenode is not
in safemode.
> The savenamespace command already acquires the FSNamesystem writelock. There is no need
to require that the namenode is in safemode too.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message