hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lohit Vijayarenu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3631) Transfer of image from secondary name node should not interrupt service
Date Wed, 06 Aug 2008 16:32:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620305#action_12620305
] 

Lohit Vijayarenu commented on HADOOP-3631:
------------------------------------------

After talking to Konstantin realized that having a shared directory would only get rid of
transferring the image back and forth between NameNode and SecondaryNameNode. Since we have
a requirement of maintaining the updated image consistent on all dfs.name.dir configured directories,
we would still have to copy the image from shared directory to those directories. This does
not solve the contention to the disk since we save image and edits on same disk. Many ppl
already have suggested having edits and image in separate disks which would help us in this
case. So, the above [approach|https://issues.apache.org/jira/browse/HADOOP-3631?focusedCommentId=12616652#action_12616652]
wont solve the actual problem.

> Transfer of image from secondary name node should not interrupt service
> -----------------------------------------------------------------------
>
>                 Key: HADOOP-3631
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3631
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.17.0
>            Reporter: Robert Chansler
>            Priority: Critical
>             Fix For: 0.19.0
>
>
> The transfer of the new image prepared by the secondary name node can interfere with
client services. Clients observe delays in completing RPCs. In general, administrative activities
should not be observed by the clients. For large clusters, administrators are reluctant to
run the secondary name node leading to excessive edit logs. (Excessive in the sense that if
the cluster must be restarted, a long time is required to process the log.)
> Maybe the new image does not have to be transfered; it could be fetched when needed.
> Maybe the priority of the transfer task can be reduced so that the transfer is not observed.
> Maybe a different transfer protocol is more appropriate.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message