hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5498) Improve datanode startup time
Date Thu, 13 Feb 2014 17:22:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900516#comment-13900516
] 

Kihwal Lee commented on HDFS-5498:
----------------------------------

bq. saveDfsUsed(); will store the cache file, its called only in BlockPoolSlice#shutdown(),
but in real environment this will never be called. So no cache fill be written.

Actually it is being called during shutdown.
{panel}
        at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.BlockPoolSlice.shutdown(BlockPoolSlice.java:391)
        at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeImpl.shutdownBlockPool(FsVolumeImpl.java:265)
        at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsVolumeList.removeBlockPool(FsVolumeList.java:242)
        at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.shutdownBlockPool(FsDatasetImpl.java:1779)
        at org.apache.hadoop.hdfs.server.datanode.DataNode.shutdownBlockPool(DataNode.java:909)
        at org.apache.hadoop.hdfs.server.datanode.BPOfferService.shutdownActor(BPOfferService.java:362)
        at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.cleanUp(BPServiceActor.java:611)
        at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:811)
        at java.lang.Thread.run(Thread.java:695)
{panel}

> Improve datanode startup time
> -----------------------------
>
>                 Key: HDFS-5498
>                 URL: https://issues.apache.org/jira/browse/HDFS-5498
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>         Attachments: HDFS-5498.with_du_change.patch, HDFS-5498.with_du_change.patch
>
>
> Similarly to HDFS-5027, an improvement  can be made for getVomeMap(). This is the phase
in which ReplicaMap.is populated.  But it will be even better if datanode scans only once
and do both.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message