hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1256) Dfs image loading and edits loading creates multiple instances of DatanodeDescriptor for the same datanode
Date Fri, 13 Apr 2007 17:51:15 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12488729
] 

Konstantin Shvachko commented on HADOOP-1256:
---------------------------------------------

General comment:

We should treat differently optimization patches.
It would be good to have some measurements that prove the optimization really
works and worth the complexity involved.

These bugs are related to the optimization issue HADOOP-971, which introduced
a new data-structure in the name-node in order to accelerate access to data-nodes by name.
It involves more complexity in synchronizing the new map with the other structures.
We still don't know what the benefits are.

We have TestDFSIO and NNBench to measure the performance.
In this particular case the cluster should have a lot of data-nodes, so it probably needs
a custom benchmark, which should run imo both on a small cluster to show the existing
performance does not degrade and on a large one to show the advantages.

I'd propose to make it a requirement for committing optimization patches.


> Dfs image loading and edits loading creates multiple instances of DatanodeDescriptor
for the same datanode
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-1256
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1256
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.12.3
>            Reporter: Hairong Kuang
>         Assigned To: Hairong Kuang
>             Fix For: 0.13.0
>
>         Attachments: nodeMap.patch
>
>
> This leads to multiple instances of DatanodeDescriptors for the same datanode stored
in Host2DatanodeMap.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message