hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7261) storageMap is accessed without synchronization in DatanodeDescriptor#updateHeartbeatState()
Date Thu, 05 Mar 2015 18:50:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349245#comment-14349245
] 

Colin Patrick McCabe commented on HDFS-7261:
--------------------------------------------

Thanks, Brahma.

It looks like storageMap is also being accessed without synchronization here:
{code}
    if (storageMap.size() != reports.length) {
      pruneStorageMap(reports);
    }
 {code}

and here:
{code}
  private void pruneStorageMap(final StorageReport[] reports) {
    if (LOG.isDebugEnabled()) {
      LOG.debug("Number of storages reported in heartbeat=" + reports.length +
                    "; Number of storages in storageMap=" + storageMap.size());
    }

    HashMap<String, DatanodeStorageInfo> excessStorages;

    synchronized (storageMap) {
      // Init excessStorages with all known storages.
{code}

We should be able to avoid the lack of synchronization, and avoid holding the lock for a long
time, by getting the size() before we release the lock.

> storageMap is accessed without synchronization in DatanodeDescriptor#updateHeartbeatState()
> -------------------------------------------------------------------------------------------
>
>                 Key: HDFS-7261
>                 URL: https://issues.apache.org/jira/browse/HDFS-7261
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Ted Yu
>            Assignee: Brahma Reddy Battula
>            Priority: Minor
>         Attachments: HDFS-7261.patch
>
>
> Here is the code:
> {code}
>       failedStorageInfos = new HashSet<DatanodeStorageInfo>(
>           storageMap.values());
> {code}
> In other places, the lock on "DatanodeDescriptor.storageMap" is held:
> {code}
>     synchronized (storageMap) {
>       final Collection<DatanodeStorageInfo> storages = storageMap.values();
>       return storages.toArray(new DatanodeStorageInfo[storages.size()]);
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message