hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fenghua Hu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10804) Use separate lock for ReplicaMap
Date Wed, 28 Sep 2016 07:04:21 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15528661#comment-15528661

Fenghua Hu commented on HDFS-10804:

Thanks [~arpitagarwal] for updating the description to reflect the recent changes in HDFS-10828.
I updated a new version, could you please review it? 

Note that I haven't change other places, i think if it's feasible, we will need to change
many places, for example:
we could safely remove datasetLock.acquire() from following code:

  public FsVolumeImpl getVolume(final ExtendedBlock b) {
    try (AutoCloseableLock lock = datasetLock.acquire()) {
      final ReplicaInfo r =
          volumeMap.get(b.getBlockPoolId(), b.getLocalBlock());
      return r != null ? (FsVolumeImpl) r.getVolume() : null;

similar functions include: getStoredBlock() and some others functions which uses lock for
ReplicaMap exclusive access.
Once we agree to use separate lock for ReplicaMap, I'll modify them correspondingly. It's
expected to improve performance.

> Use separate lock for ReplicaMap
> --------------------------------
>                 Key: HDFS-10804
>                 URL: https://issues.apache.org/jira/browse/HDFS-10804
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: hdfs
>    Affects Versions: 3.0.0-beta1
>            Reporter: Fenghua Hu
>            Assignee: Fenghua Hu
>            Priority: Minor
>         Attachments: HDFS-10804-001.patch, HDFS-10804-002.patch, HDFS-10804-003.patch
> In currently implementation, ReplicaMap takes an external lock for synchronization.
> In function FsDatasetImpl#FsDatasetImpl(), the object is for synchronization is the same
lock object used by FsDatasetImpl routines.
> and in private FsDatasetImpl#addVolume(), the same lock is used for synchronization as
> {code}
> ReplicaMap tempVolumeMap = new ReplicaMap(datasetLock);
> {code}
> We can potentially eliminate the heavyweight lock for synchronizing ReplicaMap instances.
If it's not necessary, this could reduce lock contention on the datasetLock object and improve
> Could you please give me some suggestions? Thanks a lot!
> Fenghua

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message