hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-11251) ConcurrentModificationException during DataNode#refreshVolumes
Date Wed, 21 Dec 2016 20:01:02 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Manoj Govindassamy updated HDFS-11251:
    Attachment: HDFS-11251.01.patch

Attaching v01 patch to address the following.
1. Made {{Storage#storageDirs}} a CopyOnWriteArrayList instead of a normal ArrayList so that
concurrent iterators and writers don't end up in {{ConcurrentModificationException}}. In this
case, substitution with CopyOnWriteArrayList is ok and should not cause any peformance degradation
as Volume add and removal are not frequent operations. 
2. Tests updated to add more volumes during write and to expose the race condition. On a private
branched I recreated the original CME problem with this test along with some delays introduced
in {{Storage#containsStorageDir}}

This patch will throw a CheckStyle issue on {{Storage#storageDirs}} not being private. But,
this checkstyle issue existed even before the patch and touching the same line again will
cause the checkstyle to show up again. Will file a new jira to cleanup the specifier for {{Storage#storageDirs}}
and its access patterns in the derived classes.

[~eddyxu], [~xiaochen], [~linyiqun], Can you please review the patch ?

> ConcurrentModificationException during DataNode#refreshVolumes
> --------------------------------------------------------------
>                 Key: HDFS-11251
>                 URL: https://issues.apache.org/jira/browse/HDFS-11251
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Jason Lowe
>            Assignee: Manoj Govindassamy
>         Attachments: HDFS-11251.01.patch
> The testAddVolumesDuringWrite case failed with a ReconfigurationException which appears
to have been caused by a ConcurrentModificationException.  Stacktrace details to follow.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message