hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-11267) Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in Storage
Date Wed, 28 Dec 2016 01:32:58 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Manoj Govindassamy updated HDFS-11267:
    Attachment: HDFS-11267.01.patch

Attaching v01 patch to address the following:

1. Avoid storageDirs getting re-defined in NNStorage. The redefinition is not needed as HDFS-11251
is anyway making the parent member variable stroageDir a {{CopyOnWriteArrayList}}
2. Make {{Storage#storageDirs}} a private member and expose get() method for the same.
3. Clean up all direct accessors of storageDirs with getStorageDirs() method

[~eddyxu], can you please take a look at the patch ?

> Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in Storage
> -----------------------------------------------------------------------------------
>                 Key: HDFS-11267
>                 URL: https://issues.apache.org/jira/browse/HDFS-11267
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Manoj Govindassamy
>            Assignee: Manoj Govindassamy
>         Attachments: HDFS-11267.01.patch
> In the abstract class {{Storage}}, {{storageDirs}} is a protected variable and all its
derived classes like {{NNStorage}}, {{JNStorage}}, {{DataStorage}}.. are iterating over this
non-thread safe variable without any proper locks. Any parallel modification operation like
add or remove volume can mutate the backing storageDirs list and any iterators on this list
around the  same time can face {{ConcurrentModificationException}}. It would be good to make
the variable private and restrict the access via getters and setters. Any thread safe restriction
need to be done can then be placed on the parent class only.
> Also, {{NNStorage}} redefines parent class Storage's {{storageDirs}}, making it inconsistent
with other derived classes. Would be cleaner if {{NNStorage}} can avoid re-defining it locally.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message