hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Kelly (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HDFS-1557) Separate Storage from FSImage
Date Thu, 27 Jan 2011 17:40:45 GMT

     [ https://issues.apache.org/jira/browse/HDFS-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ivan Kelly updated HDFS-1557:

    Status: Patch Available  (was: Open)

Addressed Suresh's comments.
NNStorage use synchronized method errorDirectory() to notify listeners of error. The listener
implement synchrnonized method to handle the error. Is it possible for listeners (say FSImage)
to be calling from its synchronized section, a synchronized method on NNStorage? This could
cause dead locks.
NNStorage now using CopyOnWriteArrayLists now, so errorDirectory is no longer synchronised.
(see below)

format(), registerListener() should this be synchronized as it manipulates listeners?
listeners is now a CopyOnWriteArrayList.

Storage#storageDirs are manipulated in NNStorage and Storage. The way it is done is not thread
safe. Perhaps the existing code is thread safe it self. This could be addressed in a separate
Well spotted. I dont think any of this is threadsafe, given that storageDirs is modified in
numerous places, and is being constantly being iterated over which could trigger a concurrent
modification exception.

I've made storageDirs and removedStorageDirs a CopyOnWriteArrayList now.

Consider making the following method package private: isPreUpgradableLayout(), setRestoreFailedStorage()
(both variants), attemptResotreRemovedStorage()... The are other methods that could only be
used with in the package. This makes sure this is not a class for outside consumption. I would
further consider making NNStorage non public class.
Tightened up all the access privileges which I could on that class now to package private.
Unfortunately, NNStorage itself must remain public because of UpgradeUtilities in testing
being in a different package.

> Separate Storage from FSImage
> -----------------------------
>                 Key: HDFS-1557
>                 URL: https://issues.apache.org/jira/browse/HDFS-1557
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: name-node
>    Affects Versions: 0.21.0
>            Reporter: Ivan Kelly
>            Assignee: Ivan Kelly
>             Fix For: 0.23.0
>         Attachments: HDFS-1557-branch-0.22.diff, HDFS-1557-branch-0.22.diff, HDFS-1557-trunk.diff,
HDFS-1557-trunk.diff, HDFS-1557-trunk.diff, HDFS-1557.diff, HDFS-1557.diff, HDFS-1557.diff,
HDFS-1557.diff, HDFS-1557.diff
> FSImage currently derives from Storage and FSEditLog has to call methods directly on
FSImage to access the filesystem. This JIRA is to separate the Storage class out into NNStorage
so that FSEditLog is less dependent on FSImage. From this point, the other parts of the circular
dependency should be easy to fix.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message