hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-12614) Add a generic .isOffline() method to filesystems to probe availability
Date Fri, 04 Dec 2015 15:34:11 GMT
Steve Loughran created HADOOP-12614:

             Summary: Add a generic .isOffline() method to filesystems to probe availability
                 Key: HADOOP-12614
                 URL: https://issues.apache.org/jira/browse/HADOOP-12614
             Project: Hadoop Common
          Issue Type: Improvement
          Components: fs
    Affects Versions: 2.7.1
            Reporter: Steve Loughran
            Priority: Minor

Looking at some of the spark `HistoryServer` code, they do reflection games to check whether
HDFS is in safe mode or not, games which vary with version and could be at risk of failing
with the client/server split (fortunately, it's all client-side). Nor do the checks apply
to other filesystems, which could have their own online/offline state.

I propose adding the new methods {{FileSystem.isOffline()}}, {{FileContext.isOffline()}},
to return true if an FS knows that it is offline. For HDFS: Safe mode. For other filesystems?
Maybe network state, disk being r/W, etc. Their choice. The default would be false: an FS
is not offline,

obvously, {{!isOffline()}} doesn't guarantee the FS is fully functional; that's why I propose
{{isOffline()}}; less dangerous than the opposite values of {{isLive()}} or {{isAvailable()}},
which may be making promises which cannot hold 

This message was sent by Atlassian JIRA

View raw message