hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12614) Add a generic .isOffline() method to filesystems to probe availability
Date Tue, 15 Dec 2015 18:17:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15058470#comment-15058470

Colin Patrick McCabe commented on HADOOP-12614:

I was just going to make that very comment.  It seems more reasonable to catch {{SafeModeException}}
(which is a very old exception, so they could probably do that and still continue supporting
Hadoop 1.x).

> Add a generic .isOffline() method to filesystems to probe availability
> ----------------------------------------------------------------------
>                 Key: HADOOP-12614
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12614
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>    Affects Versions: 2.7.1
>            Reporter: Steve Loughran
>            Priority: Minor
> Looking at some of the spark `HistoryServer` code, they do reflection games to check
whether HDFS is in safe mode or not, games which vary with version and could be at risk of
failing with the client/server split (fortunately, it's all client-side). Nor do the checks
apply to other filesystems, which could have their own online/offline state.
> I propose adding the new methods {{FileSystem.isOffline()}}, {{FileContext.isOffline()}},
to return true if an FS knows that it is offline. For HDFS: Safe mode. For other filesystems?
Maybe network state, disk being r/W, etc. Their choice. The default would be false: an FS
is not offline,
> obvously, {{!isOffline()}} doesn't guarantee the FS is fully functional; that's why I
propose {{isOffline()}}; less dangerous than the opposite values of {{isLive()}} or {{isAvailable()}},
which may be making promises which cannot hold 

This message was sent by Atlassian JIRA

View raw message