flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1266) Generalize Flink's DistributedFileSystemClass to a Hadoop FileSystem wrapper
Date Mon, 15 Dec 2014 18:09:13 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14246928#comment-14246928
] 

ASF GitHub Bot commented on FLINK-1266:
---------------------------------------

GitHub user rmetzger opened a pull request:

    https://github.com/apache/incubator-flink/pull/268

    [FLINK-1266] Generalize DistributedFileSystem implementation

    to HadoopFileSystem wrapper, which supports all subclasses of org.apache.hadoop.fs.FileSystem.
    This allows us to let users use all file systems with support for HDFS.
    The change has been tested with Tachyon, Google Cloud Storage Hadoop Adapter and HDFS.
    
    The change also cleans up the Hadoop dependency exclusions.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/rmetzger/incubator-flink flink1266

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-flink/pull/268.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #268
    
----
commit 5c6c0d93dcb4030eb599f79927da9ace53292bb1
Author: Robert Metzger <rmetzger@apache.org>
Date:   2014-12-12T15:45:02Z

    [FLINK-1266] Generalize DistributedFileSystem implementation
    to HadoopFileSystem wrapper, which supports all subclasses of org.apache.hadoop.fs.FileSystem.
    This allows us to let users use all file systems with support for HDFS.
    The change has been tested with Tachyon, Google Cloud Storage Hadoop Adapter and HDFS.
    
    The change also cleans up the Hadoop dependency exclusions.

----


> Generalize Flink's DistributedFileSystemClass to a Hadoop FileSystem wrapper
> ----------------------------------------------------------------------------
>
>                 Key: FLINK-1266
>                 URL: https://issues.apache.org/jira/browse/FLINK-1266
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Robert Metzger
>            Assignee: Robert Metzger
>            Priority: Minor
>
> Currently, the Hadoop's DistributedFileSystem class is somewhat hardwired into the Flink
DistributedFileSystem class.
> That is actually not necessary because Flink's DistributedFileSystem class is only assuming
Hadoop's FileSystem class.
> By generalizing the Flink DFS class, we can easily support other file systems implementing
the Hadoop FS class, such as the GoogleHadoopFileSystem, Tachyon and others.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message