hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7601) Move common fs implementations to a hadoop-fs module
Date Thu, 01 Sep 2011 17:10:09 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13095432#comment-13095432
] 

Luke Lu commented on HADOOP-7601:
---------------------------------

bq. This is not true, it loads as many file systems classes as defined in the service file.

That's good to know, thanks :) But the real problem is that in order to query the schemes
you have to instantiate every file system object which can be very expensive (esp. all the
resulting loading of dependent classes) and unnecessary (people typically only use one dfs
implementation) and mandate a side effect free default constructor for every FileSystem implementation.
Loading a single provider (per jar) then querying for support and using a factory is much
more efficient and flexible. ServiceLoader is a primitive load time only DI solution with
limited functionality. That's probably the reason that I never use non-provider bindings in
the service files :)

> Move common fs implementations to a hadoop-fs module
> ----------------------------------------------------
>
>                 Key: HADOOP-7601
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7601
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs
>            Reporter: Luke Lu
>             Fix For: 0.23.0
>
>
> Much of the hadoop-common dependencies is from the fs implementations. We have more fs
implementations on the way (ceph, lafs etc). I propose that we move all the fs implementations
to a hadoop-fs module under hadoop-common-project.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message