hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8490) Add Configuration to FileSystem cache key
Date Thu, 07 Jun 2012 16:00:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291087#comment-13291087
] 

Kihwal Lee commented on HADOOP-8490:
------------------------------------

Making conf as part of the  cache key solves number of problems, but may not be ideal by itself.


We need to think about what are the parts of conf that make sense to be dynamic. Some might
need to be allowed to change run-time, others need to stay the same once an input or output
stream is created. Creating unique instances for all these variations may not be ideal. Probably
FileSystem and the gut should only look at small static part of conf and allow each usage
to tweak the behavior in a way that does not interfere with others that share objects/resources.
 

I believe FileSystem cache was originally intended to work this way.  The problem is, it currently
doesn't. We need a better way to maintain this separation.







                
> Add Configuration to FileSystem cache key
> -----------------------------------------
>
>                 Key: HADOOP-8490
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8490
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.23.0, 0.24.0, 2.0.0-alpha
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>
> The {{FileSystem#get(URI, Configuration}} does not take the given {{Configuration}} into
consideration before returning an existing fs instance from the cache with a possibly different
conf.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message