hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3003) FileSystem cache key should be updated after a FileSystem object is created
Date Tue, 18 Mar 2008 21:55:24 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12580117#action_12580117
] 

Doug Cutting commented on HADOOP-3003:
--------------------------------------

+1

A thought: the filesystem cache keys are now <user,scheme,authority>.  Since URIs have
a user field, it is tempting to use a  URI for the key, instead of defining a new class: scheme://user@host:port/.
 The only problem would be if the FileSystem URI's authority includes a user, especially if
it is a different user.  For example, the S3 FileSystem puts a user in the authority, but
that should be consistent with the S3 login.  Unfortunately, its probably not consistent with
the way we're determining the username used by the cache, using UnixUserGroupInformation.login().
 The bug is that login() should be per-filesystem, not static.


> FileSystem cache key should be updated after a FileSystem object is created
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-3003
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3003
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.16.2, 0.17.0
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>             Fix For: 0.16.2, 0.17.0
>
>         Attachments: 3003_20080311.patch, 3003_20080313.patch, 3003_20080318.patch
>
>
> In FileSystem.get(uri, conf), it first creates a cache key from the uri and the conf
and then lookups the corresponding FileSystem object in the cache.  If the object is not found,
it initializes a FileSystem object and put it to the cache with the key.  However, during
FileSystem creation, the conf might be modified.  In such case, the key should be updated
before putting it to the cache.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message