hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8319) FileContext does not support setWriteChecksum
Date Thu, 03 May 2012 16:32:52 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267565#comment-13267565
] 

Daryn Sharp commented on HADOOP-8319:
-------------------------------------

First, a general question: why change the visible of {{LocalFs}} ctor to public?

This patch doesn't work when a path symlinks off to a different afs.  {{fc.setWriteChecksum(boolean,
Path)}} will instantiate the resolved afs, set the flag, the afs instance goes "poof", and
the subsequent create will get another instance of the afs with the default value.  One solution
might be to cache all the afs used by the context so the value persists.

However, I'm not comfortable with the misleading semantics of {{fc.setWriteChecksum(boolean,
Path)}}.  The path argument implies the flag applies only to the given path, but in this patch
it applies to the entire afs of the given path.  I'd be more comfortable with mimicing fs
and just have {{fc.setWriteChecksum(boolean)}} set an instance var in the fc. {{fc.create}}
calls the resolved {{afs.setWriteChecksum}}, and then {{afs.create}}.

What I think ultimately needs to be done is for fs/afs {{create}} to accept a flag for whether
checksums are written.  This provides more granularity, avoids threading issues where one
thread changes the value for all threads, prevents viewfs from applying the setting to all
its mounts, and would solve the unexpected behavior in {{FileSystem}} where the cache causes
the value to change in every instance/reference to the fs.
                
> FileContext does not support setWriteChecksum
> ---------------------------------------------
>
>                 Key: HADOOP-8319
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8319
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 3.0.0
>            Reporter: John George
>            Assignee: John George
>         Attachments: HADOOP-8319.patch, HADOOP-8319.patch, HADOOP-8319.patch, HADOOP-8319.patch
>
>
> File Context does not support setWriteChecksum and hence users trying
> to use this functionality fails.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message