hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7178) copyToLocal API is creating .crc files in local, even after setting verifyChecksum to false at client side.
Date Thu, 24 Mar 2011 06:10:06 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13010569#comment-13010569

Uma Maheswara Rao G commented on HADOOP-7178:

Hi Daryn Sharp,
Thanks for your comments,

Should the code really be casting to a ChecksumFileSystem w/o first checking instanceof?
  For CopyToLocalFile api, we will select LocalFilesystem as local file system. Here gteLocal()
api used to get that local file system.This API  will always return LocalFileSystem which
extends CheckSumFileFystem
  So, instanceOf check not required.

Should the default getVerifyChecksum() return instanceof ChecksumFileSystem instead of always
true? That may help prevent changing other filesystem classes which may do not implement setVerifyChecksum.
  Here By deafult local file system will be LocalFileSystem. This will create crc files. So,
I put default as true. If user wanted to disable it, he will provide the implementation for
setVerifyChecksum() and getVerifyChecksum() like DistributedFileSystem. 

It looks like more fs classes may need to be changed than just Distributed. Other filesystem
appear to have private booleans for verifyChecksum. 
  Here we are not forcing implementation classes to implement this methods. If Concrete classes
does not want to manage this checkSumVerification flag then default implemenation will work.

> copyToLocal API is creating .crc files in local, even after setting verifyChecksum to
false at client side.
> -----------------------------------------------------------------------------------------------------------
>                 Key: HADOOP-7178
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7178
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: HADOOP-7178_COMMON.patch, HADOOP-7178_HDFS.patch
> When we copy the files from DFS to local, it is creating the .crc file in local filesystem
for the verification of checksum even if we disable the checksum verification at client side.
>         When user does not want to do any checksum verification, then what will be the
use in creating of these files in local file system.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message