hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9209) Add shell command to dump file checksums
Date Wed, 16 Jan 2013 17:08:13 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555214#comment-13555214
] 

Kihwal Lee commented on HADOOP-9209:
------------------------------------

bq. Does that mesh with your understanding?

Yes.

The block size is a factor in determining crcPerBlock, which is a part of the algorithm name.
But, when the file size is less than the block size, crcPerBlock will be 0 (as in the test
cases in the patch). The only case it might confuse users is when two identical such files
with different preferred block sizes. If the files get appended, as soon as the file gets
bigger than the block size of one, the two checksums and algorithm name will look different.


                
> Add shell command to dump file checksums
> ----------------------------------------
>
>                 Key: HADOOP-9209
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9209
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs, tools
>    Affects Versions: 3.0.0, 2.0.3-alpha
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-9209.txt, hadoop-9209.txt
>
>
> Occasionally while working with tools like distcp, or debugging certain issues, it's
useful to be able to quickly see the checksum of a file. We currently have the APIs to efficiently
calculate a checksum, but we don't expose it to users. This JIRA is to add a "fs -checksum"
command which dumps the checksum information for the specified file(s).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message