hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shashi Vishwakarma <shashi.vish...@gmail.com>
Subject Comparing CheckSum of Local and HDFS File
Date Fri, 07 Aug 2015 10:28:56 GMT
Hi

I have a small confusion regarding checksum verification.Lets say , i have
a file abc.txt and I transferred this file to hdfs. How do I ensure about
data integrity?

I followed below steps to check that file is correctly transferred.

*On Local File System:*

md5sum abc.txt

276fb620d097728ba1983928935d6121  TestFile

*On Hadoop Cluster :*

 hadoop fs -checksum /abc.txt

/abc.txt      MD5-of-0MD5-of-512CRC32C
 000002000000000000000000911156a9cf0d906c56db7c8141320df0

Both output looks different to me. Let me know if I am doing anything wrong.

How do I verify if my file is transferred properly into HDFS?

Thanks
Shashi

Mime
View raw message