hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thamizh <tceg...@yahoo.co.in>
Subject Reg HDFS checksum
Date Fri, 08 Apr 2011 12:17:46 GMT
Hi All,

This is question regarding "HDFS checksum" computation.

I understood that When we read a file from HDFS by default it verifies the checksum and your
read would not succeed if the file is corrupted. Also CRC is internal to hadoop.

Here are my questions:
1. How can I use "hadoop dfs -get [-ignoreCrc] [-crc] <src> <localdst>" command?

2. I used "get" command for a .gz file with -crc option ( "hadoop dfs -get -crc input1/test.gz
/home/hadoop/test/. " ).
    Does this check for .crc file created in hadoop? When I tried this, I got an error
"-crc option is not valid when source file system does not have crc files. Automatically turn
the option off." means that hadoop does not create crc for this file?
Is this correct?

3. How can I enable hadoop to create CRC file?



  Thamizhannal P
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message