hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Golby <sgo...@conductor.com>
Subject Something like md5sum of files on HDFS
Date Fri, 19 Nov 2010 17:35:12 GMT
Hi Everyone,

What options do I have to do something like a md5sum checksum on files/directories living
on HDFS ?

We are making backups of files on hdfs and I'd like to do something similar to the command
below beforehand (so when I restore I can check the files are good).

I'd prefer not to -CopyToLocal every file before deleting them. :-)

# md5sum /usr/share/doc/wireless-tools/*
34982c2c6fd5508c99a550787197fbc8  /usr/share/doc/wireless-tools/changelog.Debian.gz
5901cff15bffb756035102da5f938a4d  /usr/share/doc/wireless-tools/changelog.gz
3e04edc4db9ebc119b03bf4d7f234e31  /usr/share/doc/wireless-tools/copyright
c13dcf5d4f0c1f6bc4343ec6be9d2c11  /usr/share/doc/wireless-tools/DISTRIBUTIONS.txt.gz
930d4fc01b4c5303fcc9f56f05457102  /usr/share/doc/wireless-tools/HOTPLUG-UDEV.txt.gz
0bf104866a56435626d0144d66c6d4d7  /usr/share/doc/wireless-tools/PCMCIA.txt.gz
62dd6ac4c427336d5a93a6ef77ca0d3e  /usr/share/doc/wireless-tools/README.Debian
97775c5bb982d2924051eb5777fc69c0  /usr/share/doc/wireless-tools/README.gz

Does anyone have a simple solution which a linux admin could deploy ?  ie) doesn't require
too much Java code to get the job done.


View raw message