hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Mildebrandt <ch...@woodenrhino.com>
Subject When are the checksums for local files generated?
Date Thu, 20 Mar 2014 18:49:42 GMT

For the intermediate files the mapper generates (which will get sent to the
reducers), when are the checksums generated? Are they generated when a
request comes in for transfer, or are they generated when the file is
initially created? Also, are checksums checked/verified if the file is
accessed again by a local reducer (doesn't have to go over the network)?


View raw message