hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tharindu Mathew <mcclou...@gmail.com>
Subject Data locality for a custom input format
Date Sat, 12 Nov 2011 13:42:48 GMT
Hi hadoop devs,

I'm implementing a custom input format and want to understand how to make
use of data locality.

AFAIU, only file input format makes use of data locality since the job
tracker picks data locality based on the block location defined in the file
input split.

So, the job tracker code is partly responsible for this. So providing data
locality for a custom input format would be to either either extend file
input format or modify job tracker code (if that makes sense even).

Is my understanding correct?



blog: http://mackiemathew.com/

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message