accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roshan Punnoose <rosh...@gmail.com>
Subject Accumulo Input Format over hadoop blocks
Date Mon, 09 Jul 2012 18:24:08 GMT
This might be a very easy question, but I was wondering how the Accumulo
Input Format handled a tablet file splitting over multiple nodes.

For example, if I have a tablet file that is 1GB large, where my hadoop
block size is 256MB. Then there is a possibility that up to 4 nodes could
be holding the data from my tablet file. However, when Accumulo Input
Format creates mappers, it creates a mapper for every tablet. This might
mean that 3 blocks are transferred over the network to where the mapper is
running to ensure data locality.

Am I correct in this assumption? Or is there something else the
TabletServer is doing underneath to make sure all the data actually resides
in one server, so there is no network overhead of moving blocks before a
Map Reduce job.

Thanks!
Roshan

Mime
View raw message