hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sugandha Naolekar <sugandha....@gmail.com>
Subject Can the file storage in HDFS be customized?
Date Tue, 25 Feb 2014 10:29:33 GMT
Hello,

I have a huge shapefile which has some 500 polygon  geometries. Is there a
way to store this shapefile in such a format in HDFS that each block will
have 100 polygon geometries. And each block representing a quad core
machine.

Thus, 5 machines, with 5 blocks, which have in total 500 polygon
geometries.

Internally, I would like to read each of the block of HDFS in such a way
where, each polygon geometry is fed to the map() task. THus, 100 map()
tasks per block per machine.

--
Thanks & Regards,
Sugandha Naolekar

Mime
View raw message