hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andre Kelpe <ake...@concurrentinc.com>
Subject Re: Can the file storage in HDFS be customized?
Date Tue, 25 Feb 2014 12:31:00 GMT
This might get you further: https://github.com/mraad/Shapefile

- André


On Tue, Feb 25, 2014 at 11:29 AM, Sugandha Naolekar
<sugandha.n87@gmail.com>wrote:

> Hello,
>
> I have a huge shapefile which has some 500 polygon  geometries. Is there a
> way to store this shapefile in such a format in HDFS that each block will
> have 100 polygon geometries. And each block representing a quad core
> machine.
>
> Thus, 5 machines, with 5 blocks, which have in total 500 polygon
> geometries.
>
> Internally, I would like to read each of the block of HDFS in such a way
> where, each polygon geometry is fed to the map() task. THus, 100 map()
> tasks per block per machine.
>
> --
> Thanks & Regards,
> Sugandha Naolekar
>
>
>
>


-- 
André Kelpe
andre@concurrentinc.com
http://concurrentinc.com

Mime
View raw message