hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mirko Kämpf <mirko.kae...@gmail.com>
Subject Re: partition file by content based through HDFS
Date Sun, 11 May 2014 20:54:11 GMT

HDFS blocks are not "content aware". Such a separation like you requested,
could be done via Hive or Pig with some lines of code, than you would have
multiple files which can be organized in partitions as well, but such
partitions are on a different abstraction level, not on blocks, but within
hive tables.

Best wishes,

2014-05-11 14:41 GMT+01:00 Karim Awara <karim.awara@kaust.edu.sa>:

> Hi,
> When a user is uploading a file from the local disk to its HDFS, can I
> make it partition the file into blocks based on its content?  Meaning, if I
> have a file with one integer column, can i say, I want the hdfs block to
> have even numbers?
> --
> Best Regards,
> Karim Ahmed Awara
> ------------------------------
> This message and its contents, including attachments are intended solely
> for the original recipient. If you are not the intended recipient or have
> received this message in error, please notify me immediately and delete
> this message from your computer system. Any unauthorized use or
> distribution is prohibited. Please consider the environment before printing
> this email.

View raw message