hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinayakumar B <vinayakum...@apache.org>
Subject Re: Block creation in HDFS
Date Tue, 17 Feb 2015 10:25:40 GMT
Hi abhishek,
Is Your partitions of same sizes? If yes, then you can set that as block
size.

If not you can use the latest feature.. variable block size.
To verify your use case.
You can close the current block after each partition data is written and
append to new block for new partition data.
This feature is not yet available in any of the release. Hope to see in
future 2.7 release. As of now you can verify in any of the trunk/branch-2
builds.

Hope this helps.

-Vinay
On Feb 17, 2015 8:30 AM, "Abhishek Das" <abhishek.besu@gmail.com> wrote:

> Hi,
>
> I am new in this group. I had a question regarding block creation in HDFS.
> By default the file is split into multiple blocks of size equal to block
> size. I need to introduce new block creation policy into the system. In my
> case the main file is divided into multiple partitions. My goal is to
> create the blocks where data is represented from each partition of the
> file. Is it possible to introduce the new policy ? If yes, what would the
> starting point in the code I should look at.
>
> Regards,
> Abhishek Das
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message