hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Question about FileInputFormat splits
Date Fri, 04 Jan 2013 23:45:34 GMT
Hello,

> than a block. But if you have more nodes in your cluster than data blocks,
> you may get faster execution if you allow splits smaller than a block. Is

You're right. So, we're working on partitioning issues now.

> you may get faster execution if you allow splits smaller than a block. Is
> there any way to use splits smaller than a block in Hama 0.6.0?

Yes. But, Hama 0.6.1 version will support it.

On Sat, Jan 5, 2013 at 4:59 AM, Leonidas Fegaras <fegaras@cse.uta.edu> wrote:
> Dear Hama developers,
> It seems that the splits generated by the FileInputFormat in Hama 0.6.0
> cannot be smaller than a block. In Hama 0.5.0, I could set any split size
> using  job.set("bsp.min.split.size",...) and set the task numbers using
> job.setNumBspTask(...). This is ignored by Hama 0.6.0 for a split smaller
> than a block. But if you have more nodes in your cluster than data blocks,
> you may get faster execution if you allow splits smaller than a block. Is
> there any way to use splits smaller than a block in Hama 0.6.0?
> Thanks for your help,
> Leonidas
>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message