hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuzhang Han <yuzhanghan1...@gmail.com>
Subject Different container sizes for MR tasks, job does not complete
Date Fri, 14 Jun 2013 20:05:07 GMT
Hi,

I am using MapReduce on YARN. I want to make tasks of the same job run 
in containers with different sizes. For example: Job1 = <Task1, ..., 
Task8>. Task1 := 1280 MB; Task2 to 8  := 1024 MB.
To achieve this, I manually call 
reqEvent.getCapability().setMemory(MEMORY_SIZE) in 
RMContainerAllocator.java with MEMORY_SIZE = 1280 or 1024. The problem 
is, if one task has different memory than the others, then the whole job 
stops at 80% forever. When all the tasks have the same memory, the job 
completes correctly and fast. Does some one has any idea?

Thanks a lot.

Yuzhang

Mime
View raw message