hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hitesh Shah <hit...@hortonworks.com>
Subject Re: getAllocatedContainers() is not returning when ran against 2.0.3-alpha
Date Wed, 03 Apr 2013 15:50:28 GMT
If I understand your question, you are expecting all the containers to be allocated in one
go? Or are you seeing your application hang because it asked for 10 containers but it only
received a total of 9 even after repeated calls to the RM?  

There is no guarantee that you will be allocated all your requested containers in a single
allocate() call. The application is expected to keep on making allocate calls to the RM at
regular intervals and the RM will give it any newly allocated containers in the period  since
the last allocate call.

The AMRMClient is  a library that should make it easier for someone to write an application
master. Yes - it is the recommended approach. Please file jiras if you see use-cases that
are not addressed by it when implementing your AM.

-- Hitesh

On Apr 3, 2013, at 8:05 AM, Krishna Kishore Bonagiri wrote:

> Hi,
>  I am running an application that had been written and working well with the hadoop-2.0.0-alpha
but when I am running the same against 2.0.3-alpha, the getAllocatedContainers() method called
on AMResponse is not returning all the containers allocated sometimes. For example, I request
for 10 containers and this method gives me only 9 containers sometimes, and when I looked
at the log of Resource Manager, the 10th container is also allocated. It happens only sometimes
randomly and works fine all other times. If I send one more request for the remaining container
to RM, it could allocate that container. I am running only one application at a time, but
1000s of them one after another.
>   Is this problem seen by anyone else? I have seen that there are significant number
of changes in the way ApplicationMaster.java is written now for the Distributed Shell example.
There is now AMRMClient instead of AMRMProtocol. There is method addContainerRequest() which
is not there earlier, and there is ConatainerRequest class now which was ResourceRequest earlier,
etc.  Is it adivsable to re-write my application in a similar way?
> Thanks,
> Kishore

View raw message