hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Avoid Killing of YARN container on excess virtual memory usage
Date Thu, 18 Oct 2012 14:46:30 GMT
I filed https://issues.apache.org/jira/browse/YARN-168.

On Thu, Oct 18, 2012 at 5:07 PM, Harsh J <harsh@cloudera.com> wrote:
> This is possible to do, but you've hit a bug with the current YARN
> implementation. Ideally you should be able to configure the vmem-pmem
> ratio (or an equivalent config) to be -1, to indicate disabling of
> virtual memory checks completely (and there's indeed checks for this),
> but it seems like we are enforcing the ratio to be at least 1.0 (and
> hence negatives are disallowed).
> You can't workaround by setting the NM's offered resource.mb to -1
> either, as you'll lose out on controlling maximum allocations.
> Please file a YARN bug on JIRA. The code at fault lies under
> ContainersMonitorImpl#init(…).
> On Thu, Oct 18, 2012 at 4:00 PM, Krishna Kishore Bonagiri
> <write2kishore@gmail.com> wrote:
>> Hi,
>>   Is there a way we can ask the YARN RM for not killing a container when it
>> uses excess virtual memory than the maximum it can use as per the
>> specification in the configuration file yarn-site.xml? We can't always
>> estimate the amount of virtual memory needed for our application running on
>> a container, but we don't want to get it killed in a case it exceeds the
>> maximum limit.
>>   Please suggest as to how can we come across this issue.
>> Thanks,
>> Kishore
> --
> Harsh J

Harsh J

View raw message