Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
From: Benjamin Kim <bbuild11@gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Subject: Completed Tasks in YARN will not release resources
Message-Id: <51BDA3B8-3B71-4D1C-853A-2ED41626F157@gmail.com>
Date: Sat, 30 Apr 2016 05:56:32 -0700
To: user@hadoop.apache.org
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))

Has anyone encountered this problem with YARN. It all started after an =
attempt to upgrade from CDH 5.4.8 to CDH 5.5.2.

I ran jobs overnight, and they never completed. But, it did take down =
the YARN ResourceManager and multiple NodeManagers after 5 or 6 hours. =
There was one job that out of 450 mappers, only 64 completed, 386 =
pending, and 0 running. The pending mappers are in a Scheduled state.

Each data node has 24 cores, 64 GB, 6 drives x 2 TB. NodeManager is =
allocated 45GB; Mappers are allocated 4GB (3.2GB Heap); Reducers are =
allocated 8GB (6.4GB Heap); AM is allocated 8GB (6.4 GB Heap).

There was a note for each completed task attempt stating that it was in =
the finished state for too long. I believe that the task is not =
releasing the container when it=E2=80=99s done or is not communicating =
it back.

Does anyone have any ideas?

Thanks,
Ben


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org