Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 24 May 2017 17:43:04 +0000 (UTC)
From: "Botong Huang (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13074582.1495647681000.285667.1495647784231@Atlassian.JIRA>
In-Reply-To: <JIRA.13074582.1495647681000@Atlassian.JIRA>
References: <JIRA.13074582.1495647681000@Atlassian.JIRA> <JIRA.13074582.1495647681739@jira-lw-us.apache.org>
Subject: [jira] [Updated] (YARN-6640)  AM heartbeat stuck when responseId
 overflows MAX_INT
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Wed, 24 May 2017 17:43:10 -0000


     [ https://issues.apache.org/jira/browse/YARN-6640?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:all-tabpanel ]

Botong Huang updated YARN-6640:
-------------------------------
    Description:=20
The current code in {{ApplicationMasterService}}:=20
if ((request.getResponseId() + 1) =3D=3D lastResponse.getResponseId()) {/* =
old heartbeat */  return lastResponse;}
else if (request.getResponseId() + 1 < lastResponse.getResponseId()) { thro=
w ... }
process the heartbeat...


When a heartbeat comes in, in usual case we are expecting request.getRespon=
seId() =3D=3D lastResponse.getResponseId(). The =E2=80=9Cif=E2=80=9C is for=
 the duplicate heartbeat that=E2=80=99s one step old, the =E2=80=9Celse if=
=E2=80=9D is to throw and complain for heartbeats more than two steps old, =
otherwise we accept the new heartbeat and process it.

So the bug is: when lastResponse.getResponseId() =3D=3D MAX_INT, the newest=
 heartbeat comes in with responseId =3D=3D MAX_INT. However reponseId + 1 w=
ill be MIN_INT, and we will fall into the =E2=80=9Celse if=E2=80=9D case an=
d RM will throw. Then we are stuck here=E2=80=A6

  was:
The current code in {{ApplicationMasterService}}:=20
if ((request.getResponseId() + 1) =3D=3D lastResponse.getResponseId()) {/* =
old heartbeat */  return lastResponse;}=20
else if (request.getResponseId() + 1 < lastResponse.getResponseId()) { thro=
w ... }
process the heartbeat...


When a heartbeat comes in, in usual case we are expecting request.getRespon=
seId() =3D=3D lastResponse.getResponseId(). The =E2=80=9Cif=E2=80=9C is for=
 the duplicate heartbeat that=E2=80=99s one step old, the =E2=80=9Celse if=
=E2=80=9D is to throw and complain for heartbeats more than two steps old, =
otherwise we accept the new heartbeat and process it.

So the bug is: when lastResponse.getResponseId() =3D=3D MAX_INT, the newest=
 heartbeat comes in with responseId =3D=3D MAX_INT. However reponseId + 1 w=
ill be MIN_INT, and we will fall into the =E2=80=9Celse if=E2=80=9D case an=
d RM will throw. Then we are stuck here=E2=80=A6


>  AM heartbeat stuck when responseId overflows MAX_INT
> -----------------------------------------------------
>
>                 Key: YARN-6640
>                 URL: https://issues.apache.org/jira/browse/YARN-6640
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Botong Huang
>            Assignee: Botong Huang
>            Priority: Minor
>
> The current code in {{ApplicationMasterService}}:=20
> if ((request.getResponseId() + 1) =3D=3D lastResponse.getResponseId()) {/=
* old heartbeat */  return lastResponse;}
> else if (request.getResponseId() + 1 < lastResponse.getResponseId()) { th=
row ... }
> process the heartbeat...
> When a heartbeat comes in, in usual case we are expecting request.getResp=
onseId() =3D=3D lastResponse.getResponseId(). The =E2=80=9Cif=E2=80=9C is f=
or the duplicate heartbeat that=E2=80=99s one step old, the =E2=80=9Celse i=
f=E2=80=9D is to throw and complain for heartbeats more than two steps old,=
 otherwise we accept the new heartbeat and process it.
> So the bug is: when lastResponse.getResponseId() =3D=3D MAX_INT, the newe=
st heartbeat comes in with responseId =3D=3D MAX_INT. However reponseId + 1=
 will be MIN_INT, and we will fall into the =E2=80=9Celse if=E2=80=9D case =
and RM will throw. Then we are stuck here=E2=80=A6


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org