Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Thu, 2 Mar 2017 23:41:45 +0000 (UTC)
From: "Siddharth Seth (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.13047834.1488494946000.48836.1488498105569@Atlassian.JIRA>
In-Reply-To: <JIRA.13047834.1488494946000@Atlassian.JIRA>
References: <JIRA.13047834.1488494946000@Atlassian.JIRA> <JIRA.13047834.1488494946779@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HIVE-16094) queued containers may timeout if they
 don't get to run for a long time
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 02 Mar 2017 23:41:55 -0000


     [ https://issues.apache.org/jira/browse/HIVE-16094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Siddharth Seth updated HIVE-16094:
----------------------------------
    Attachment: HIVE-16094.01.patch

The problem was that if an am was picked up by the queueDrainer when it had 0 fragments, it would not be put back. registerFragment would only add a new entry to the queue if the am was not known.

AMNodeInfo instances were originally meant to be used across multiple queries belonging to an AM. We could still achieve that by going back to the old model of reference counting.

However, I think it's cleaner to maintain an AMNodeInfo instance per query instance. So - the patch changes the key to be the queryIdentifier. An instance of amNodeInfo is always maintained in the queue. A heartbeat is only sent if there are pending fragments. It is removed from the queue after query completion, or if an error is hit.

cc [~prasanth_j] for review.

> queued containers may timeout if they don't get to run for a long time
> ----------------------------------------------------------------------
>
>                 Key: HIVE-16094
>                 URL: https://issues.apache.org/jira/browse/HIVE-16094
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 2.2.0
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>            Priority: Critical
>         Attachments: HIVE-16094.01.patch
>
>
> I believe this happened after HIVE-15958 - since we end up keeping amNodeInfo in knownAppMaters, and that can result in the callable not being scheduled on new task registration.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)