Mailing-List: contact dev-help@pig.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@pig.apache.org
Date: Mon, 5 Jun 2017 17:40:04 +0000 (UTC)
From: "Rohini Palaniswamy (JIRA)" <jira@apache.org>
To: pig-dev@hadoop.apache.org
Message-ID: <JIRA.12904938.1444844151000.366953.1496684404315@Atlassian.JIRA>
In-Reply-To: <JIRA.12904938.1444844151000@Atlassian.JIRA>
References: <JIRA.12904938.1444844151000@Atlassian.JIRA> <JIRA.12904938.1444844151339@jira-lw-us.apache.org>
Subject: [jira] [Commented] (PIG-4700) Enable progress reporting for Tasks
 in Tez
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 05 Jun 2017 17:40:08 -0000


    [ https://issues.apache.org/jira/browse/PIG-4700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037262#comment-16037262 ] 

Rohini Palaniswamy commented on PIG-4700:
-----------------------------------------

bq. Could you make it 1000? i.e schedule it every second
  Actually it can be optimized better. Progress information is sent as part of task heartbeat. In setups like ours heartbeat interval is set to 3 seconds so that the AM can handle 10s of thousands of tasks. So we should set it to a value slightly lower than heartbeat interval.

{code}
progressHelper.scheduleProgressTaskService(100, Math.max(1000, (conf.getInt(TezConfiguration.TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS, TezConfiguration.TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS_DEFAULT) - 50)));
{code}

> Enable progress reporting for Tasks in Tez
> ------------------------------------------
>
>                 Key: PIG-4700
>                 URL: https://issues.apache.org/jira/browse/PIG-4700
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Rohini Palaniswamy
>            Assignee: Satish Subhashrao Saley
>             Fix For: 0.18.0
>
>         Attachments: PIG-4700-1.patch, PIG-4700-2.patch, PIG-4700-3.patch, PIG-4700-4.patch, PIG-4700-5.patch
>
>


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)