Return-Path: X-Original-To: apmail-pig-dev-archive@www.apache.org Delivered-To: apmail-pig-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 328D517E86 for ; Tue, 28 Oct 2014 23:13:34 +0000 (UTC) Received: (qmail 62115 invoked by uid 500); 28 Oct 2014 23:13:34 -0000 Delivered-To: apmail-pig-dev-archive@pig.apache.org Received: (qmail 62060 invoked by uid 500); 28 Oct 2014 23:13:34 -0000 Mailing-List: contact dev-help@pig.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@pig.apache.org Delivered-To: mailing list dev@pig.apache.org Received: (qmail 62049 invoked by uid 500); 28 Oct 2014 23:13:33 -0000 Delivered-To: apmail-hadoop-pig-dev@hadoop.apache.org Received: (qmail 62046 invoked by uid 99); 28 Oct 2014 23:13:33 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Oct 2014 23:13:33 +0000 Date: Tue, 28 Oct 2014 23:13:33 +0000 (UTC) From: "Daniel Dai (JIRA)" To: pig-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (PIG-4245) 1-1 edge vertices should use same jvm opts MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PIG-4245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14187678#comment-14187678 ] Daniel Dai commented on PIG-4245: --------------------------------- +1 > 1-1 edge vertices should use same jvm opts > ------------------------------------------- > > Key: PIG-4245 > URL: https://issues.apache.org/jira/browse/PIG-4245 > Project: Pig > Issue Type: Sub-task > Components: tez > Reporter: Rohini Palaniswamy > Assignee: Rohini Palaniswamy > Fix For: 0.14.0 > > Attachments: PIG-4245-1.patch > > > We use map jvm opts for root vertices and reduce jvm opts for intermediate and leaf vertices. For a simple order by or skewed join the Sampler will be a root vertex and Partitioner will be a intermediate vertex. If the map opts and reduce opts are different containers will not be reused and there will be lot of resource wastage when the reduce opts specify higher memory than map which is usually the case. The partitioner does not do much and just does identity mapping. So we should be good using map opts there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)