From reviews-return-27115-archive-asf-public=cust-asf.ponee.io@impala.apache.org Thu Jan 11 23:52:34 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id ABB89180656 for ; Thu, 11 Jan 2018 23:52:34 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 9BD37160C23; Thu, 11 Jan 2018 22:52:34 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E9618160C13 for ; Thu, 11 Jan 2018 23:52:33 +0100 (CET) Received: (qmail 21553 invoked by uid 500); 11 Jan 2018 22:52:33 -0000 Mailing-List: contact reviews-help@impala.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@impala.apache.org Received: (qmail 21542 invoked by uid 99); 11 Jan 2018 22:52:32 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Jan 2018 22:52:32 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 6D664C47BA for ; Thu, 11 Jan 2018 22:52:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.362 X-Spam-Level: ** X-Spam-Status: No, score=2.362 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=2, RDNS_DYNAMIC=0.363, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id As4cwqwc6HQI for ; Thu, 11 Jan 2018 22:52:31 +0000 (UTC) Received: from ip-10-146-233-104.ec2.internal (ec2-75-101-130-251.compute-1.amazonaws.com [75.101.130.251]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id CACF45F58B for ; Thu, 11 Jan 2018 22:52:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ip-10-146-233-104.ec2.internal (8.14.4/8.14.4) with ESMTP id w0BMqTTh032737; Thu, 11 Jan 2018 22:52:29 GMT Message-Id: <201801112252.w0BMqTTh032737@ip-10-146-233-104.ec2.internal> X-Gerrit-PatchSet: 4 Date: Thu, 11 Jan 2018 22:52:29 +0000 From: "Impala Public Jenkins (Code Review)" To: Bharath Vissapragada , impala-cr@cloudera.com, reviews@impala.incubator.apache.org X-Gerrit-MessageType: merged Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-6384=3A_RequestPoolService_should_honor_custom_group_mapping_config=0A?= X-Gerrit-Change-Id: Ibb93870c0cc37e2432a643a274931f1d3d13fb96 X-Gerrit-Change-Number: 9000 X-Gerrit-ChangeURL: X-Gerrit-Commit: 20daa4d516987242ed7da5fe9474267666cf8add In-Reply-To: References: Reply-To: impala-cr@cloudera.com, bharathv@cloudera.com, reviews@impala.incubator.apache.org MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.14.2 Content-Type: multipart/alternative; boundary="Iv7xhQsRe6o="; charset=UTF-8 --Iv7xhQsRe6o= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Impala Public Jenkins has submitted this change and it was merged=2E ( http= ://gerrit=2Ecloudera=2Eorg:8080/9000 ) Change subject: IMPALA-6384: Reques= tPoolService should honor custom group mapping config =2E=2E=2E=2E=2E=2E=2E= =2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E= =2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E= =2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E IMPALA-6384: RequestPoolService sh= ould honor custom group mapping config Due to the way in which we instanti= ate fair scheduler allocation loader, we donot read the config overrides fr= om the HDFS config files=2E This is an unexpected behavior from users' POV= since we typically support overrides like custom user -> group mapping via= HDFS config (for ex: LDAPGroupsMapping) that eventually affects the query = -> pool assignment=2E Fix: This patch loads the hadoop default configurati= on so that the underlying QueuePlacementPolicy is based on user specified o= verrides=2E Testing (manual): Changed the core-site=2Exml to use LDAPGroup= sMapping instead of the default ShellBasedUnixGroupsMapping and confirmed t= hat the correct group mapping plugin is loaded, by adding additional loggin= g=2E Also, modified TestRequestPoolService to assert that the core-site xm= l overrides are loaded=2E Change-Id: Ibb93870c0cc37e2432a643a274931f1d3d13= fb96 Reviewed-on: http://gerrit=2Ecloudera=2Eorg:8080/9000 Reviewed-by: Bha= rath Vissapragada Tested-by: Impala Public Jenkin= s --- M common/yarn-extras/src/main/java/org/apache/impala/yarn/server/reso= urcemanager/scheduler/fair/QueuePlacementPolicy=2Ejava M fe/src/main/java/o= rg/apache/impala/util/RequestPoolService=2Ejava M fe/src/test/java/org/apac= he/impala/util/TestRequestPoolService=2Ejava M testdata/cluster/node_templa= tes/common/etc/hadoop/conf/core-site=2Exml=2Etmpl 4 files changed, 32 inser= tions(+), 2 deletions(-) Approvals: Bharath Vissapragada: Looks good to = me, approved Impala Public Jenkins: Verified -- To view, visit http://g= errit=2Ecloudera=2Eorg:8080/9000 To unsubscribe, visit http://gerrit=2Eclou= dera=2Eorg:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master = Gerrit-MessageType: merged Gerrit-Change-Id: Ibb93870c0cc37e2432a643a274931= f1d3d13fb96 Gerrit-Change-Number: 9000 Gerrit-PatchSet: 4 Gerrit-Owner: Bha= rath Vissapragada Gerrit-Reviewer: Bharath Vissap= ragada Gerrit-Reviewer: Dimitris Tsirogiannis Gerrit-Reviewer: Impala Public Jenkins Gerrit-R= eviewer: Tim Armstrong Gerrit-Reviewer: Zoram T= hanga --Iv7xhQsRe6o=--