impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Impala Public Jenkins (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-6384: RequestPoolService should honor custom group mapping config
Date Thu, 11 Jan 2018 22:52:29 GMT
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/9000
)

Change subject: IMPALA-6384: RequestPoolService should honor custom group mapping config
......................................................................

IMPALA-6384: RequestPoolService should honor custom group mapping config

Due to the way in which we instantiate fair scheduler allocation
loader, we donot read the config overrides from the HDFS config
files.

This is an unexpected behavior from users' POV since we typically
support overrides like custom user -> group mapping via HDFS
config (for ex: LDAPGroupsMapping) that eventually affects the
query -> pool assignment.

Fix: This patch loads the hadoop default configuration so that the
underlying QueuePlacementPolicy is based on user specified overrides.

Testing (manual): Changed the core-site.xml to use LDAPGroupsMapping
instead of the default ShellBasedUnixGroupsMapping and confirmed that
the correct group mapping plugin is loaded, by adding additional logging.

Also, modified TestRequestPoolService to assert that the core-site xml
overrides are loaded.

Change-Id: Ibb93870c0cc37e2432a643a274931f1d3d13fb96
Reviewed-on: http://gerrit.cloudera.org:8080/9000
Reviewed-by: Bharath Vissapragada <bharathv@cloudera.com>
Tested-by: Impala Public Jenkins
---
M common/yarn-extras/src/main/java/org/apache/impala/yarn/server/resourcemanager/scheduler/fair/QueuePlacementPolicy.java
M fe/src/main/java/org/apache/impala/util/RequestPoolService.java
M fe/src/test/java/org/apache/impala/util/TestRequestPoolService.java
M testdata/cluster/node_templates/common/etc/hadoop/conf/core-site.xml.tmpl
4 files changed, 32 insertions(+), 2 deletions(-)

Approvals:
  Bharath Vissapragada: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/9000
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ibb93870c0cc37e2432a643a274931f1d3d13fb96
Gerrit-Change-Number: 9000
Gerrit-PatchSet: 4
Gerrit-Owner: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Bharath Vissapragada <bharathv@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Zoram Thanga <zoram@cloudera.com>

Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message