impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Volker (Code Review)" <>
Subject [Impala-CR](cdh5-trunk) IMPALA-2979: Fix scheduling on remote hosts
Date Fri, 15 Jul 2016 14:07:58 GMT
Lars Volker has uploaded a new patch set (#25).

Change subject: IMPALA-2979: Fix scheduling on remote hosts

IMPALA-2979: Fix scheduling on remote hosts

Also fixes: IMPALA-2400, IMPALA-3043

This change fixes scheduling scan-ranges on remote hosts by adding
remote backend selection capability to SimpleScheduler. Prior to this
change the scheduler would try to select a local backend even when
remote scheduling was requested.

This change also allows pseudo-randomized remote backend selection to
prevent convoying, which could happen when different independent
schedulers had the same internal state, e.g. after a cluster restart. To
enable the new behavior set the query option SCHEDULE_RANDOM_REPLICA to

This change also fixes IMPALA-2400: Unpredictable locality behavior
for reading Parquet files

This change also fixes IMPALA-3043: SimpleScheduler does not handle
hosts with multiple IP addresses correctly

This change also does some clean-up in scheduler.h and

Change-Id: I044f83806fcde820fcb38047cf6b8e780d803858
M be/src/scheduling/scheduler.h
M be/src/scheduling/
M be/src/scheduling/
M be/src/scheduling/simple-scheduler.h
M be/src/service/
M be/src/service/query-options.h
M common/thrift/ImpalaInternalService.thrift
M common/thrift/ImpalaService.thrift
M fe/src/main/java/com/cloudera/impala/analysis/
M fe/src/test/java/com/cloudera/impala/analysis/
M fe/src/test/java/com/cloudera/impala/analysis/
M fe/src/test/java/com/cloudera/impala/analysis/
12 files changed, 945 insertions(+), 474 deletions(-)

  git pull ssh:// refs/changes/00/2200/25
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I044f83806fcde820fcb38047cf6b8e780d803858
Gerrit-PatchSet: 25
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Lars Volker <>
Gerrit-Reviewer: Lars Volker <>
Gerrit-Reviewer: Marcel Kornacker <>
Gerrit-Reviewer: Matthew Jacobs <>
Gerrit-Reviewer: Mostafa Mokhtar <>
Gerrit-Reviewer: Sailesh Mukil <>
Gerrit-Reviewer: anujphadke <>

View raw message