impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Volker (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-2918: Unit test framework for simple scheduler
Date Tue, 03 May 2016 14:11:17 GMT
Hello Marcel Kornacker, Matthew Jacobs,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/2431

to look at the new patch set (#15).

Change subject: IMPALA-2918: Unit test framework for simple scheduler
......................................................................

IMPALA-2918: Unit test framework for simple scheduler

The SimpleScheduler class is currently hard to change because of the lack of
comprehensive tests. This change adds support classes to make writing tests
easier.

The overall testing approach looks like this: Each test builds a list of hosts,
a physical schema, and a plan, to all of which elements can be added using
various helper methods.  Then scheduling can be tested by instantiating
SchedulerWrapper and calling Compute(...). The result can be verified using a
set of helper methods. There are also helper methods to modify the internal
state of the scheduler between subsequent calls to SchedulerWrapper::Compute().

The model currently comes with some known limitations:

- Files map 1:1 to blocks and to scan ranges.
- All files have the same size (i.e., 1MB). Tables that differ in size can be
  expressed as having a different number of files.
- We don't support multiple backends on a single host.
- Ports are assigned to hosts automatically and are not configurable by the
  test.

Change-Id: Ia7aee9b16067e8728a8e96d4def3e568ad21f4bf
---
M be/src/scheduling/scheduler.h
M be/src/scheduling/simple-scheduler-test.cc
M be/src/scheduling/simple-scheduler.cc
M be/src/scheduling/simple-scheduler.h
4 files changed, 1,141 insertions(+), 192 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/31/2431/15
-- 
To view, visit http://gerrit.cloudera.org:8080/2431
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia7aee9b16067e8728a8e96d4def3e568ad21f4bf
Gerrit-PatchSet: 15
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Lars Volker <lv@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Silvius Rus <srus@cloudera.com>

Mime
View raw message