impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Juan Yu (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-2076: Correct execution time tracking for DataStreamSender.
Date Wed, 20 Apr 2016 20:52:49 GMT
Hello Henry Robinson,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/2578

to look at the new patch set (#7).

Change subject: IMPALA-2076: Correct execution time tracking for DataStreamSender.
......................................................................

IMPALA-2076: Correct execution time tracking for DataStreamSender.

DataStreamSender uses multiple channels (each channel uses one thread)
to send data. We want to know the active wall-clock time used by
DataStreamSender no matter how many threads are used so we only count
concurrent sending time and row batch serialization time. We don't want
to count recv_TransmitData() time because it is mainly to wait for
receiver to deserialize data and process data, not using much network,
plus those part are already tracked by Exchange node.
Added a ConcurrentStopWatch class to get concurrent transmit time.
If a thread is already running, the following thread won't reset the
stop watch. And the stop watch is stopped only after all threads finish
their work.

Change-Id: I9c530b2056d852c8bcac6263e9e6b1a6bede1047
---
A be/src/runtime/backend-client.h
M be/src/runtime/client-cache.h
M be/src/runtime/coordinator.cc
M be/src/runtime/coordinator.h
M be/src/runtime/data-stream-sender.cc
M be/src/runtime/data-stream-sender.h
M be/src/runtime/data-stream-test.cc
M be/src/runtime/exec-env.cc
M be/src/runtime/exec-env.h
M be/src/runtime/runtime-filter.cc
M be/src/runtime/runtime-state.h
M be/src/service/fragment-exec-state.cc
M be/src/service/fragment-exec-state.h
M be/src/util/runtime-profile-test.cc
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M be/src/util/stopwatch.h
17 files changed, 370 insertions(+), 44 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/78/2578/7
-- 
To view, visit http://gerrit.cloudera.org:8080/2578
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9c530b2056d852c8bcac6263e9e6b1a6bede1047
Gerrit-PatchSet: 7
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Juan Yu <jyu@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Henry Robinson <henry@cloudera.com>
Gerrit-Reviewer: Juan Yu <jyu@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sailesh@cloudera.com>

Mime
View raw message