impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5417: make I/O buffer queue fixed-size
Date Thu, 03 Aug 2017 23:58:26 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-5417: make I/O buffer queue fixed-size
......................................................................


Patch Set 7:

I thought about it some more and I'm not sure that adding meaningful timers is simple. We
already have TotalStorageWaitTime that measures the time the client spent blocked for a fragment.
It might make sense to add a timer per exec node called "IoWaitTime" or "StorageWaitTime"
that gives the granular timing.

It's harder to produce a meaningful time for how long the queue was full, i.e. we were blocked
on compute, because there could be many scan ranges per thread. We could include a counter
of the number of times the queue was filled up, e.g. "IoBufferQueueFullCount", but that might
not be terribly intuitive.

I think that would give us enough information to see if the queue was too small - in that
case we'd see significant IoWaitTime and non-zero IoBufferQueueFullCount

-- 
To view, visit http://gerrit.cloudera.org:8080/7408
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: If7cc3f7199f5320db00b7face97a96cdadb6f83f
Gerrit-PatchSet: 7
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: No

Mime
View raw message