impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-5417: make I/O buffer queue fixed-size
Date Thu, 03 Aug 2017 23:58:26 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-5417: make I/O buffer queue fixed-size

Patch Set 7:

I thought about it some more and I'm not sure that adding meaningful timers is simple. We
already have TotalStorageWaitTime that measures the time the client spent blocked for a fragment.
It might make sense to add a timer per exec node called "IoWaitTime" or "StorageWaitTime"
that gives the granular timing.

It's harder to produce a meaningful time for how long the queue was full, i.e. we were blocked
on compute, because there could be many scan ranges per thread. We could include a counter
of the number of times the queue was filled up, e.g. "IoBufferQueueFullCount", but that might
not be terribly intuitive.

I think that would give us enough information to see if the queue was too small - in that
case we'd see significant IoWaitTime and non-zero IoBufferQueueFullCount

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: If7cc3f7199f5320db00b7face97a96cdadb6f83f
Gerrit-PatchSet: 7
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <>
Gerrit-Reviewer: Matthew Jacobs <>
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Michael Ho <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-HasComments: No

View raw message