impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (JIRA)" <>
Subject [jira] [Resolved] (IMPALA-5166) Clean up BufferPool profile counters
Date Mon, 22 May 2017 15:02:04 GMT


Tim Armstrong resolved IMPALA-5166.
       Resolution: Fixed
    Fix Version/s: Impala 2.9.0

IMPALA-5166: clean up BufferPool counters

Misc changes to improve usability of the profiles.

* Separate out detailed BufferPool metrics into a "Buffer pool"
* Only create the limit counter if there is a limit
* Show BufferPool using in query MemTracker (it was accidentally
  disabled before because there was no query-level profile).
* Reduce clutter in MemTracker dump by only showing buffer pool
  reservation, not usage (the usage was misleading anyway because
  it didn't include child usage).
* Remove TotalUnpinnedBytes, which had limited value - WriteIoBytes
  and PeakUnpinnedBytes can answer most of the same questions - i.e.
  did it unpin any pages, and how many did it need to write to disk.
* Add buffer pool metrics to /memz (if buffer pool is enabled) and
  reorder /memz so more useful information is up the top.

Change-Id: I34b7f4d94c3d396ac89026c7559d6b2c6e02697c
Reviewed-by: Tim Armstrong <>
Tested-by: Impala Public Jenkins

> Clean up BufferPool profile counters
> ------------------------------------
>                 Key: IMPALA-5166
>                 URL:
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend
>    Affects Versions: Impala 2.9.0
>            Reporter: Tim Armstrong
>            Assignee: Tim Armstrong
>             Fix For: Impala 2.9.0
> We should clean-up the profile counters that are added to each node so that irrelevant
info is hidden and relevant info is easy to find:
> - Hide the limit if it is MAX_INT64
> - Remove the BufferPool prefix and move into a "Buffer pool" section, or similar
> - Remove or rename potentially confusing counters, e.g. 
> -- the cumulative allocation counters should be removed or renamed to be clearer that
they are cumulative
> -- PeakUnpinnedBytes is confusing unless you understand the internals

This message was sent by Atlassian JIRA

View raw message