impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IMPALA-5166) Clean up BufferPool profile counters
Date Mon, 22 May 2017 15:02:04 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-5166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tim Armstrong resolved IMPALA-5166.
-----------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.9.0



IMPALA-5166: clean up BufferPool counters

Misc changes to improve usability of the profiles.

* Separate out detailed BufferPool metrics into a "Buffer pool"
  sub-profile.
* Only create the limit counter if there is a limit
* Show BufferPool using in query MemTracker (it was accidentally
  disabled before because there was no query-level profile).
* Reduce clutter in MemTracker dump by only showing buffer pool
  reservation, not usage (the usage was misleading anyway because
  it didn't include child usage).
* Remove TotalUnpinnedBytes, which had limited value - WriteIoBytes
  and PeakUnpinnedBytes can answer most of the same questions - i.e.
  did it unpin any pages, and how many did it need to write to disk.
* Add buffer pool metrics to /memz (if buffer pool is enabled) and
  reorder /memz so more useful information is up the top.

Change-Id: I34b7f4d94c3d396ac89026c7559d6b2c6e02697c
Reviewed-on: http://gerrit.cloudera.org:8080/6690
Reviewed-by: Tim Armstrong <tarmstrong@cloudera.com>
Tested-by: Impala Public Jenkins
---

> Clean up BufferPool profile counters
> ------------------------------------
>
>                 Key: IMPALA-5166
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5166
>             Project: IMPALA
>          Issue Type: Sub-task
>          Components: Backend
>    Affects Versions: Impala 2.9.0
>            Reporter: Tim Armstrong
>            Assignee: Tim Armstrong
>             Fix For: Impala 2.9.0
>
>
> We should clean-up the profile counters that are added to each node so that irrelevant
info is hidden and relevant info is easy to find:
> - Hide the limit if it is MAX_INT64
> - Remove the BufferPool prefix and move into a "Buffer pool" section, or similar
> - Remove or rename potentially confusing counters, e.g. 
> -- the cumulative allocation counters should be removed or renamed to be clearer that
they are cumulative
> -- PeakUnpinnedBytes is confusing unless you understand the internals



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message