hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew Byng-Maddick (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8481) Add cost tracking to API calls for resource management
Date Fri, 08 Jan 2016 13:35:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15089209#comment-15089209
] 

Matthew Byng-Maddick commented on HBASE-8481:
---------------------------------------------

In discussion with Lars, the metrics we should expose are:
* for a block cache miss, the size of the block read (this is already there, I suspect, though
maybe not quite as explicitly)
* for a block cache miss in reading a KV, the size of the KV read

Between the two, we can expose a trend of how much excess data we're reading off the disk
to help with making decisions on how to size blocks in the HFile.

> Add cost tracking to API calls for resource management
> ------------------------------------------------------
>
>                 Key: HBASE-8481
>                 URL: https://issues.apache.org/jira/browse/HBASE-8481
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Lars George
>
> All calls that require resource should be tracked internally using a lightweight mechanism,
such as a simple counter record. The idea is to be able to tell from calls how many resources
they did require, for example a Get or Scan, how much true I/O was used. This should then
be exposed to the Coprocessor context so that a plugin can make cost based decisions going
forward with other calls. The record should include custom tags that in the preXYZ() calls
can be used to for example tag a record with a user group and therefore be able to aggregate
the statistics and accounting for groups of users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message