hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars George <lars.geo...@gmail.com>
Subject Re: RPC Metrics
Date Wed, 18 May 2011 17:16:58 GMT
So, running YSCB, which uses the client-side write buffer, you see this


the put_num_ops was at 41 before, so was not incremented. Internally
the HTable uses the new multi() call, which is sending N puts in one
go to the server. The RPC layer counts it as one, therefore much
smaller than the number of puts.

The multi call does


which means it adds the right number to the requestCount, but that is
a lumped together metric counting everything.

In fact, I would even claim this is wrong. The request count should be
increased by one! But the other metrics for write operation should be
increased appropriately. But that is also not right, as the entire
put(Pair[]) is counted as


as one.

So, what are the real values for N client side puts to server metrics
you ask? I can clearly say: no idea!



On Wed, May 18, 2011 at 6:57 PM, Lars George <lars.george@gmail.com> wrote:
> Hi Mathias,
> Keep in mind that they are kept as MetricsTimeVaryingRate meaning they
> trail by "period" seconds. The values are accrued over the poll
> interval and then updated at the end.
> But you are saying there is a huge difference? I think this is due to
> the puts not actually using puts anymore, but the "multi" calls, ie.
> multiPut and multi. Could you verify the numbers you see there?
> Lars
> On Wed, May 18, 2011 at 4:48 PM, Mathias Herberts
> <mathias.herberts@gmail.com> wrote:
>> While on the topic of metrics, some of them seem misleading,
>> in my setup, 'hbase.metrics.put_num_ops' does not seem to reflect the
>> number of put ops received by a RS, it's simply way below and seems to
>> only reflect puts done in .META., is that supposed to be the case?
>> On Wed, May 18, 2011 at 16:42, Stack <stack@duboce.net> wrote:
>>> MIA I'd say.  They look useful.
>>> St.Ack
>>> On Wed, May 18, 2011 at 6:34 AM, Lars George <lars.george@gmail.com> wrote:
>>>> Hi,
>>>> Did anyone else notice that the RPCMetrics are supposed to export four
>>>> internal values:
>>>>  public MetricsTimeVaryingRate rpcQueueTime =
>>>>          new MetricsTimeVaryingRate("RpcQueueTime", registry);
>>>>  public MetricsTimeVaryingRate rpcProcessingTime =
>>>>          new MetricsTimeVaryingRate("RpcProcessingTime", registry);
>>>>  public MetricsIntValue numOpenConnections =
>>>>          new MetricsIntValue("NumOpenConnections", registry);
>>>>  public MetricsIntValue callQueueLen =
>>>>          new MetricsIntValue("callQueueLen", registry);
>>>> The first two show just fine on JMX and Context based output, but the
>>>> latter two don't. Nowhere.
>>>> Anyone having a clue where they went MIA?
>>>> Lars

View raw message