hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
Date Tue, 24 Mar 2015 05:58:54 GMT

     [ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-11544:
--------------------------
    Attachment: Allocation_Hot_Spots.html

jprofiler allows you get report on heap. It don't look too good. There is a NextState for
every three KVs it seems.  See attached report. Did it a few times and proportion is about
constant.

[~jonathan.lawlor] and I will huddle on the morrow.  We'll be back.

Thanks for the review that these allocations could be problematic. They don't seem to impinge
upon throughput going by my little measurements, though they do make for some more GC so probably
short-lived, but yeah, lets not take step backward on long-term project for getting object
creation out of the scan line.

> [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even
if it means OOME
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-11544
>                 URL: https://issues.apache.org/jira/browse/HBASE-11544
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: Jonathan Lawlor
>            Priority: Critical
>             Fix For: 2.0.0, 1.1.0
>
>         Attachments: Allocation_Hot_Spots.html, HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch,
HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, HBASE-11544-v4.patch, HBASE-11544-v5.patch,
HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, HBASE-11544-v8-branch-1.patch,
HBASE-11544-v8.patch, gc.j.png, h.png, hits.j.png, m.png, mean.png, net.j.png, q (2).png
>
>
> Running some tests, I set hbase.client.scanner.caching=1000.  Dataset has large cells.
 I kept OOME'ing.
> Serverside, we should measure how much we've accumulated and return to the client whatever
we've gathered once we pass out a certain size threshold rather than keep accumulating till
we OOME.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message