hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
Date Wed, 04 Mar 2015 09:30:05 GMT

     [ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-11544:
    Attachment: mean.png

Here's some pictures. I ran three profiles:

# Current branch-1.0
# Branch-1.0 with the 1.0 patch from here
# Above but I set configuration so size was such that maybe 10-20% of the returns required

Dataset was 100M rows of ten columns zipfian sized between 0 and 8k. Avg row size somewhere
between 160 and 220.

For each set up, did two tests: one with many clients and one with just two clients.

Looks like patch doesn't change general profile.  With config in place, a bit more GC probably
because a bit more work was done -- extra rpcs -- but hit rate seems a bit higher... more
throughput.  One odd thing is that the mean time seems to have gone up a little when partials
going on.

Patch looking good. Let me commit tomorrow. I suppose its fine to start w/ 2MB as default

Need a release note [~jonathan.lawlor]

> [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even
if it means OOME
> ------------------------------------------------------------------------------------------------------
>                 Key: HBASE-11544
>                 URL: https://issues.apache.org/jira/browse/HBASE-11544
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: Jonathan Lawlor
>            Priority: Critical
>              Labels: beginner
>         Attachments: HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch,
HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, HBASE-11544-v4.patch, HBASE-11544-v5.patch,
HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, gc.j.png, hits.j.png, mean.png,
> Running some tests, I set hbase.client.scanner.caching=1000.  Dataset has large cells.
 I kept OOME'ing.
> Serverside, we should measure how much we've accumulated and return to the client whatever
we've gathered once we pass out a certain size threshold rather than keep accumulating till
we OOME.

This message was sent by Atlassian JIRA

View raw message