Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 4 Mar 2015 09:30:05 +0000 (UTC)
From: "stack (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12728285.1405720476000.73685.1425461405875@Atlassian.JIRA>
In-Reply-To: <JIRA.12728285.1405720476000@Atlassian.JIRA>
References: <JIRA.12728285.1405720476000@Atlassian.JIRA>
 <JIRA.12728285.1405720476875@arcas>
Subject: [jira] [Updated] (HBASE-11544) [Ergonomics]
 hbase.client.scanner.caching is dogged and will try to return batch even if
 it means OOME
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack updated HBASE-11544:
--------------------------
    Attachment: mean.png
                hits.j.png
                gc.j.png
                net.j.png

Here's some pictures. I ran three profiles:

# Current branch-1.0
# Branch-1.0 with the 1.0 patch from here
# Above but I set configuration so size was such that maybe 10-20% of the returns required partials.

Dataset was 100M rows of ten columns zipfian sized between 0 and 8k. Avg row size somewhere between 160 and 220.

For each set up, did two tests: one with many clients and one with just two clients.

Looks like patch doesn't change general profile.  With config in place, a bit more GC probably because a bit more work was done -- extra rpcs -- but hit rate seems a bit higher... more throughput.  One odd thing is that the mean time seems to have gone up a little when partials going on.

Patch looking good. Let me commit tomorrow. I suppose its fine to start w/ 2MB as default size.

Need a release note [~jonathan.lawlor]

> [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-11544
>                 URL: https://issues.apache.org/jira/browse/HBASE-11544
>             Project: HBase
>          Issue Type: Bug
>            Reporter: stack
>            Assignee: Jonathan Lawlor
>            Priority: Critical
>              Labels: beginner
>         Attachments: HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v6.patch, gc.j.png, hits.j.png, mean.png, net.j.png
>
>
> Running some tests, I set hbase.client.scanner.caching=1000.  Dataset has large cells.  I kept OOME'ing.
> Serverside, we should measure how much we've accumulated and return to the client whatever we've gathered once we pass out a certain size threshold rather than keep accumulating till we OOME.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)