hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Warhaftig (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-15563) 'counter' may overflow in BoundedGroupingStrategy
Date Sat, 30 Apr 2016 19:46:12 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Matt Warhaftig updated HBASE-15563:
    Attachment: hbase-15563-v1.patch

Attached patch 'hbase-15563-v1.patch' switches counter to AtomicLong.  Existing WAL test cases
continue to pass but no new dedicated boundary tests are added.

"before the overflow we will run into OOM because of a huge groupNameCache."
Agreed, with 6GB of memory dedicated to just a ConcurrentHashMap and trying to simulate the
groupNameCache region entries I topped out at 28 million cache entries.  Counter's {{Integer.MAX_VALUE}}
limit was a few orders of magnitude larger at 2 billion.  Still, since ConcurrentHashMap size
is unbound, maybe with server-sized memory allocations the limiting factor could shift to

> 'counter' may overflow in BoundedGroupingStrategy
> -------------------------------------------------
>                 Key: HBASE-15563
>                 URL: https://issues.apache.org/jira/browse/HBASE-15563
>             Project: HBase
>          Issue Type: Bug
>          Components: wal
>            Reporter: Duo Zhang
>            Assignee: Matt Warhaftig
>            Priority: Minor
>              Labels: beginner
>         Attachments: hbase-15563-v1.patch
> {code}
> groupName = groupNames[counter.getAndIncrement() % groupNames.length];
> {code}
> Theoretically, counter can overflow and becomes negative then causes an ArrayIndexOutOfBoundsException.
> But in practice, we need 2 billions different identifiers to make this happen, and before
the overflow we will run into OOM because of a huge groupNameCache...
> So not sure if it is worth to fix

This message was sent by Atlassian JIRA

View raw message