hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Lawlor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13269) Limit result array preallocation to avoid OOME with large scan caching values
Date Tue, 17 Mar 2015 21:41:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366142#comment-14366142
] 

Jonathan Lawlor commented on HBASE-13269:
-----------------------------------------

After a quick discussion in HBASE-11544 the preallocation was removed because of this issue
(there didn't seem to be any way to accurately estimate the number of Results that will be
returned since the caching and result size limits work together). This means that in branch-1+
there is no preallocation (this change should have been backported further to prevent the
OOM in earlier versions)

If a more sophisticated solution is found here, then that change could be added to branch-1+
(HBASE-11544 is only in branch-1+) otherwise it may just be best to remove the preallocation
all together.

> Limit result array preallocation to avoid OOME with large scan caching values
> -----------------------------------------------------------------------------
>
>                 Key: HBASE-13269
>                 URL: https://issues.apache.org/jira/browse/HBASE-13269
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>             Fix For: 1.0.1, 0.98.12
>
>
> Scan#setCaching(Integer.MAX_VALUE) will likely terminate the regionserver with an OOME
due to preallocation of the result array according to this parameter.  We should limit the
preallocation to some sane value. Definitely affects 0.98 (fix needed to HRegionServer) and
1.0.x (fix needed to RsRPCServices), not sure about later versions. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message