hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nspiegelb...@apache.org
Subject svn commit: r1233732 - /hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
Date Fri, 20 Jan 2012 02:23:06 GMT
Author: nspiegelberg
Date: Fri Jan 20 02:23:05 2012
New Revision: 1233732

URL: http://svn.apache.org/viewvc?rev=1233732&view=rev
[HBASE-3421] Fetch one KV at a time at compaction

Summary: Compactions have been failing in production, bringing the regionserver
down with an OOM. The hypothesis is that there is a row with lots of KVs and
they are brought into memory all at once. This fix makes a compaction fetch one
KV at a time. Compared to the open-source fix
(https://issues.apache.org/jira/browse/HBASE-3421) we are setting the limit to 1
instead of a configurable number because there is no real batching benefit
(everything is already happening on the server side).

Test Plan: Unit tests. Push to dark launch.

Reviewers: kannan

Reviewed By: kannan

CC: gqchen, hbase-eng@lists, kannan

Differential Revision: https://phabricator.fb.com/D390381

Revert Plan: OK


Modified: hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
URL: http://svn.apache.org/viewvc/hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java?rev=1233732&r1=1233731&r2=1233732&view=diff
--- hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java (original)
+++ hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Fri
Jan 20 02:23:05 2012
@@ -590,7 +590,7 @@ public class Store extends SchemaConfigu
           final List<KeyValue> kvs = new ArrayList<KeyValue>();
           boolean hasMore;
           do {
-            hasMore = scanner.next(kvs);
+            hasMore = scanner.next(kvs, 1);
             if (!kvs.isEmpty()) {
               for (KeyValue kv : kvs) {
                 // If we know that this KV is going to be included always, then let us

View raw message