hbase-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From li...@apache.org
Subject svn commit: r1417126 - in /hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver: Store.java StoreScanner.java
Date Tue, 04 Dec 2012 19:19:15 GMT
Author: liyin
Date: Tue Dec  4 19:19:14 2012
New Revision: 1417126

URL: http://svn.apache.org/viewvc?rev=1417126&view=rev
Log:
[HBASE-7266] Using pread for non-compaction read request

Author: liyintang

Summary:
There are 2 kinds of read operation in HBase: pread and seek+read.
Pread, positional read, is stateless and create a new connection between the DFSClient and
DataNode for each operation. While seek+read is to seek to a specific postion and prefetch
blocks from data nodes. The benefit of seek+read is that it will cache the prefetch result
but the downside is it is stateful and needs to synchronized.

So far, both compaction and scan are using seek+read, which caused some resource contention.
So using the pread for the scan request can avoid the resource contention. In addition, the
region server is able to do the prefetch for the scan request so that it won't be necessary
to let the DFSClient to prefetch the data any more.

I will run through the scan benchmark (with no block cache) with verify the performance.

Test Plan:
unit tests:
http://hbaseproxy002.prn1.facebook.com/mr_jt/hbasebackup004-ash3/jobdetails.jsp?jobid=job_201211281833_3437

Reviewers: kannan, kranganathan

Reviewed By: kannan

CC: hbase-eng@

Differential Revision: https://phabricator.fb.com/D645412

Modified:
    hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
    hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java

Modified: hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
URL: http://svn.apache.org/viewvc/hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java?rev=1417126&r1=1417125&r2=1417126&view=diff
==============================================================================
--- hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java (original)
+++ hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java Tue
Dec  4 19:19:14 2012
@@ -846,9 +846,7 @@ public class Store extends SchemaConfigu
    * @return all scanners for this store
    */
   protected List<KeyValueScanner> getScanners(boolean cacheBlocks,
-      boolean isGet,
-      boolean isCompaction,
-      ScanQueryMatcher matcher) throws IOException {
+      boolean isCompaction, ScanQueryMatcher matcher) throws IOException {
     List<StoreFile> storeFiles;
     List<KeyValueScanner> memStoreScanners;
     this.lock.readLock().lock();
@@ -866,7 +864,7 @@ public class Store extends SchemaConfigu
     // actually more correct, since memstore get put at the end.
     List<StoreFileScanner> sfScanners =
         StoreFileScanner.getScannersForStoreFiles(storeFiles, cacheBlocks,
-            isGet, isCompaction, matcher);
+            true, isCompaction, matcher);
     List<KeyValueScanner> scanners =
       new ArrayList<KeyValueScanner>(sfScanners.size()+1);
     scanners.addAll(sfScanners);

Modified: hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
URL: http://svn.apache.org/viewvc/hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java?rev=1417126&r1=1417125&r2=1417126&view=diff
==============================================================================
--- hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
(original)
+++ hbase/branches/0.89-fb/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
Tue Dec  4 19:19:14 2012
@@ -256,8 +256,7 @@ public class StoreScanner extends NonLaz
    */
   private List<KeyValueScanner> getScannersNoCompaction() throws IOException {
     final boolean isCompaction = false;
-    return selectScannersFrom(store.getScanners(cacheBlocks, isGet,
-        isCompaction, matcher));
+    return selectScannersFrom(store.getScanners(cacheBlocks, isCompaction, matcher));
   }
 
   /**



Mime
View raw message