hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1823) Ability for Scanners to bypass the block cache
Date Fri, 11 Sep 2009 21:41:57 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754364#action_12754364
] 

Jonathan Gray commented on HBASE-1823:
--------------------------------------

Hmm... now when running with addition to the Writable, I get this exception:

{noformat}
2009-09-11 13:14:03,903 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan ROOT region
java.io.IOException: Call to /192.168.249.102:61020 failed on local exception: java.io.EOFException
        at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:757)
        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:727)
        at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:328)
        at $Proxy1.openScanner(Unknown Source)
        at org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:160)
        at org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
        at org.apache.hadoop.hbase.master.RootScanner.initialScan(RootScanner.java:73)
        at org.apache.hadoop.hbase.master.BaseScanner.initialChore(BaseScanner.java:131)
        at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
Caused by: java.io.EOFException
        at java.io.DataInputStream.readInt(DataInputStream.java:375)
        at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:505)
        at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:449)
{noformat}

The code seems right... what am I missing here?

{noformat}
@@ -518,6 +545,7 @@
     this.stopRow = Bytes.readByteArray(in);
     this.maxVersions = in.readInt();
     this.caching = in.readInt();
+    this.cacheBlocks = in.readBoolean();
     if(in.readBoolean()) {
       this.filter = (Filter)createForName(Bytes.toString(Bytes.readByteArray(in)));
       this.filter.readFields(in);
@@ -550,6 +578,7 @@
     Bytes.writeByteArray(out, this.stopRow);
     out.writeInt(this.maxVersions);
     out.writeInt(this.caching);
+    out.writeBoolean(this.cacheBlocks);
     if(this.filter == null) {
       out.writeBoolean(false);
     } else {
{noformat}

> Ability for Scanners to bypass the block cache
> ----------------------------------------------
>
>                 Key: HBASE-1823
>                 URL: https://issues.apache.org/jira/browse/HBASE-1823
>             Project: Hadoop HBase
>          Issue Type: New Feature
>          Components: client, regionserver
>    Affects Versions: 0.20.0
>            Reporter: Jonathan Gray
>            Assignee: Jonathan Gray
>             Fix For: 0.20.1, 0.21.0
>
>         Attachments: HBASE-1823-v1.patch, HBASE-1823-v2.patch, HBASE-1823-v3.patch
>
>
> There are a number of use cases where exposing the ability to not cache blocks during
a scan would be valuable.  For example, running row counts.
> The LRU is scan-resistant, so it does provide some protection already, but even in that
case all you prevent is the eviction of hot blocks.  The LRU still runs many evictions and
the blocks are referenced for much longer periods of time, so this adds enormous stress to
the GC.
> Compactions already do this.  This issue is about exposing that as a switch to the client-side
Scan object (will also enable it for MR jobs then).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message