hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: HBase-0.94.2-SNAPSHOT Scanning Bug
Date Fri, 24 Aug 2012 01:28:12 GMT
We tried reproducing on a local node but it doesn't show up. It did
show as soon as we put it on our dev cluster.

J-D

On Thu, Aug 23, 2012 at 6:10 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:
> It is interesting, though, because I have been running my local perf testing with this
change included and have not seen this issue.
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: lars hofhansl <lhofhansl@yahoo.com>
> To: "dev@hbase.apache.org" <dev@hbase.apache.org>
> Cc:
> Sent: Thursday, August 23, 2012 6:05 PM
> Subject: Re: HBase-0.94.2-SNAPSHOT Scanning Bug
>
> This:
>
> "IPC Server handler 43 on 10304" daemon prio=10 tid=0x00007f16b8b1f000 nid=0x6414 runnable
[0x00007f16b47c6000]
>    java.lang.Thread.State: RUNNABLE
>     at org.apache.hadoop.hbase.KeyValue.createFirstOnRowColTS(KeyValue.java:1893)
>     at org.apache.hadoop.hbase.regionserver.StoreFileScanner.requestSeek(StoreFileScanner.java:310)
>     at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:297)
>     at org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:256)
>     at org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:522)
>     - locked <0x00000006cec5bd88> (a org.apache.hadoop.hbase.regionserver.StoreScanner)
>     at org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:54)
>     at org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.requestSeek(NonLazyKeyValueScanner.java:38)
>     at org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:297)
>     at org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:256)
>     at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRow(HRegion.java:3508)
>
> points to my change: https://issues.apache.org/jira/browse/HBASE-6577
>
> The trace is interesting: RegionScannerImpl.nextRow now seeks to the last KV in the row
and then iterates as before.
> However, then the reseek internally seeks to the first KV of the column, and somehow
this interaction makes no progress forward.
>
> I'll revert that change.
>
> -- Lars
>
>
> ________________________________
> From: Elliott Clark <eclark@stumbleupon.com>
> To: dev@hbase.apache.org
> Sent: Thursday, August 23, 2012 5:39 PM
> Subject: HBase-0.94.2-SNAPSHOT Scanning Bug
>
> I recently tried to update one of our clusters to a version of 0.94.2
> seen here: https://github.com/stumbleupon/hbase/commits/su_prod_94
>
> When doing that all of the nodes started taking all available cpu
> time.  Not much interesting was in the logs however jstacks looked
> like this: http://pastebin.com/raw.php?i=fw6P5RKE  Everything is
> spinning in scans.  A version of 0.94.1 works perfectly and reverting
> solved all issues.  I don't really have enough data to point at any
> jira as the cause I was just wondering if anyone had some insight into
> the few commits between 0.94.1 release and the head of the above
> github that could cause scans to spin.
>
> Thanks
>

Mime
View raw message