hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: A possible bug in the scanner.
Date Wed, 13 Apr 2011 16:42:18 GMT
This could be HBASE-2077

J-D

On Wed, Apr 13, 2011 at 9:15 AM, Gary Helmling <ghelmling@gmail.com> wrote:
> Hi Vidhya,
>
> So it sounds like the timeout thread is timing out the scanner when it takes
> more than 60 seconds reading through the large column family store file
> without returning anything to the client?  Even without the TTL expiration
> being applied, I think I've heard of this in other cases where a very
> restrictive filter was used on a large table scan.
>
> If this is the case, it certainly seems like we should handle it better.  We
> could do something as simple as refreshing the scanner timestamp every X
> rows when iterating server side.
>
> I'll check the code and open a JIRA (if we don't have one existing).  Thanks
> for detailing the problem.
>
> --gh
>
>
>
> On Wed, Apr 13, 2011 at 7:44 AM, Vidhyashankar Venkataraman <
> vidhyash@yahoo-inc.com> wrote:
>
>> Hi
>>   We had enabled scanner caching but I don't think it is the same issue
>> because scanner.next in this case is blocking: the scanner is busy in the
>> region server but hasn't returned anything yet since a row to be returned
>> hasn't been found yet (all rows have expired but are still there since they
>> havent been compacted yet).
>>
>> Vidhya
>>
>> On 4/13/11 1:44 AM, "Ted Yu" <yuzhihong@gmail.com> wrote:
>>
>> Have you read the following thread ?
>> "ScannerTimeoutException when a scan enables caching, no exception when it
>> doesn't"Did you enable caching ? If not, it is different issue.
>>
>> On Wed, Apr 13, 2011 at 12:40 AM, Vidhyashankar Venkataraman <
>> vidhyash@yahoo-inc.com> wrote:
>>
>> > (This could be a known issue. Please let me know if it is).
>> >
>> > We had a set of uncompacted store files in a region. One of the column
>> > families had a store file of 5 Gigs. The other column families were
>> pretty
>> > small (a few megabytes at most).
>> >
>> >  It so turned out that all these files had rows whose TTL had expired.
>> Now
>> > when this region was scanned (which should yield a result of a null set),
>> we
>> > got Scanner timeouts and UnknownScannerExceptions.
>> >
>> > And when we tried scanning the region without the large column family,
>> the
>> > scanner returned back safely with no result.
>> >
>> > So, I major compacted it and the scan started working correctly.
>> >
>> > So it looks like timeouts happen if the scanner does not return any
>> output
>> > for a specified time.
>> > Which isn't exactly the correct thing to do, because it could be the case
>> > that the scanner was indeed busy but it just so happened that there are
>> no
>> > rows yet to return back to the client.
>> >
>> > We can try increasing the scanner timeout, but this doesn't resolve the
>> > underlying problem. Is this a know issue?
>> >
>> > Thank you
>> > Vidhya
>> >
>>
>>
>

Mime
View raw message