hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Helmling <ghelml...@gmail.com>
Subject Re: A possible bug in the scanner.
Date Wed, 13 Apr 2011 17:14:09 GMT
On Wed, Apr 13, 2011 at 10:03 AM, Vidhyashankar Venkataraman <
vidhyash@yahoo-inc.com> wrote:

> >> Even without the TTL expiration being applied, I think I've heard of
> this in other cases where a very
> >> restrictive filter was used on a large table scan.
> Thanks, I was about to say that in a follow-up mail! We use a filter to
> scan records, produce a list of delete records and bulk load them back to
> HBase. And the same problem will exist even in that case.
>
> And in response to JD's suggestions, this problem 'might' be related
> (mid-way I see JD's comment on scanner timeouts during GCs which is quite
> analogous to the problem that I had pointed out): I can't quite pinpoint
> exactly what the bug tries to address and if any fix has come out of it. JD,
> can you let me know the status of the JIRA?
>
> Thank you
> V
>
>
In follow up to my earlier comment on periodically renewing the lease, I
suppose in the case of a 60+ second GC pause this won't be sufficient and
we'd still timeout.  So maybe we do need a better solution.  It would
address just the data filtering issue though.  So it could be an improvement
over what we currently have.

>From HBASE-2077, the idea of multiple simultaneous RPC calls in to the same
scanner (and hence the need for ref counting instead of simple boolean or
state) does seem a bit odd though?  Would this be needed for a future
parallel scanner implementation?  Or do we have any clear cases where this
is currently used?

--gh

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message