hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Lawlor (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-13090) Progress heartbeats for long running scanners
Date Fri, 13 Mar 2015 21:24:41 GMT

     [ https://issues.apache.org/jira/browse/HBASE-13090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jonathan Lawlor updated HBASE-13090:
    Attachment: HBASE-13090-v2.patch

Attaching a patch that includes the ScannerLimit idea. It does seem to make things a little
nicer and presents a cleaner interface for dealing with RegionScanner#next() and InternalScanner#next
when many limits need to be specified. What do you guys think? 

This patch has also managed to get rid of the ugly postHeapNext method that was included before.
Now in tests we use a custom key value heap class to insert delays between column family fetches
to simulate long running scans on the server side. I am still looking into how I could remove
the sleeps and replace them with latches.

Will update reviewboard with latest diff

> Progress heartbeats for long running scanners
> ---------------------------------------------
>                 Key: HBASE-13090
>                 URL: https://issues.apache.org/jira/browse/HBASE-13090
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>            Assignee: Jonathan Lawlor
>         Attachments: HBASE-13090-v1.patch, HBASE-13090-v2.patch
> It can be necessary to set very long timeouts for clients that issue scans over large
regions when all data in the region might be filtered out depending on scan criteria. This
is a usability concern because it can be hard to identify what worst case timeout to use until
scans are occasionally/intermittently failing in production, depending on variable scan criteria.
It would be better if the client-server scan protocol can send back periodic progress heartbeats
to clients as long as server scanners are alive and making progress.
> This is related but orthogonal to streaming scan (HBASE-13071). 

This message was sent by Atlassian JIRA

View raw message