phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-1751) Perform aggregations, sorting, etc, in the preScannerNext instead of postScannerOpen
Date Wed, 24 Aug 2016 20:23:20 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435607#comment-15435607
] 

Andrew Purtell edited comment on PHOENIX-1751 at 8/24/16 8:22 PM:
------------------------------------------------------------------

bq. Here's the patch for 0.98 with the test passing

What did you do to get the test to pass?

0.98 doesn't have scanner heartbeating. That's a 1.1 and up feature. So if your pre hook work
exceeds the RPC timeout the client will retry, and if meanwhile the server side work eventually
does complete, then it is not a lost request, the expected request ID is incremented, the
retrying client will be surprised (eventually). Suggestions:

- Break up the work in small enough chunks so the RPC timeout is not exceeded

- Don't make this change on the Phoenix branch for 0.98


was (Author: apurtell):
bq. Here's the patch for 0.98 with the test passing

What did you do to get the test to pass?

0.98 doesn't have scanner heartbeating. That's a 1.1 and up feature. So if your pre hook work
exceeds the RPC timeout the client will retry, and if meanwhile the server side work eventually
does complete, then it is not a lost request, the expected request ID is incremented, the
retrying client will be surprised (eventually). Suggestions:

- Break up the work in small enough chunks so the RPC timeout is not exceeded

- Don't make this change on the Phoenix branch for 0.98

- Implement custom endpoints for aggregations

> Perform aggregations, sorting, etc, in the preScannerNext instead of postScannerOpen
> ------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-1751
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1751
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>         Attachments: 1751-WIP-v2.txt, 1751-WIP-v2b.patch, 1751-WIP.txt, PHOENIX-1751-0.98.patch,
PHOENIX-1751-v2c.patch, PHOENIX-1751.patch, PHOENIX-1751_v3.patch, PHOENIX-1751_v4.patch
>
>
> HBase retains a lease for every scanner. Then lease expires the scan will no longer (be
allowed to) work. The leases guard against the client going away, and allow cleaning up resources
if that happens.
> At various points HBase "suspends" the lease while the region server are working on behalf
of this scanner, so that the lease won't expire even though the server is working on it.
> HBase does that during the scanning process. Crucially it suspends the leaser after the
scanner is opened, before next() is issued on it.
> The outcome of all this is that Phoenix executes aggregates, sorts, etc, with the lease
in place, and hence if these take a bit the lease can expire even though the server was working
on it.
> Phoenix should do this work in preScannerNext, being careful that the precalculation
is only performed once.
> I'll attach a sample patch soon.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message