phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Samarth Jain (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-1261) Update stats table asynchronously
Date Tue, 15 Dec 2015 01:15:46 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15057116#comment-15057116
] 

Samarth Jain edited comment on PHOENIX-1261 at 12/15/15 1:15 AM:
-----------------------------------------------------------------

Work in progress patch. I was able to manually run UPDATE STATS call which returned immediately
with the scanners chugging along on the region servers. This patch doesn't have the grammar
changes in yet i.e. by default we are running UPDATE STATS asynchronously. 

There are a couple of open questions though:
1) Should phoenix be using it's own thread pool on the region server side? Or is there a way
to get hold of an existing thread pool running on region servers? 
Currently I created a size 2 thread pool (will make it configurable) which is going be shared
by all update stats calls running on a region server.

2) Currently I am making sure that the wrapped scanner returned by doPostScannerOpen doesn't
close the actual internal region scanner. We manage the closing of the internal scanners ourselves.
Are there any untoward ramifications of doing that? 

PS - will remove the System.out calls in the actual patch. Parking the wip patch for preliminary
feedback.


was (Author: samarthjain):
Work in progress patch. I was able to manually run UPDATE STATS call which returned immediately
with the scanners chugging along on the region servers. This patch doesn't have the grammar
changes in yet i.e. by default we are running UPDATE STATS asynchronously. 

There are a couple of open questions though:
1) Should phoenix be using it's own thread pool on the region server side? Or is there a way
to get hold of an existing thread pool running on region servers? 
Currently I created a size 2 thread pool (will make it configurable) which is going be shared
by all update stats calls running on a region server.

2) Currently I am making sure that the wrapped scanner returned by doPostScannerOpen doesn't
close the actual internal region scanner. We manage the closing of the internal scanners ourselves.
Are there any untoward ramifications of doing that? 

> Update stats table asynchronously
> ---------------------------------
>
>                 Key: PHOENIX-1261
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1261
>             Project: Phoenix
>          Issue Type: Sub-task
>            Reporter: James Taylor
>            Assignee: Samarth Jain
>             Fix For: 4.7.0
>
>         Attachments: 1261-wip.patch
>
>
> Instead of writing the the stats table directly in the thread performing major compaction,
we should instead write to it asynchronously, perhaps using the same asynchronous mechanism
used by tracing. Apparently HBase used to have a "custodian" table where they'd write as compaction
and other background tasks were running, and this leads to bad things happening if the table
being written to can't be reached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message