phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Samarth Jain (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-1452) Add Phoenix client-side logging and capture resource utilization metrics
Date Thu, 26 Feb 2015 22:44:04 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14339329#comment-14339329
] 

Samarth Jain edited comment on PHOENIX-1452 at 2/26/15 10:43 PM:
-----------------------------------------------------------------

Thanks for the feedback [~jfernando_sfdc]. I will proceed with this approach and see if there
are any other metrics that should be added. 
I will make the strictCovariance mode configurable with the default value of false. 

About logging the thread pool information - by exposing the number of tasks/scans submitted
to the pool and query times and count, you would be getting a fare idea of the pool contention.
So I am not sure if instrumenting each and every task and capturing the amount of time it
took to be picked up by a thread in the pool would provide us more information. What do you
think [~jfernando_sfdc]. 

[~jamestaylor] - would be interesting to get your feedback too. 


was (Author: samarthjain):
Thanks for the feedback [~jfernando_sfdc]. I will proceed with this approach and see if there
are any other metrics that should be added. 
I will make the strictCovariance mode configurable with the default value of false. 

About logging the thread pool information - by exposing the number of tasks/scans submitted
to the pool as well as the over all time it took for the query to complete (and the number
of query count), you would be getting a fare idea of the pool contention. So I am not sure
if instrumenting each and every task and capturing the amount of time it took to be picked
up by a thread in the pool would provide us more information. What do you think [~jfernando_sfdc].


[~jamestaylor] - would be interesting to get your feedback too. 

> Add Phoenix client-side logging and capture resource utilization metrics
> ------------------------------------------------------------------------
>
>                 Key: PHOENIX-1452
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1452
>             Project: Phoenix
>          Issue Type: Improvement
>    Affects Versions: 5.0.0, 4.2
>            Reporter: Jan Fernando
>            Assignee: Samarth Jain
>         Attachments: wip.patch
>
>
> For performance testing and tuning of features that use Phoenix and for production monitoring
it would be really helpful to easily be able to extract statistics about Phoenix's client-side
Thread Pool and Queue Depth usage to help with tuning and being able to correlate the impact
of tuning these 2 parameters to query performance.
> For global per JVM logging one of the following would meet my needs, with a preference
for #2:
> 1. A simple log line that that logs the data in ThreadPoolExecutor.toString() at a configurable
interval
> 2. Exposing the ThreadPoolExecutor metrics in PhoenixRuntime or other global client exposed
class and allow client to do their own logging.
> In addition to this it would also be really valuable to have a single log line per query
that provides statistics about the level of parallelism i.e. number of parallel scans being
executed. I don't full explain plan level of data but a good heuristic to be able to track
over time how queries are utilizing the thread pool as data size grows etc. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message