phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karan Mehta (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-3062) JMXCacheBuster restarting the metrics system causes PhoenixTracingEndToEndIT to hang
Date Fri, 17 Mar 2017 20:38:41 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15930674#comment-15930674
] 

Karan Mehta edited comment on PHOENIX-3062 at 3/17/17 8:38 PM:
---------------------------------------------------------------

Here's the way I am trying to implement this. 
The {{TraceMetricSouce}} receives the span and dumps it into a {{BlockingQueue}}, whose reference
is used by another new class, {{TraceWriter}}, equivalent to the {{PhoenixMetricsSink}} class.
This class can be configured to launch appropriate number of threads, each of which would
get the data from the queue and add it to the mutation started by the Connection in that thread.
After a threshold number of spans are read, it will commit the batch. The threads are instantiated
via ScheduledThreadPool and so the timing between threads can be configured. 
{code}
        private Connection conn;
        private String tableName;
        private int counter = 0;
        public void run() {
                while (!spanQueue.isEmpty()) {
                Metric metric = spanQueue.poll();
                if (null == metric) break;
                System.out.println("Metric Rcvd Id: " + metric.info.name());
                addToBatch(metric);
                counter++;
                if (counter >= BATCH_SIZE) {
                    commitBatch(conn);
                    counter = 0;
                }
            }
        }

{code}
This also involves making changes to the classes that handle the span data such as {code}
Metric {code} and its parent classes, so that the complete dependency can be removed.



was (Author: karanmehta93):
Here's the way I am trying to implement this. 
The {{monospaced}} TraceMetricSouce {{monospaced}} receives the span and dumps it into a {{monospaced}}
BlockingQueue {{monospaced}}, whose reference is used by another new class, {{monospaced}}
TraceWriter {{monospaced}}, equivalent to the {{monospaced}} PhoenixMetricsSink {{monospaced}}
class. This class can be configured to launch appropriate number of threads, each of which
would get the data from the queue and add it to the mutation started by the Connection in
that thread. After a threshold number of spans are read, it will commit the batch. The threads
are instantiated via ScheduledThreadPool and so the timing between threads can be configured.

{code}
        private Connection conn;
        private String tableName;
        private int counter = 0;
        public void run() {
                while (!spanQueue.isEmpty()) {
                Metric metric = spanQueue.poll();
                if (null == metric) break;
                System.out.println("Metric Rcvd Id: " + metric.info.name());
                addToBatch(metric);
                counter++;
                if (counter >= BATCH_SIZE) {
                    commitBatch(conn);
                    counter = 0;
                }
            }
        }

{code}
This also involves making changes to the classes that handle the span data such as {code}
Metric {code} and its parent classes, so that the complete dependency can be removed.


> JMXCacheBuster restarting the metrics system causes PhoenixTracingEndToEndIT to hang
> ------------------------------------------------------------------------------------
>
>                 Key: PHOENIX-3062
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3062
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Enis Soztutar
>            Assignee: Karan Mehta
>             Fix For: 4.11.0
>
>         Attachments: phoenix-3062_v1.patch
>
>
> With some recent fixes in the hbase metrics system, we are now affectively restarting
the metrics system (in HBase-1.3.0, probably not affecting 1.2.0). Since we use a custom sink
in the PhoenixTracingEndToEndIT, restarting the metrics system loses the registered sink thus
causing a hang. 
> We need a fix in HBase, and Phoenix so that we will not restart the metrics during tests.

> Thanks to [~sergey.soldatov] for analyzing the initial root cause of the hang. 
> See HBASE-14166 and others. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message