hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14521) codahale metrics exceptions
Date Fri, 12 Aug 2016 01:11:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15418214#comment-15418214
] 

Sergey Shelukhin commented on HIVE-14521:
-----------------------------------------

Actually, there aren't Gbs of these errors, there are just a few thousand... still, that's
a lot. I'll file a separate bug for Gbs ;)

> codahale metrics exceptions
> ---------------------------
>
>                 Key: HIVE-14521
>                 URL: https://issues.apache.org/jira/browse/HIVE-14521
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>
> One some random setup, I see bazillions of errors like this in HS2 log, Gb-s of logs
worth:
> {noformat}
> 2016-08-08 04:52:18,619 WARN  [HiveServer2-Handler-Pool: Thread-101]: log.PerfLogger
(PerfLogger.java:beginMetrics(226)) - Error recording metrics
> java.io.IOException: Scope named api_Driver.run is not closed, cannot be opened.
>         at org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics$CodahaleMetricsScope.open(CodahaleMetrics.java:133)
>         at org.apache.hadoop.hive.common.metrics.metrics2.CodahaleMetrics.startStoredScope(CodahaleMetrics.java:220)
>         at org.apache.hadoop.hive.ql.log.PerfLogger.beginMetrics(PerfLogger.java:223)
>         at org.apache.hadoop.hive.ql.log.PerfLogger.PerfLogBegin(PerfLogger.java:143)
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:378)
>         at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:320)
>         at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1214)
>         at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1208)
>         at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:146)
>         at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:226)
>         at org.apache.hive.service.cli.operation.Operation.run(Operation.java:276)
>         at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:468)
>         at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:456)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> {noformat}
> I suspect that either, just like the metastore deadline, this needs better error handling
when whatever the metrics surround fails; or, it is just not thread safe.
> But I actually haven't looked at the code yet.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message