ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmytro Sen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-13525) Exception in collector logs for JOIN queries
Date Thu, 22 Oct 2015 16:20:27 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14969384#comment-14969384
] 

Dmytro Sen commented on AMBARI-13525:
-------------------------------------

Fix:
1. Set value of phoenix.coprocessor.maxServerCacheTimeToLiveMs to 60 seconds instead of 30
by default (all metrics in METRIC_RECORD are updated every 60 seconds)
2. Catch DoNotRetryIOException when executing the JOIN query on ams collector, log like "The
cache might have expired and have been removed. Try to increase the cache size by setting
bigger value for phoenix.coprocessor.maxMetaDataCacheSize in ams-hbase-site config. Falling
back to sort-merge join algorithm." Run all the following JOIN queries with /+ USE_SORT_MERGE_JOIN
NO_CACHE/ hints, sort-merge join algorithm doesn't use the hash cache.


> Exception in collector logs for JOIN queries
> --------------------------------------------
>
>                 Key: AMBARI-13525
>                 URL: https://issues.apache.org/jira/browse/AMBARI-13525
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-metrics
>    Affects Versions: 2.1.1
>            Reporter: Dmytro Sen
>            Assignee: Dmytro Sen
>            Priority: Critical
>             Fix For: 2.1.3
>
>         Attachments: AMBARI-13525.patch
>
>
> Exception:
> {code}
> 04:17:00,487  WARN [1836615260@qtp-1368925364-4941] GenericExceptionHandler:98 - INTERNAL_SERVER_ERROR
> javax.ws.rs.WebApplicationException: org.apache.phoenix.exception.PhoenixIOException:
org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException:
Could not find hash cache for joinId: 8��^S1�(�. The cache might have expired and
have been removed.
>         at org.apache.phoenix.coprocessor.HashJoinRegionScanner.<init>(HashJoinRegionScanner.java:96)
>         at org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:196)
>         at org.apache.phoenix.coprocessor.BaseScannerRegionObserver.postScannerOpen(BaseScannerRegionObserver.java:144)
>         at org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postScannerOpen(RegionCoprocessorHost.java:1931)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3173)
>         at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29994)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2078)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
>         at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
>         at java.lang.Thread.run(Thread.java:745)
> {code}
> - Make sure that we follow best practices for JOINs:
> https://phoenix.apache.org/joins.html
> - Verify if this results in loss of functionality



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message