From dev-return-57176-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Sat Jun 22 06:19:02 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1223218076D for ; Sat, 22 Jun 2019 08:19:01 +0200 (CEST) Received: (qmail 79131 invoked by uid 500); 22 Jun 2019 06:19:01 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 79015 invoked by uid 99); 22 Jun 2019 06:19:01 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 22 Jun 2019 06:19:01 +0000 Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 6341BE2CFA for ; Sat, 22 Jun 2019 06:19:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 234D62544A for ; Sat, 22 Jun 2019 06:19:00 +0000 (UTC) Date: Sat, 22 Jun 2019 06:19:00 +0000 (UTC) From: "Lars Hofhansl (JIRA)" To: dev@phoenix.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (PHOENIX-5364) Tests fail "Queue Full" only when run in parallel with other tests. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PHOENIX-5364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated PHOENIX-5364: ----------------------------------- Fix Version/s: 5.1.0 4.15.0 > Tests fail "Queue Full" only when run in parallel with other tests. > ------------------------------------------------------------------- > > Key: PHOENIX-5364 > URL: https://issues.apache.org/jira/browse/PHOENIX-5364 > Project: Phoenix > Issue Type: Test > Reporter: Lars Hofhansl > Assignee: Lars Hofhansl > Priority: Major > Fix For: 4.15.0, 5.1.0 > > Attachments: 5364.txt > > > Spooling defaults to true. Yet, I see some tests failing (sometimes) with exceptions like this: > {code} > org.apache.phoenix.exception.PhoenixIOException: org.apache.hadoop.hbase.DoNotRetryIOException: N000531,00D300000000XHP00A\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1560900587336.ca5a6d7b1f0951e30d15567c3aa909b1.: Queue full. Consider increasing memory threshold or spooling to disk. Max size: 1048576, Current size: 77, Number of elements:1 > at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:121) > at org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:87) > at org.apache.phoenix.iterate.NonAggregateRegionScannerFactory.getTopNScanner(NonAggregateRegionScannerFactory.java:357) > at org.apache.phoenix.iterate.NonAggregateRegionScannerFactory.getRegionScanner(NonAggregateRegionScannerFactory.java:182) > at org.apache.phoenix.coprocessor.ScanRegionObserver.doPostScannerOpen(ScanRegionObserver.java:187) > at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.overrideDelegate(BaseScannerRegionObserver.java:251) > at org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:299) > at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2904) > at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3153) > at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:36613) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2380) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:124) > at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:297) > at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:277) > Caused by: java.lang.IllegalStateException: Queue full. Consider increasing memory threshold or spooling to disk. Max size: 1048576, Current size: 77, Number of elements:1 > at org.apache.phoenix.iterate.SizeBoundQueue.add(SizeBoundQueue.java:58) > at org.apache.phoenix.iterate.OrderedResultIterator.getResultIterator(OrderedResultIterator.java:315) > at org.apache.phoenix.iterate.OrderedResultIterator.next(OrderedResultIterator.java:245) > at org.apache.phoenix.iterate.NonAggregateRegionScannerFactory.getTopNScanner(NonAggregateRegionScannerFactory.java:352) > ... 11 more > {code} > This should never happen. > However, I found two more tests that set phoenix.query.server.orderBy.spooling.enabled to false: > * OrderByWithServerClientSpoolingDisabledIT, and > * OrderByWithServerMemoryLimitIT > They need to run in their own cluster. > (Also, OrderByWithServerMemoryLimitIT seems to not run at all (it does not show in the logs of the jenkins run... Perhaps because it's a direct child of BaseTest...?)) -- This message was sent by Atlassian JIRA (v7.6.3#76005)