From dev-return-55499-archive-asf-public=cust-asf.ponee.io@phoenix.apache.org Wed Feb 20 22:50:22 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id ABE0618075F for ; Wed, 20 Feb 2019 23:50:21 +0100 (CET) Received: (qmail 13594 invoked by uid 500); 20 Feb 2019 22:50:20 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 13580 invoked by uid 99); 20 Feb 2019 22:50:20 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Feb 2019 22:50:20 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id A249318AF9A for ; Wed, 20 Feb 2019 22:50:19 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.8 X-Spam-Level: * X-Spam-Status: No, score=1.8 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=23andme.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id WudfOSO_5-6S for ; Wed, 20 Feb 2019 22:50:17 +0000 (UTC) Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 696005F476 for ; Wed, 20 Feb 2019 22:50:16 +0000 (UTC) Received: by mail-pf1-f174.google.com with SMTP id g6so12657546pfh.13 for ; Wed, 20 Feb 2019 14:50:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:date:to; bh=mLuIQ3o7yfNljX0I5bVQe6Qfbpr9pyx17JLTrWZ86u0=; b=jT1wuRWwSN0+OPx+MyY7DtCUead7uZIwk4+FNfA4VVUUoTLaolS0hSR1Kzp2awJ2Ms 8S837PJVxXC0CDyCQdmQUR9R9CCYaH4R99kIoqd65aGI2oJZQBuJroXhlyouSR8DCZ4I kuSrIc+kxp6RMhr8s8lSWX6OpvlKcTO0QHHVgq8bGNyrNDq6t4cQ7n/d/zB98VzY7xWO kDDMqopZXhwcZMTNv8IoSZhoO7zMSAfQxGbjqG5aZs9ybvSw0fTEroevoHEUioaDjjy8 rz/jEX/afGZustis2wnFm6ihvo80fmcdL6IdlN9D4J8/G6F9pvaTPCz5naLwCYImviv+ g2Nw== X-Gm-Message-State: AHQUAuY4NyoxqhuLQGbQMIUigazbizQPpinBL7+xp9btw7xKljGBO3/u Etu0J4hxIilaXHZlniQGluhmmVvv3eo= X-Google-Smtp-Source: AHgI3IZRSmc1oC7+fA2anqg58yzUEQfYRa0W6qJ17BTl3hicr3NnRNHn1RSjaUduQZcfePOgk/Xf2w== X-Received: by 2002:a63:1544:: with SMTP id 4mr31884710pgv.290.1550703008775; Wed, 20 Feb 2019 14:50:08 -0800 (PST) Received: from user003.vpnuser.23andme.net (fw41.rtr-fw.23andme.net. [64.74.136.252]) by smtp.gmail.com with ESMTPSA id a15sm33730474pgd.4.2019.02.20.14.50.07 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 20 Feb 2019 14:50:07 -0800 (PST) From: Xiaoxiao Wang Content-Type: multipart/alternative; boundary="Apple-Mail=_6FCAFF70-5895-4636-8E90-4CE9E846B7E7" Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: How Phoenix JDBC connection get hbase configuration Message-Id: Date: Wed, 20 Feb 2019 14:50:06 -0800 To: dev@phoenix.apache.org X-Mailer: Apple Mail (2.3445.102.3) --Apple-Mail=_6FCAFF70-5895-4636-8E90-4CE9E846B7E7 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, who may help We are running a Hadoop application that needs to use phoenix JDBC = connection from the workers.=20 The connection works, but when too many connection established at the = same time, it throws RPC timeouts Error: java.io.IOException: = org.apache.phoenix.exception.PhoenixIOException: Failed after = attempts=3D36, exceptions: Wed Feb 20 20:02:43 UTC 2019, null, = java.net.SocketTimeoutException: callTimeout=3D60000, = callDuration=3D60506. ... So we have figured we should probably set a higher hbase.rpc.timeout = value, but then it comes to the issue: A little bit background on how we run the application Here is how we get PhoenixConnection from java program DriverManager.getConnection("jdbc:phoenix:host", props) And we trigger the program by using=20 hadoop jar $test_jar We have tried multiple approaches to load hbase/phoenix configuration, = but none of them get respected by PhoenixConnection, here are the = methods we tried=20 * Pass hbase_conf_dir through HADOOP_CLASSPATH, so run the hadoop = application like HADOOP_CLASSPATH=3D/etc/hbase/conf/ hadoop jar = $test_jar . However, PhoenixConnection doesn=E2=80=99t respect the = parameters * Tried passing -Dhbase.rpc.timeout=3D1800, which is picked up by hbase = conf object, but not PhoniexConnection * Explicitly set those parameters and pass them to the PhoenixConnection=20= props.setProperty("hbase.rpc.timeout", "1800"); props.setProperty(=E2=80=9Cphoenix.query.timeoutMs", "1800"); Also didn=E2=80=99t get respected by PhoenixConnection * also tried what is suggested by phoenix here = https://phoenix.apache.org/#connStr , use :longRunning together with = those properties, still didn=E2=80=99t seem to work Besides all those approaches we tried, I have explicitly output those = parameters we care from the connection,=20 connection.getQueryServices().getProps()=20 The default values I got are 60000 for hbase.rpc.timeout, and 600k for = phoenix.query.timeoutMs , so I have tried to run a query lthat would run = longer than 10 mins, Ideally it should timeout, however, it runs over 20 = mins and didn=E2=80=99t timeout. So I=E2=80=99m wondering how = PhoenixConnection respect those properties? So with some of your help, we=E2=80=99d like to know if there=E2=80=99s = any thing wrong with our approaches. And we=E2=80=99d like to get rid of = those SocketTimeExceptions. We are using phoenix-core version is 4.7.0-clabs-phoenix1.3.0 , and our = phoenix-client version is phoenix-4.7.0-clabs-phoenix1.3.0.23 (we have = tried phoenix-4.14.0-HBase-1.3 as well, which didn=E2=80=99t work = either). Thanks for your time --Apple-Mail=_6FCAFF70-5895-4636-8E90-4CE9E846B7E7--