Mailing-List: contact user-help@impala.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@impala.apache.org
MIME-Version: 1.0
From: Oleksandr Baliev <aleksanderbalev@gmail.com>
Date: Tue, 2 Jan 2018 19:12:35 +0100
Message-ID: <CAHa2aWZL5D=RfUxbS6hmCN32+kW+Gk60BY1TKs1WxVReeyuESw@mail.gmail.com>
Subject: Impala query (via JDBC) timeout
To: user@impala.apache.org
Content-Type: multipart/alternative; boundary="001a11428f0a1f25a30561cf0b61"
archived-at: Tue, 02 Jan 2018 18:12:40 -0000

--001a11428f0a1f25a30561cf0b61
Content-Type: text/plain; charset="UTF-8"

Hello,

I'm using impala in conjunction with CDH 5.10.1 and want to have timeouts
for long running queries. What I've found is:
1. Try to QUERY_TIMEOUT_S to close idle queries, as I understand when
client is closed and don't consume data.
2. for jdbc driver (version 2.5.37 or 4.1) there is setQueryTimeout(int).
which should set xlient side timeout as I understand (
https://www.cloudera.com/documentation/other/connectors/impala-jdbc/latest/Cloudera-JDBC-Driver-for-Impala-Install-Guide.pdf)
. It works, but:
2.1 when timeout occurs the query is in "waiting to be closed", even if
close Statement (for Java). And only if close Connection this query will we
removed from "in_flight_queries".
2.2 not sure, haven't checked fully, but seems this timeout can happen only
when query on earlier stage of executing, maybe when it's executing it's
okay, but when data is transmitting it cannot be timeout-ed.
3. Write some cron script to verify such queries via impalad /queries?json
and try to close them to do not consume resources.

To be honest I would prefer for now to have some external script which
could tell if there are any of such queries and react
manually/automatically + introduce some connection reconnect if there were
any of such timeouted queries on application level. Also maybe there are
some another situations when queries are nor closing, so I don't know maybe
just reconnect every 20 minutes :D But this solutions are just workaround
and I don't really can rely on them since they look a bit fragile.

So the question: is there any good way for the queries via jdbc to be
gracefully closed by given timeout ?

Will be very thankful for any hints :)
Sasha

--001a11428f0a1f25a30561cf0b61
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hello,=C2=A0<div><br></div><div>I&#39;m using impala in co=
njunction with CDH 5.10.1 and want to have timeouts for long running querie=
s. What I&#39;ve found is:</div><div>1. Try to QUERY_TIMEOUT_S to close idl=
e queries, as I understand when client is closed and don&#39;t consume data=
.=C2=A0</div><div>2. for jdbc driver (version 2.5.37 or 4.1) there is setQu=
eryTimeout(int). which should set xlient side timeout as I understand (<a h=
ref=3D"https://www.cloudera.com/documentation/other/connectors/impala-jdbc/=
latest/Cloudera-JDBC-Driver-for-Impala-Install-Guide.pdf">https://www.cloud=
era.com/documentation/other/connectors/impala-jdbc/latest/Cloudera-JDBC-Dri=
ver-for-Impala-Install-Guide.pdf</a>)=C2=A0 . It works, but:</div><div>2.1 =
when timeout occurs the query is in &quot;waiting to be closed&quot;, even =
if close Statement (for Java). And only if close Connection this query will=
 we removed from &quot;<span style=3D"color:rgb(0,0,0);white-space:pre-wrap=
">in_flight_queries</span>&quot;.=C2=A0</div><div>2.2 not sure, haven&#39;t=
 checked fully, but seems this timeout can happen only when query on earlie=
r stage of executing, maybe when it&#39;s executing it&#39;s okay, but when=
 data is transmitting it cannot be timeout-ed.=C2=A0</div><div>3. Write som=
e cron script to verify such queries via impalad /queries?json and try to c=
lose them to do not consume resources.</div><div><br></div><div>To be hones=
t I would prefer for now to have some external script which could tell if t=
here are any of such queries and react manually/automatically + introduce s=
ome connection reconnect if there were any of such timeouted queries on app=
lication level. Also maybe there are some another situations when queries a=
re nor closing, so I don&#39;t know maybe just reconnect every 20 minutes :=
D But this solutions are just workaround and I don&#39;t really can rely on=
 them since they look a bit fragile.</div><div><br></div><div>So the questi=
on: is there any good way for the queries via jdbc to be gracefully closed =
by given timeout ?=C2=A0</div><div><br></div><div>Will be very thankful for=
 any hints :)=C2=A0</div><div>Sasha</div><div><br></div><div><br></div></di=
v>

--001a11428f0a1f25a30561cf0b61--