impala-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleksandr Baliev <aleksanderba...@gmail.com>
Subject Impala query (via JDBC) timeout
Date Tue, 02 Jan 2018 18:12:35 GMT
Hello,

I'm using impala in conjunction with CDH 5.10.1 and want to have timeouts
for long running queries. What I've found is:
1. Try to QUERY_TIMEOUT_S to close idle queries, as I understand when
client is closed and don't consume data.
2. for jdbc driver (version 2.5.37 or 4.1) there is setQueryTimeout(int).
which should set xlient side timeout as I understand (
https://www.cloudera.com/documentation/other/connectors/impala-jdbc/latest/Cloudera-JDBC-Driver-for-Impala-Install-Guide.pdf)
. It works, but:
2.1 when timeout occurs the query is in "waiting to be closed", even if
close Statement (for Java). And only if close Connection this query will we
removed from "in_flight_queries".
2.2 not sure, haven't checked fully, but seems this timeout can happen only
when query on earlier stage of executing, maybe when it's executing it's
okay, but when data is transmitting it cannot be timeout-ed.
3. Write some cron script to verify such queries via impalad /queries?json
and try to close them to do not consume resources.

To be honest I would prefer for now to have some external script which
could tell if there are any of such queries and react
manually/automatically + introduce some connection reconnect if there were
any of such timeouted queries on application level. Also maybe there are
some another situations when queries are nor closing, so I don't know maybe
just reconnect every 20 minutes :D But this solutions are just workaround
and I don't really can rely on them since they look a bit fragile.

So the question: is there any good way for the queries via jdbc to be
gracefully closed by given timeout ?

Will be very thankful for any hints :)
Sasha

Mime
View raw message