phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4413) Possible queryserver memory leak when reusing connections and statements
Date Wed, 29 Nov 2017 16:59:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271095#comment-16271095
] 

Josh Elser commented on PHOENIX-4413:
-------------------------------------

Nope, I'd need some more information from [~alexaraujo] to better characterize the performance
and/or memory impact. Some general suggestions:

* Investigate tweaking PQS JVM properties. You may be hitting issues simply due to the impact
of the GC.
* Use jmap or similar to inspect the contents of the JVM heap to understand if the number
of live objects (especially connections and statements) are growing unboundedly
* Turn on DEBUG logging in PQS (I think). It might be at the TRACE level, but I know that
there is logging when the server evicts connections/statements automatically
* Use your profiler of choice (e.g. yourkit, java flight recorder) to better quantify what
changes in terms of performance.

Alternatively, if you are a C# felllow and not a Java person, encapsulating your scenario
into something that we can reproduce easily is helpful. I've done a little bit of work in
Avatica around self-contained docker images which might help isolate the problem: https://calcite.apache.org/avatica/docs/docker.html.
If there is a problem lurking, it's unlikely to be only seen with Phoenix.

Even in the case of misbehaving clients, PQS will clean up idle connections and statements.
see https://phoenix.apache.org/server.html, "Configurations relating to the server connection
cache." and "Configurations relating to the server statement cache.".

> Possible queryserver memory leak when reusing connections and statements
> ------------------------------------------------------------------------
>
>                 Key: PHOENIX-4413
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4413
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.12.0
>            Reporter: Alex Araujo
>
> While testing client-side connection pooling using the [C# client from Microsoft|https://github.com/Azure/hdinsight-phoenix-sharp],
we attempted to avoid creating new connections and statements for every Phoenix statement
execution (essentially just a simple SELECT for single key).  The results were very positive
from a performance perspective.  However, after a certain amount of statements executed in
this manner, memory on the PQS appears to spike, and performance degrades significantly.
> Steps to Recreate
> Setup
> 1) Create the table: "CREATE TABLE  <TableName> (TestKey varchar(255) PRIMARY KEY,
TestValue varchar(10000))".
> 2) Populate the table with 100 random TestKey and TestValue records.
> Execution (if done with one thread, this can take up to 24 hours, so we multithreaded
it)
> 1) Create connection using OpenConnectionRequestAsync.
> 2) Create statement using CreateStatementRequestAsync.
> 3) Loop n times, selecting a record with a single random key: "SELECT TestKey, TestValue
FROM <TableName> WHERE TestKey = <TestKey>'' issued using PrepareAndExecuteRequestAsync.
> 4) Close statement.
> 5) Close connection.
> Teardown
> 1) Drop the table.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message