tinkerpop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephen Mallette (Jira)" <j...@apache.org>
Subject [jira] [Commented] (TINKERPOP-2205) Use one connection per request for Java client
Date Wed, 02 Oct 2019 18:30:00 GMT

    [ https://issues.apache.org/jira/browse/TINKERPOP-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943062#comment-16943062
] 

Stephen Mallette commented on TINKERPOP-2205:
---------------------------------------------

I've been running a utility on {{driver-35}} that tests different driver configurations. It
required some modifications to account for the configuration changes. I immediately hit two
problems:

1. Driver timed-out getting connections or 
2. Driver couldn't create a channel (i.e. socket)

The documentation seemed to cover the first issue with upgrading as it mentioned:

{quote}determine how many requests you anticipate to run in parallel from your client{quote}

but the following only made it into the PR description:

{quote}A client generating high TPS from a single machine will have to modify the OS setting
for max number of open files, since each connection corresponds to a single file in linux
OS.{quote}

I will add some documentation around that.  

Having now formally used {{driver-35}} and really poked about within it, I think I like that
the code relies on more standard netty stuff which is further supported in:

https://github.com/apache/tinkerpop/pull/1205

I also like the configuration simplicity that was promised in this change. However, I've not
yet been able to achieve the same "request per second" that I was getting before prior to
the changes, but I still may not have the best configuration to achieve that, so I'm still
trying. Another good point now is that I've passed millions of messages through the driver
at this point and it has behaved in a fairly stable way - there have been no odd errors or
noticeable problems - good sign.



> Use one connection per request for Java client
> ----------------------------------------------
>
>                 Key: TINKERPOP-2205
>                 URL: https://issues.apache.org/jira/browse/TINKERPOP-2205
>             Project: TinkerPop
>          Issue Type: Improvement
>          Components: driver
>    Affects Versions: 3.3.6
>            Reporter: Divij Vaidya
>            Assignee: Stephen Mallette
>            Priority: Major
>              Labels: deprecation
>
> This issue is a tracking item for the conversation in the mailing list [[1]|https://lists.apache.org/thread.html/77728cb77d4eab90f15680595e653ffc6055b74db29cbd4dcd5f0339@%3Cdev.tinkerpop.apache.org%3E]
which highlights multiple problems and shortcomings in the existing Java client and proposes
a design change in the client connection pooling to address the same. More specifically,
the problems addressed are as follows:
>  # Difficulty in configuring the client for optimum performance.
>  # Undocumented dependency of configuration parameters on each other.
>  # A bad request can impact other requests on the same channel.
>  # Host is marked as dead even if it is busy serving requests.
>  # No way to free up server resources if the client has stopped consuming results.
>  # No differentiation between retriable and non-retriable exceptions from the application
code.
>  # Keep alive is only sent when a query is executing, which means that a connection open
for a very long time with no query being sent will be closed by the server.
>  # Race condition if the server response reaches before result queue has been registered.
>  # Unpredictable behaviour if the server sends an exception followed by a genuine response
for the same request.
>  # A concurrent hash map (tracking pending requests) is a point of contention amongst
threads.
> [1]https://lists.apache.org/thread.html/77728cb77d4eab90f15680595e653ffc6055b74db29cbd4dcd5f0339@%3Cdev.tinkerpop.apache.org%3E



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message