livy-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-livy] wypoon opened a new pull request #284: [LIVY-752] Fix implementation of limits on connections.
Date Mon, 02 Mar 2020 03:10:39 GMT
wypoon opened a new pull request #284: [LIVY-752] Fix implementation of limits on connections.
URL: https://github.com/apache/incubator-livy/pull/284
 
 
   ## What changes were proposed in this pull request?
   
   `LivyThriftSessionManager` keeps a `ConcurrentHashMap[String, AtomicLong]` named `connectionsCount`
to track the number of connections per user, etc. The `incrementConnectionsCount` and `decrementConnectionsCount`
methods in `LivyThriftSessionManager` check that `connectionsCount` does not contain a key
(instead of contains the key) before getting the value and incrementing or decrementing the
count (leading to a `NullPointerException`). Even accounting for the incorrect condition,
they do not use the `ConcurrentHashMap` correctly. There is a race -- a thread can get a count,
find that it's within a limit, create a new session and then increment the count, while in
the meantime, another thread could have incremented the count and so the limit is now actually
exceeded.
   
   We increment all relevant counts optimistically before creating a new session, check if
any limits are violated, and if so, decrement all incremented counts.
   
   ## How was this patch tested?
   
   Tested by deploying the change on a cluster and setting livy.server.thrift.limit.connections.per.user.
Verified that the number of connections reaches but does not exceed the limit.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message