hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas Nair" <the...@hortonworks.com>
Subject Re: Review Request 14719: Better thread management for HiveServer2 async threads
Date Thu, 24 Oct 2013 08:43:22 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14719/#review27449
-----------------------------------------------------------



service/src/java/org/apache/hive/service/cli/session/SessionManager.java
<https://reviews.apache.org/r/14719/#comment53340>

    The timeout is applicable only if the number of threads are more than corePoolSize. But
since corePoolSize==maxPoolSize, it won't have any impact. 
    
    I think we can set the max to configurable value (we should change the config param name
to hive.server2.async.exec.max.threads and use the old config name as a deprecated alias).
Also add a config param for min, and use that as the value for corePoolSize. 
    The default value of timeout I think can be in minutes (say 3 mins?), which is more representative
of longer gaps that might be there between queries that come from a users session.  How about
using a max default of 200 and a min of 5 .
    
    Also, I think we should also use a bounded queue. It is better to give an error that makes
sense, rather than wait for running out of resources and giving a hard to decipher error.
We should also handle rejections from the thread pool, when the limit has been hit.


- Thejas Nair


On Oct. 17, 2013, 8:32 p.m., Vaibhav Gumashta wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14719/
> -----------------------------------------------------------
> 
> (Updated Oct. 17, 2013, 8:32 p.m.)
> 
> 
> Review request for hive and Thejas Nair.
> 
> 
> Bugs: HIVE-5229
>     https://issues.apache.org/jira/browse/HIVE-5229
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Improves the current strategy by not keeping all the async threads alive when there are
no corresponding requests. The async threads terminate after a configurable keepalive time
if there are no new requests to handle.
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java fb3570b 
>   conf/hive-default.xml.template b7234b4 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java f392d62 
> 
> Diff: https://reviews.apache.org/r/14719/diff/
> 
> 
> Testing
> -------
> 
> TestEmbeddedThriftBinaryCLIService, TestThriftBinaryCLIService, TestJdbcDriver2
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message