hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Johannes Alberti <johan...@altiscale.com>
Subject Re: For Apache Hive HS2 , what is the largest heap size setting that works well?
Date Tue, 28 Nov 2017 22:19:55 GMT
Hi Sharanya,

Can you share your current GC settings and Java version. Are you using Java 8/9 w/ G1 already?

Regards,

Johannes

Sent from my iPhone

> On Nov 28, 2017, at 12:57 PM, Sharanya Santhanam <ssanthan.oss@gmail.com> wrote:
> 
> Hello , 
> 
> I am currently trying to upgrade hive version on our prod clusters form V1.2 to v2.1

> We also want to adopt HS2 on the new upgraded cluster. Earlier all queries were submitted
via Hive cli. 
> 
> Would like to understand how large a single HS2 Heap size can be ? And is there any formula
to figure out the how many concurrent sessions I can support with this particular heap setting?

> 
> 
> We currently have a upper limit of supporting 300 concurrent sessions ( hive.server2.thrift.max.worker.threads=300).
Based on this we set the max  heap size to 70 GB , but seeing many long GC pauses. 
> 
> 
> Would like to understand what is the industry standard for max HS2 Heap size. Are there
any recommendations on what JMV GC setting work best for supporting  such high number of concurrent
sessions? 
> 
> Thanks,
> Sharanya 

Mime
View raw message