hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <>
Subject Re: Hive Metastore Bottleneck
Date Thu, 31 Mar 2016 05:41:23 GMT
Is the MySQL database virtualized? Bottlenecks to storage of the MySQL database? Network could
be a bottleneck? Firewalls blocking new connections in case of a sudden connection increase?

> On 30 Mar 2016, at 23:28, Udit Mehta <> wrote:
> Hi all,
> We are currently running Hive in production and staging with the metastore connecting
to a MySql database in the backend. The traffic in production accessing the metastore is more
than staging which is expected. We have had a sudden increase in traffic which has led to
the metastore operation taking a lot longer than before. The same query on staging takes a
lot less due to the lesser traffic on the staging cluster.
> We tried increasing the heap space for the metastore process as well as bumped up the
memory for the mysql database. Both these changes did not seem to help much and we still see
delays. Is there any other config we can increase to counter this increased traffic? I am
looking at config for max threads as well but im not sure if this is the right path ahead.
> Im wondering if the metastore is a bottleneck here or im missing something.
> Looking forward to your reply,
> Udit

View raw message