hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nagarjuna Vissarapu <>
Subject Re: dynamic partitioning issues in hive 0.13.0
Date Tue, 02 Dec 2014 02:11:45 GMT
Can you please check you mysql service which stores metada
On Dec 2, 2014 4:18 AM, "Viral Bajaria" <> wrote:

> Hi,
> I have been running into 2 issues with dynamic partitioning queries.
> The query runs fine and I get errors like:
> *Failed with exception
> MetaException(message:javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MPartition@5bbd373e" using
> VALUES (?,?,?,?,?,?)" failed : Deadlock found when trying to get lock; try
> restarting transaction*
> Detailed exception:
> Why would there be a deadlock on adding a new partition info to the
> metastore ? It seems weird that would even happen given that the partition
> did not even exist prior to this query run.
> I am thinking it could be an issue because I have 100s of such queries
> running and the INDEX on that table might be causing the issue ?
> The 2nd issue is more painful. After the 1st error mentioned above, I have
> a process which figures out what failed and then re-runs the data for that
> month. After the query finishes successfully, it ends up adding only 1
> partition instead of partition for each day in that month. If I run the
> query manually and look at the data returned then it returns data for all
> partitions, yet for some reason dynamic partition does not ad all. The
> query is an "INSERT OVERWRITE" query.
> In the end I have to drop all partitions for that month before re-running
> the query and then everything works fine. Without that the partitions that
> were missed on first attempt don't show up.
> The missing partitions are in itself a mystery.
> Any points on how to debug this ?
> Thanks,
> Viral

View raw message