hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Yang (JIRA)" <>
Subject [jira] Commented: (HIVE-1219) More robust handling of metastore connection failures
Date Wed, 17 Mar 2010 18:29:27 GMT


Paul Yang commented on HIVE-1219:

I think I am already doing that in this patch. In each metastore method, I create a Command
object that has the core logic in the run() method. In executeWithRetry(), I call
 All the retry logic is in executeWithRetry(). Prasad, can you clarify what you mean?

> More robust handling of metastore connection failures
> -----------------------------------------------------
>                 Key: HIVE-1219
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Metastore
>            Reporter: Paul Yang
>            Assignee: Paul Yang
>             Fix For: 0.6.0
>         Attachments: HIVE-1219.1.patch, HIVE-1219.2.patch, HIVE-1219.3.patch, HIVE-1219.4.patch
> Currently, if metastore's connection to the datastore is broken, the query fails and
the exception such as the following is thrown
> {code}
> 2010-01-28 11:50:20,885 ERROR exec.MoveTask ( - Failed
with exception Unable to fetch table tmp_table
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table tmp_table
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(
> at org.apache.hadoop.hive.ql.Driver.launchTask(
> at org.apache.hadoop.hive.ql.Driver.execute(
> at org.apache.hadoop.hive.ql.Driver.runCommand(
> at
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(
> at org.apache.hadoop.hive.cli.CliDriver.processLine(
> at org.apache.hadoop.hive.cli.CliDriver.processReader(
> at org.apache.hadoop.hive.cli.CliDriver.main(
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> at java.lang.reflect.Method.invoke(
> at org.apache.hadoop.util.RunJar.main(
> Caused by: javax.jdo.JDODataStoreException: Communications link failure
> Last packet sent to the server was 1 ms ago.
> NestedThrowables:
> com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
> Last packet sent to the server was 1 ms ago.
> at org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(
> {code}
> In order to reduce the impact of transient network issues and momentarily unavailable
datastores, two possible improvements are:
> 1. Retrying the metastore command in case of connection failure before propagating up
the exception.
> 2. Retrieving the datastore hostname / connection URL through the use of an extension.
This extension would be useful in the case where a remote service maintained the location
of the currently available datastore. In case of hostname changes or failovers to a backup
datastore, the extension would allow hive clients to run without manual intervention.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message