hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-1219) More robust handling of metastore connection failures
Date Wed, 24 Mar 2010 17:31:27 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12849300#action_12849300
] 

Namit Jain commented on HIVE-1219:
----------------------------------

     [echo] Compiling: 
    [javac] Compiling 36 source files to /data/users/njain/hive_commit1/hive_commit1/build/metastore/classes
    [javac] /data/users/njain/hive_commit1/hive_commit1/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java:255:
getConf() is already defined in org.apache.hadoop.hive.metastore.HiveMetaStore.HMSHandler
    [javac]     private Configuration getConf() {
    [javac]                           ^
    [javac] Note: /data/users/njain/hive_commit1/hive_commit1/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
uses or overrides a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] 1 error



I get the following error

> More robust handling of metastore connection failures
> -----------------------------------------------------
>
>                 Key: HIVE-1219
>                 URL: https://issues.apache.org/jira/browse/HIVE-1219
>             Project: Hadoop Hive
>          Issue Type: New Feature
>          Components: Metastore
>            Reporter: Paul Yang
>            Assignee: Paul Yang
>             Fix For: 0.6.0
>
>         Attachments: HIVE-1219.1.patch, HIVE-1219.2.patch, HIVE-1219.3.patch, HIVE-1219.4.patch,
HIVE-1219.5.patch, HIVE-1219.6.patch, HIVE-1219.7.patch, HIVE-1219.8.patch
>
>
> Currently, if metastore's connection to the datastore is broken, the query fails and
the exception such as the following is thrown
> {code}
> 2010-01-28 11:50:20,885 ERROR exec.MoveTask (SessionState.java:printError(248)) - Failed
with exception Unable to fetch table tmp_table
> org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table tmp_table
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:362)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:333)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:112)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:99)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:64)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:582)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:462)
> at org.apache.hadoop.hive.ql.Driver.runCommand(Driver.java:324)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:312)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:123)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:181)
> at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:200)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:256)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: javax.jdo.JDODataStoreException: Communications link failure
> Last packet sent to the server was 1 ms ago.
> NestedThrowables:
> com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
> Last packet sent to the server was 1 ms ago.
> at org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:289)
> {code}
> In order to reduce the impact of transient network issues and momentarily unavailable
datastores, two possible improvements are:
> 1. Retrying the metastore command in case of connection failure before propagating up
the exception.
> 2. Retrieving the datastore hostname / connection URL through the use of an extension.
This extension would be useful in the case where a remote service maintained the location
of the currently available datastore. In case of hostname changes or failovers to a backup
datastore, the extension would allow hive clients to run without manual intervention.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message