Mailing-List: contact hive-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hive-dev@hadoop.apache.org
Message-ID: <1672728557.1237486970548.JavaMail.jira@brutus>
Date: Thu, 19 Mar 2009 11:22:50 -0700 (PDT)
From: "Namit Jain (JIRA)" <jira@apache.org>
To: hive-dev@hadoop.apache.org
Subject: [jira] Commented: (HIVE-316) External table definitions should be
 allowed outside of Warehouse Filesystem
In-Reply-To: <781804289.1235778195684.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HIVE-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683554#action_12683554 ] 

Namit Jain commented on HIVE-316:
---------------------------------

While I was running tests for merging, I got the following error:

Note that this happened while running tests for the second time. I have not debugged it, but I suspect it might have something to do with the directory already present (created the first time).

    [junit] Begin query: external_table_join.q
    [junit] Loading data to table srcpart partition {ds=2008-04-08, hr=11}
    [junit] OK
    [junit] Loading data to table srcpart partition {ds=2008-04-08, hr=12}
    [junit] OK
    [junit] Loading data to table srcpart partition {ds=2008-04-09, hr=11}
    [junit] OK
    [junit] Loading data to table srcpart partition {ds=2008-04-09, hr=12}
    [junit] OK
    [junit] Loading data to table srcbucket
    [junit] OK
    [junit] Loading data to table srcbucket
    [junit] OK
    [junit] Loading data to table src
    [junit] OK
    [junit] Exception: Client Execution failed with error code = 9
    [junit] junit.framework.AssertionFailedError: Client Execution failed with error code = 9
    [junit] 	at junit.framework.Assert.fail(Assert.java:47)
    [junit] 	at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_external_table_join(TestCliDriver.java:705)
    [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    [junit] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    [junit] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    [junit] 	at java.lang.reflect.Method.invoke(Method.java:597)
    [junit] 	at junit.framework.TestCase.runTest(TestCase.java:154)
    [junit] 	at junit.framework.TestCase.runBare(TestCase.java:127)
    [junit] 	at junit.framework.TestResult$1.protect(TestResult.java:106)
    [junit] 	at junit.framework.TestResult.runProtected(TestResult.java:124)
    [junit] 	at junit.framework.TestResult.run(TestResult.java:109)
    [junit] 	at junit.framework.TestCase.run(TestCase.java:118)
    [junit] 	at junit.framework.TestSuite.runTest(TestSuite.java:208)
    [junit] 	at junit.framework.TestSuite.run(TestSuite.java:203)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:297)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:672)
    [junit] 	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:567)
    [junit] Hive history file=/data/users/njain/hive_commit/trunk/ql/../build/ql/tmp/hive_job_log_njain_200903181609_1260283057.txt

> External table definitions should be allowed outside of Warehouse Filesystem
> ----------------------------------------------------------------------------
>
>                 Key: HIVE-316
>                 URL: https://issues.apache.org/jira/browse/HIVE-316
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Metastore
>            Reporter: Richard Lee
>            Assignee: Richard Lee
>         Attachments: external_table1.q, external_table1.q.out, external_table_join.q, hive-external-filesystems.diff, hive-external-filesystems2.diff, hive-external-filesystems3.diff, hive-external-filesystems5.diff
>
>
> I have a situation where I have hive's datastore pointed at an hdfs, but would like to create an external table on data accessable from an outside data storage solution exported via nfs.  
> Presently, Warehouse.java aggregates only a single FileSystem object which limits all tables, both internal and external to being relative to the URl specified in the hive configuration.  I feel like the Warehouse code should prefer to use the configured warehouse URI for non-absolute Paths, but honor paths outside of the Warehouse; particularly when they are defined in external tables.
> I was going to implement this by adding a Map of FileSystem objects to the Warehouse object.  This map gets populated with FileSystem objects when operations cannot be performed by either the warehouse FS, or any other FS object in the map.  I am not sure what impact this change would have on hive overall... or if this is the only place that this change would need to be made.
> Please advise.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.