hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Lee (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-316) External table definitions should be allowed outside of Warehouse Filesystem
Date Mon, 23 Mar 2009 18:13:50 GMT

    [ https://issues.apache.org/jira/browse/HIVE-316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12688363#action_12688363
] 

Richard Lee commented on HIVE-316:
----------------------------------

So the hard coded path seems to be a red herring. 

in test-common.xml there are two modes in which the junit tests are run... If a specific class
is specified, the test passes.  If no class is specified, all tests are run and the test fails.
 Looking at the resulting .out file, I see that it was unable to figure out how to handle
tmpfs:// ... so the SET fs.testfs.impl = X at the top of the .q file was ignored.

It looks like some form of race condition on the job conf around setting file system handlers...
I don't think that this problem is specific to my test case.




> External table definitions should be allowed outside of Warehouse Filesystem
> ----------------------------------------------------------------------------
>
>                 Key: HIVE-316
>                 URL: https://issues.apache.org/jira/browse/HIVE-316
>             Project: Hadoop Hive
>          Issue Type: Improvement
>          Components: Metastore
>            Reporter: Richard Lee
>            Assignee: Richard Lee
>         Attachments: external_table1.q, external_table1.q.out, external_table_join.q,
hive-external-filesystems.diff, hive-external-filesystems2.diff, hive-external-filesystems3.diff,
hive-external-filesystems5.diff
>
>
> I have a situation where I have hive's datastore pointed at an hdfs, but would like to
create an external table on data accessable from an outside data storage solution exported
via nfs.  
> Presently, Warehouse.java aggregates only a single FileSystem object which limits all
tables, both internal and external to being relative to the URl specified in the hive configuration.
 I feel like the Warehouse code should prefer to use the configured warehouse URI for non-absolute
Paths, but honor paths outside of the Warehouse; particularly when they are defined in external
tables.
> I was going to implement this by adding a Map of FileSystem objects to the Warehouse
object.  This map gets populated with FileSystem objects when operations cannot be performed
by either the warehouse FS, or any other FS object in the map.  I am not sure what impact
this change would have on hive overall... or if this is the only place that this change would
need to be made.
> Please advise.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message