drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venki Korukanti (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (DRILL-2034) Use the correct schema when creating FileSystem object in HiveScan
Date Mon, 19 Jan 2015 19:05:34 GMT

    [ https://issues.apache.org/jira/browse/DRILL-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14282872#comment-14282872
] 

Venki Korukanti edited comment on DRILL-2034 at 1/19/15 7:05 PM:
-----------------------------------------------------------------

Currently all hive test tables use local FS for storage. We need one more storage (such as
MiniHDFS) or update "fs.default.name" in Hive storage plugin config before the test to non-local
FS. I will add a test using the latter approach. 


was (Author: vkorukanti):
Currently all hive test tables use local FS for storage. We need one more storage (such as
MiniHDFS) or update "fs.default.name" in Hive storage plugin config before the test to non-local
FS. I will add a test using the later approach. 

> Use the correct schema when creating FileSystem object in HiveScan
> ------------------------------------------------------------------
>
>                 Key: DRILL-2034
>                 URL: https://issues.apache.org/jira/browse/DRILL-2034
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Hive
>            Reporter: Venki Korukanti
>            Assignee: Venki Korukanti
>             Fix For: 0.8.0
>
>         Attachments: 0001-DRILL-2034-Use-the-correct-schema-when-creating-File.patch
>
>
> We currently get the FileSystem object using FileSystem.get(JobConf) which always returns
the FileSystem implementation of default FS setting in conf. This causes problem when Hive
MetaStore has tables stored in different storages. For example one table has data on HDFS
and other on S3 filesystem. 
> {code}
>         Path path = new Path(table.getSd().getLocation());
>         FileSystem fs = FileSystem.get(job);
>         if (fs.exists(path)) {
>            ....
> {code}
> We should call Path.getFileSystem() which takes the URI schema from the path.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message