hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15203) Hive export command does export to non HDFS file system
Date Tue, 15 Nov 2016 06:58:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666316#comment-15666316
] 

Thejas M Nair commented on HIVE-15203:
--------------------------------------

Thanks to [~cnauroth] for analysis of the issue -

It looks like exports run some logic that hard-codes the URI scheme to "hdfs", but then pulls
the authority (host) from the default file system:

https://github.com/apache/hive/blob/rel/release-2.1.0/ql/src/java/org/apache/hadoop/hive/ql/parse/ExportSemanticAnalyzer.java#L68
https://github.com/apache/hive/blob/rel/release-2.1.0/ql/src/java/org/apache/hadoop/hive/ql/parse/EximUtil.java#L94-L99
{code}
      // set correct scheme and authority
      if (StringUtils.isEmpty(scheme)) {
        if (testMode) {
          scheme = "pfile";
        } else {
          scheme = "hdfs";
        }
      }

      // if scheme is specified but not authority then use the default
      // authority
      if (StringUtils.isEmpty(authority)) {
        URI defaultURI = FileSystem.get(conf).getUri();
        authority = defaultURI.getAuthority();
      }
{code}

I think that's our problem.  This logic is definitely wrong for the case of exporting to any
file system that is not HDFS.

> Hive export command does export to non HDFS file system
> -------------------------------------------------------
>
>                 Key: HIVE-15203
>                 URL: https://issues.apache.org/jira/browse/HIVE-15203
>             Project: Hive
>          Issue Type: Bug
>          Components: repl
>            Reporter: Thejas M Nair
>
> Hive export command does export to non HDFS file system.
> If a non hdfs filessystem is the default file system, then export command tries to use
hdfs scheme against the url of the default file system.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message