hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-232) metastore.warehouse configuration should use inherited hadoop configuration
Date Thu, 15 Jan 2009 03:17:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12663990#action_12663990

Joydeep Sen Sarma commented on HIVE-232:

ok with the change in the metastore. DB.java also has a reference to METASTOREWAREHOUSEDIR
- but not clear that it's being used at all.

the change in LoadSemanticAnalyzer is probably not necessary (a test case might have made
this clear). if i am reading this right - the net effect being desired is that the the from
path should use fs.default.name for scheme/authority - right?

but this was already happening, a little twined but reading the (current) code:

    fs = FileSystem.get(fromURI, conf);
    String fromAuthority = null;

    // fall back to configuration based scheme if necessary
    if(StringUtils.isEmpty(fromScheme)) {
      fromScheme = fs.getUri().getScheme();
      fromAuthority = fs.getUri().getAuthority();

    if(fromScheme.equals("hdfs")) {
      fromAuthority = StringUtils.isEmpty(fromURI.getAuthority()) ?
        fs.getUri().getAuthority() : fromURI.getAuthority();

the fs initialization will get the scheme and authority from fs.default.name if the fromURI
does not supply them. Then we test one by one if scheme and authority are already supplied
- if not - we use the one supplied by fs (so use fs.default.name for both of these as a default).

unless i am missing something - we are just trying to do the same thing in a slightly different

> metastore.warehouse configuration should use inherited hadoop configuration
> ---------------------------------------------------------------------------
>                 Key: HIVE-232
>                 URL: https://issues.apache.org/jira/browse/HIVE-232
>             Project: Hadoop Hive
>          Issue Type: Bug
>            Reporter: Josh Ferguson
>         Attachments: hive-232.patch
> the hive.metastore.warehouse.dir configuration property in hive-*.xml needs to use the
protocol, host, and port when it is inherited from the fs.name property in hadoop-site.xml.
> When it doesn't and no protocol is found then a broad range of "Move" operations when
the source and target are both in the DFS will fail.
> Currently this can be worked around by prepending the protocol, host and port of the
hadoop nameserver into the value of the hive.metastore.warehouse.dir property.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message