hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <>
Subject [jira] Commented: (HIVE-232) metastore.warehouse configuration should use inherited hadoop configuration
Date Thu, 15 Jan 2009 03:17:59 GMT


Joydeep Sen Sarma commented on HIVE-232:

ok with the change in the metastore. also has a reference to METASTOREWAREHOUSEDIR
- but not clear that it's being used at all.

the change in LoadSemanticAnalyzer is probably not necessary (a test case might have made
this clear). if i am reading this right - the net effect being desired is that the the from
path should use for scheme/authority - right?

but this was already happening, a little twined but reading the (current) code:

    fs = FileSystem.get(fromURI, conf);
    String fromAuthority = null;

    // fall back to configuration based scheme if necessary
    if(StringUtils.isEmpty(fromScheme)) {
      fromScheme = fs.getUri().getScheme();
      fromAuthority = fs.getUri().getAuthority();

    if(fromScheme.equals("hdfs")) {
      fromAuthority = StringUtils.isEmpty(fromURI.getAuthority()) ?
        fs.getUri().getAuthority() : fromURI.getAuthority();

the fs initialization will get the scheme and authority from if the fromURI
does not supply them. Then we test one by one if scheme and authority are already supplied
- if not - we use the one supplied by fs (so use for both of these as a default).

unless i am missing something - we are just trying to do the same thing in a slightly different

> metastore.warehouse configuration should use inherited hadoop configuration
> ---------------------------------------------------------------------------
>                 Key: HIVE-232
>                 URL:
>             Project: Hadoop Hive
>          Issue Type: Bug
>            Reporter: Josh Ferguson
>         Attachments: hive-232.patch
> the hive.metastore.warehouse.dir configuration property in hive-*.xml needs to use the
protocol, host, and port when it is inherited from the property in hadoop-site.xml.
> When it doesn't and no protocol is found then a broad range of "Move" operations when
the source and target are both in the DFS will fail.
> Currently this can be worked around by prepending the protocol, host and port of the
hadoop nameserver into the value of the hive.metastore.warehouse.dir property.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message