hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MIS (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-1451) Creating a table stores the full address of namenode in the metadata. This leads to problems when the namenode address changes.
Date Wed, 31 Aug 2011 17:18:10 GMT

    [ https://issues.apache.org/jira/browse/HIVE-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13094706#comment-13094706
] 

MIS commented on HIVE-1451:
---------------------------

+1 for the issue. 
This is one of those features which many assume exists by default, but doesn't. 

I too have run into this and resolved it by changing the DB_LOCATION_URI column and LOCATION
in the tables DBS and SDS respectively to point to the latest namenode URI. {My metastore
was on MySql}.

This issue will help us from manually changing namenode URI in db should the address of the
namenode change.


> Creating a table stores the full address of namenode in the metadata. This leads to problems
when the namenode address changes.
> -------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-1451
>                 URL: https://issues.apache.org/jira/browse/HIVE-1451
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore, Query Processor
>    Affects Versions: 0.5.0
>         Environment: Any
>            Reporter: Arvind Prabhakar
>
> Here is an excerpt from table metadata for an arbitrary table {{table1}}:
> {noformat}
> hive> describe extended table1;
> OK
> ...
> Detailed Table Information	...
> location:hdfs://localhost:9000/user/arvind/hive/warehouse/table1, 
> ...
> {noformat}
> As can be seen, the full address of namenode is captured in the location information
for the table. This information is later used to run any queries on the table - thus making
it impossible to change the namenode location once the table has been created. For example,
for the above table, a query will fail if the namenode is migrated from port 9000 to 8020:
> {noformat}
> hive> select * from table1;
> OK
> Failed with exception java.io.IOException:java.net.ConnectException: Call to localhost/127.0.0.1:9000
> failed on connection exception: java.net.ConnectException: Connection refused
> Time taken: 10.78 seconds
> hive> 
> {noformat}
> It should be possible to change the namenode location regardless of when the tables are
created. Also, any query execution should work with the configured namenode at that point
in time rather than requiring the configuration to be exactly the same at the time when the
tables were created.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message