ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-9548) Updating FSRoot for Hive Metastore doesn't seem to work (breaks Hive functionality after move NameNode and NameNode HA)
Date Tue, 10 Feb 2015 14:56:11 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-9548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314258#comment-14314258
] 

Hudson commented on AMBARI-9548:
--------------------------------

SUCCESS: Integrated in Ambari-trunk-Commit #1722 (See [https://builds.apache.org/job/Ambari-trunk-Commit/1722/])
AMBARI-9548. Updating FSRoot for Hive Metastore doesn't seem to work (breaks Hive functionality
after move NameNode and NameNode HA) (aonishuk) (aonishuk: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=43150b95aa3a2974193338d23e4b3e3632927bd3)
* ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_server.py
* ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_service.py


> Updating FSRoot for Hive Metastore doesn't seem to work (breaks Hive functionality after
move NameNode and NameNode HA)
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-9548
>                 URL: https://issues.apache.org/jira/browse/AMBARI-9548
>             Project: Ambari
>          Issue Type: Bug
>            Reporter: Andrew Onischuk
>            Assignee: Andrew Onischuk
>             Fix For: 2.0.0
>
>
> In hive_service.py, we have some logic to check and update FS root for Hive.  
> I believe this was added to handle the situation when the NameNode is moved.  
> I also think that this code would also automate updating the pointer to
> nameservice ID rather than a specific namenode after enabling NameNode HA.  
> However, this code does not seem correct:
>     
>     
>     
>     def check_fs_root():
>       import params  
>       fs_root_url = format("{fs_root}{hive_apps_whs_dir}")
>       cmd = format("metatool -listFSRoot 2>/dev/null | grep hdfs:// | grep -v '.db$'")
>       code, out = shell.call(cmd, user=params.hive_user)
>       if code == 0 and fs_root_url.strip() != out.strip():
>         cmd = format("metatool -updateLocation {fs_root}{hive_apps_whs_dir} {out}")
>         Execute(cmd,
>                 environment= {'PATH' : params.execute_path },
>                 user=params.hive_user)
>     
> Problem 1: "metatool" is not in the PATH for Hive user, so this does not run.  
> Problem 2: "metatool" is a wrapper around "hive --service metatool". However,
> it does not take any arguments for the hive config directory. Simply running
> "metatool -listRootFS", even if the correct PATH was set for metatool, uses
> /etc/hive/conf. However, we do not set "hive.metastore.warehouse.dir"
> parameter in that file. We need to explicitly use /etc/hive/conf.server.
>     
>     
>     
>     /usr/hdp/current/hive-client/bin/metatool -listFSRoot
>     
> incorrectly yields
>     
>     
>     
>     hdfs://yusaku-1.c.pramod-thangali.internal:8020/user/hive/warehouse
>     
> The above command is equivalent to:
>     
>     
>     
>     hive --service metatool -listFSRoot
>     
> and shows the same incorrect output.
> The following correctly shows the fs root:
>     
>     
>     
>     hive --config /etc/hive/conf.server--service metatool -listFSRoot
>     
> hdfs://yusaku-1.c.pramod-thangali.internal:8020/apps/hive/warehouse
> Net net, we should be using "hive" command with config correctly set, rather
> than the "metatool" wrapper.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message