ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Onischuk (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AMBARI-9548) Updating FSRoot for Hive Metastore doesn't seem to work (breaks Hive functionality after move NameNode and NameNode HA)
Date Tue, 10 Feb 2015 13:22:12 GMT
Andrew Onischuk created AMBARI-9548:
---------------------------------------

             Summary: Updating FSRoot for Hive Metastore doesn't seem to work (breaks Hive
functionality after move NameNode and NameNode HA)
                 Key: AMBARI-9548
                 URL: https://issues.apache.org/jira/browse/AMBARI-9548
             Project: Ambari
          Issue Type: Bug
            Reporter: Andrew Onischuk
            Assignee: Andrew Onischuk
             Fix For: 2.0.0


In hive_service.py, we have some logic to check and update FS root for Hive.  
I believe this was added to handle the situation when the NameNode is moved.  
I also think that this code would also automate updating the pointer to
nameservice ID rather than a specific namenode after enabling NameNode HA.  
However, this code does not seem correct:

    
    
    
    def check_fs_root():
      import params  
      fs_root_url = format("{fs_root}{hive_apps_whs_dir}")
      cmd = format("metatool -listFSRoot 2>/dev/null | grep hdfs:// | grep -v '.db$'")
      code, out = shell.call(cmd, user=params.hive_user)
      if code == 0 and fs_root_url.strip() != out.strip():
        cmd = format("metatool -updateLocation {fs_root}{hive_apps_whs_dir} {out}")
        Execute(cmd,
                environment= {'PATH' : params.execute_path },
                user=params.hive_user)
    

Problem 1: "metatool" is not in the PATH for Hive user, so this does not run.  
Problem 2: "metatool" is a wrapper around "hive --service metatool". However,
it does not take any arguments for the hive config directory. Simply running
"metatool -listRootFS", even if the correct PATH was set for metatool, uses
/etc/hive/conf. However, we do not set "hive.metastore.warehouse.dir"
parameter in that file. We need to explicitly use /etc/hive/conf.server.

    
    
    
    /usr/hdp/current/hive-client/bin/metatool -listFSRoot
    

incorrectly yields

    
    
    
    hdfs://yusaku-1.c.pramod-thangali.internal:8020/user/hive/warehouse
    

The above command is equivalent to:

    
    
    
    hive --service metatool -listFSRoot
    

and shows the same incorrect output.

The following correctly shows the fs root:

    
    
    
    hive --config /etc/hive/conf.server--service metatool -listFSRoot
    

hdfs://yusaku-1.c.pramod-thangali.internal:8020/apps/hive/warehouse

Net net, we should be using "hive" command with config correctly set, rather
than the "metatool" wrapper.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message