hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3595) Hive should adapt new FsShell commands since Hadoop 2 has changed FsShell argument structures
Date Thu, 18 Oct 2012 09:10:03 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478842#comment-13478842
] 

Harsh J commented on HIVE-3595:
-------------------------------

A quick scan suggests we just need to fix the one instance of "rmr" in use at Hive#replaceFiles:

{code}
      // point of no return -- delete oldPath
      if (oldPath != null) {
        try {
          FileSystem fs2 = oldPath.getFileSystem(conf);
          if (fs2.exists(oldPath)) {
            // use FsShell to move data to .Trash first rather than delete permanently
            FsShell fshell = new FsShell();
            fshell.setConf(conf);
            fshell.run(new String[]{"-rmr", oldPath.toString()});
          }
        } catch (Exception e) {
          //swallow the exception
          LOG.warn("Directory " + oldPath.toString() + " canot be removed.");
        }
      }
{code}

If we can wrap that "-rmr" to use "-rmr" only for < 0.23 hadoop versions, this can be closed.
For higher versions the logic ought to use "-rm -r".
                
> Hive should adapt new FsShell commands since Hadoop 2 has changed FsShell argument structures
> ---------------------------------------------------------------------------------------------
>
>                 Key: HIVE-3595
>                 URL: https://issues.apache.org/jira/browse/HIVE-3595
>             Project: Hive
>          Issue Type: Improvement
>          Components: Shims
>    Affects Versions: 0.9.0
>            Reporter: Harsh J
>            Priority: Minor
>
> A simple example is that hive calls "-rmr" in the FsShell class, which in Hadoop 2 is
"rm -r". This helps avoid printing an unnecessary Deprecated warning in Hive when the Hadoop23
(or hadoop-2) shim is in use.
> We should wrap the logic and call the right commands of hadoop-2 to avoid this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message