hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Akira AJISAKA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10392) Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem)
Date Wed, 30 Jul 2014 04:21:38 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14078866#comment-14078866
] 

Akira AJISAKA commented on HADOOP-10392:
----------------------------------------

Re-reading the code, I've changed my mind.

In StreamJob.java#parseArgv(), If someone uses StreamJob pointing another FileSystem (such
as hdfs://...) by '-file' option, the path is changed to LocalFileSystem (file://...) by {{Path.makeQualified(FileSystem)}}.
{code}
  packageFiles_.add(file);
  try {
    Path path = new Path(file);
    FileSystem localFs = FileSystem.getLocal(config_);
    String finalPath = path.makeQualified(localFs).toString();
{code}
However, the code later checks if the file exists, so StreamJob fails if a user sets another
FileSystem by -file option.
{code}
   validate(packageFiles_);
{code}
Therefore, I'm thinking it's okay to use {{localFs.makeQualified(path)}}. If the code is changed
to use it, StreamJob fails there in the example but it's not changed whether StreamJob fails
or not.

In Gridmix.java,
{code}
  final FileSystem inputFs = ioPath.getFileSystem(conf);
  ioPath = ioPath.makeQualified(inputFs);
  boolean succeeded = false;
  try {
    succeeded = FileSystem.mkdirs(inputFs, ioPath, new FsPermission((short)0777));
{code}
ioPath.getFileSystem(conf) returns the FileSystem which owns ioPath. Therefore if ioPath.makeQualified(inputFs)
is changed to inputFs.makeQualified(ioPath), it doesn't fail.

> Use FileSystem#makeQualified(Path) instead of Path#makeQualified(FileSystem)
> ----------------------------------------------------------------------------
>
>                 Key: HADOOP-10392
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10392
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: 2.3.0
>            Reporter: Akira AJISAKA
>            Assignee: Akira AJISAKA
>            Priority: Minor
>              Labels: newbie
>         Attachments: HADOOP-10392.2.patch, HADOOP-10392.3.patch, HADOOP-10392.4.patch,
HADOOP-10392.4.patch, HADOOP-10392.5.patch, HADOOP-10392.patch
>
>
> There're some methods calling Path.makeQualified(FileSystem), which causes javac warning.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message