Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 2 Nov 2017 19:47:00 +0000 (UTC)
From: "Mukul Kumar Singh (JIRA)" <jira@apache.org>
To: hdfs-dev@hadoop.apache.org
Message-ID: <JIRA.13115727.1509652013000.135797.1509652020397@Atlassian.JIRA>
In-Reply-To: <JIRA.13115727.1509652013000@Atlassian.JIRA>
References: <JIRA.13115727.1509652013000@Atlassian.JIRA> <JIRA.13115727.1509652013644@jira-lw-us.apache.org>
Subject: [jira] [Created] (HDFS-12768) Ozone: Qualify OzoneFileSystem paths
 in Filesystem APIs
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 02 Nov 2017 19:47:06 -0000

Mukul Kumar Singh created HDFS-12768:
----------------------------------------

             Summary: Ozone: Qualify OzoneFileSystem paths in Filesystem APIs
                 Key: HDFS-12768
                 URL: https://issues.apache.org/jira/browse/HDFS-12768
             Project: Hadoop HDFS
          Issue Type: Sub-task
          Components: ozone
    Affects Versions: HDFS-7240
            Reporter: Mukul Kumar Singh
            Assignee: Mukul Kumar Singh
            Priority: Major
             Fix For: HDFS-7240


This is based on [~stevel@apache.org]'s comments on HDFS-7240. This jira will be used to qualify filesystem paths before they are used, this jira will also address rest of the review comments for filesystem api as well.

General
1) various places use LOG.info("text " + something). they should all move to LOG.info("text {}", something)
2) Once OzoneException -> IOE, you can cut the catch and translate here.
3) qualify path before all uses. That's needed to stop them being relative, and to catch things like someone calling ozfs.rename("o3://bucket/src", "s3a://bucket/dest"), delete("s3a://bucket/path"), etc, as well as problems with validation happening before paths are made absolute.
4) RenameIterator.iterate() it's going to log @ warn whenever it can't delete a temp file because it doesn't exist, which may be a distraction in failures. Better: if(!tmpFile.delete() && tmpFile.exists()), as that will only warn if the temp file is actually there.

OzoneFileSystem.rename().
1) Qualify all the paths before doing directory validation. Otherwise you can defeat the "don't rename into self checks" rename("/path/src", "/path/../path/src/dest").
2) Log @ debu all the paths taken before returning so you can debug if needed.
3) S3A rename ended up having a special RenameFailedException() which innerRename() raises, with text and return code. Outer rename logs the text and returns the return code. This means that all failing paths have an exception clearly thrown, and when we eventually make rename/3 public, it's lined up to throw exceptions back to the caller. Consider copying this code.

OzoneFileSystem.delete
1) qualify path before use
 2) dont' log at error if you can't delete a nonexistent path, it is used everywhere for silent cleanup. Cut it

OzoneFileSystem.ListStatusIterator
1) make status field final

OzoneFileSystem.mkdir
1) do qualify path first.

OzoneFileSystem.getFileStatus
1) getKeyInfo() catches all exceptions and maps to null, which is interpreted not found and eventually surfaces as FNFE. This is misleading if the failure is for any other reason.
2) Once OzoneException -> IOException, getKeyInfo() should only catch & downgrade the explicit not found (404?) responses.

OzoneFileSystem.listKeys()
unless this needs to be tagged as VisibleForTesting, make private. 

OzoneFileSystem.getDefaultBlockSize()
implement getDefaultBlockSize(); add a config option to let people set it. add a sensible default like 64 or 128 MB.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org