hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mukul Kumar Singh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-12572) Ozone: OzoneFileSystem: delete/list status/rename/mkdir APIs
Date Mon, 09 Oct 2017 17:55:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-12572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16197400#comment-16197400
] 

Mukul Kumar Singh commented on HDFS-12572:
------------------------------------------

Thanks for the review [~vagarychen],

OzoneFileSystem.java

1. Looks like in the rename, it is after all subdirectories and files are renamed, the src
path gets removed? Seems this means the rename can potentially generates doubled size of entries
of the directory being renamed. Would it be better (if possible) to rename one key at a time,
deletes it, the move to next one? I'm thinking of adding bucket.deleteKey around line 267-268,
and remove delete(src, true); on line 358. Not sure if this is enough though.
bq. This is needed because lets say that rename fails somewhere in between, then the source
will still be in stable state when we have both the copies. In the future, renames can be
done in KSM using HDFS-11909. This will ensure both atomic rename and will avoid data copy
as well.


2. OzoneListingIterator make it an inner class of OzoneFileSystem? Because it seems file system
calls delete, list status and rename are and will be the only use of this class.
bq. Done

3. in mkdir, seems when an inode in the path is a file, mkdir gets rollback (line 463). Should
we also roll back if createDir fails (line 473)
bq. Done

4. "Directory is represented using empty key with no value.” did you mean root directory?
bq. Done, updated the comment, I wanted to tell that directory is represented by a null value.

5. OzoneListingIterator#iterate and processKey() both return a boolean but seems nowhere depends
on this return value. So this seems a bit unnecessary to me. Do we need to have processKey
and iterate to return anything at all?
bq. delete operation needs a return value, which is returned by the iterate function.

BucketProcessTemplate.java

Just log a word "Success" does not seem to be very useful, can expand this message a little
bit? Similarly for KeyProcessTemplate
bq. Had converted this already existing message to debug to avoid excessive logging during
ozone filesystem operation, the log message was not changed as part of this patch.

> Ozone: OzoneFileSystem: delete/list status/rename/mkdir APIs
> ------------------------------------------------------------
>
>                 Key: HDFS-12572
>                 URL: https://issues.apache.org/jira/browse/HDFS-12572
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Mukul Kumar Singh
>            Assignee: Mukul Kumar Singh
>              Labels: ozoneMerge
>             Fix For: HDFS-7240
>
>         Attachments: HDFS-12572-HDFS-7240.001.patch, HDFS-12572-HDFS-7240.002.patch,
HDFS-12572-HDFS-7240.003.patch, HDFS-12572-HDFS-7240.004.patch
>
>
> This jira will add the delete/list status/rename/mkdir APIs



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message