hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Virajith Jalaparti (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10636) Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File.
Date Fri, 12 Aug 2016 18:48:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419293#comment-15419293

Virajith Jalaparti commented on HDFS-10636:

Hi [~eddyxu], 

Thank you for the comments. 

I agree with points 1 and 2, and will fix them.

bq. {{breakHardlinksIfNeeded}}, {{copyMetadata}} and {{copyBlockdata}} should not be in {{ReplicaInfo}}.
Or should not use {{File}} as input.

Agree that {{copyMetadata}} and {{copyBlockdata}} should not have {{File}} as a parameter.
I will change it to {{URI}} to be more general. {{breakHardLinksIfNeeded}} has always been
in {{ReplicaInfo}}. I made it abstract and moved the implementation to {{LocalReplica}}. 

bq. {{ReplicaUnderRecovery}}. Is there a way to avoid casting {{ReplicaInfo}} to {{LocalReplica}}?

The only place where the fact that {{original}} is a {{LocalReplica}} matters is in {{ReplicaUnderRecovery::setDir()}}.
One way to address this would be to add the cast only when {{original.setDir()}} is called.
The other way to deal with this would be to add {{setDir}} to {{ReplicaInfo}} but to avoid
{{File}} as a parameter, it should take in an URI. Which do you think is better?

bq. In general, there are places in this patch that return {{ReplicaInfo}} for {{FinalizedReplica}}.
which would makes type system weaker and is not future-proof. Is it necessary to be changed?

This was intentional. The way I was thinking about it was that the state of {{ReplicaInfo}}
should be known using {{ReplicaInfo::getState()}}, and not using the type system. The current
code does the latter -- it uses the type system to ensure that replicas are in a certain state.
Not relying on the type system and using the former (use {{ReplicaInfo::getState()}}) seems
a cleaner way of doing this. What do you think? Also, {{FinalizedReplica}} in the current
type hierarchy, is a {{LocalReplica}}. So, referring to replicas using {{FinalizedReplica}}
assumes that they are {{LocalReplica}} and hence, backed by {{File}} s. 

> Modify ReplicaInfo to remove the assumption that replica metadata and data are stored
in java.io.File.
> ------------------------------------------------------------------------------------------------------
>                 Key: HDFS-10636
>                 URL: https://issues.apache.org/jira/browse/HDFS-10636
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, fs
>            Reporter: Virajith Jalaparti
>            Assignee: Virajith Jalaparti
>         Attachments: HDFS-10636.001.patch, HDFS-10636.002.patch, HDFS-10636.003.patch,
HDFS-10636.004.patch, HDFS-10636.005.patch
> Replace java.io.File related APIs from {{ReplicaInfo}}, and enable the definition of
new {{ReplicaInfo}} sub-classes whose metadata and data can be present on external storages

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message