hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11997) ChunkManager functions do not use the argument keyName
Date Mon, 19 Jun 2017 22:06:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054822#comment-16054822

Anu Engineer commented on HDFS-11997:

[~vagarychen] Thanks for your comments. You are absolutely correct. Chunk Manager never really
needs to know which key it is part of. However, some time ago we did a prototype for Disaster
Recovery (DR) using the containers. In that prototype we explored writing the chunks directly
from the datanode to S3 buckets.

If my memory serves me correct, that is when we added this. Since  having the context made
it easier to write the chunk to the remote cluster. If we decide to do that in future this
might be useful. 

You are absolutely right from a layering perspective, this does not make sense -- A chunk
never needs to know the parent context. This was only for propagating that information over
to another cluster.

Right now, we have no  strong use case for this and DR has to be re-developed anyway (probably
will not need this info for that). So this is prototyping residue that can indeed be removed.
 Please feel free to remove it.  Completely your call.


> ChunkManager functions do not use the argument keyName
> ------------------------------------------------------
>                 Key: HDFS-11997
>                 URL: https://issues.apache.org/jira/browse/HDFS-11997
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Chen Liang
>            Assignee: Chen Liang
> {{ChunkManagerImpl}}'s functions i.e. {{writeChunk}} {{readChunk}} {{deleteChunk}} all
take a {{keyName}} argument, which is not being used by any of them.
> I think this makes sense because conceptually {{ChunkManager}} should not have to know
keyName to do anything, probably except for some sort of sanity check or logging, which is
not there either. We should revisit whether we need it here. I think we should remove it to
make the Chunk syntax, and the function signatures more cleanly abstracted.
> Any comments? [~anu]

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message