hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ray (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7702) Move metadata across namenode - Effort to a real distributed namenode
Date Thu, 29 Jan 2015 21:27:35 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14297712#comment-14297712
] 

Ray commented on HDFS-7702:
---------------------------

Hi [~clamb],
Good questions.
For the failure scenarios, I will document more on it, as you can image , it's a transaction,
if one step failed the whole transaction will roll back. Administrator has to kick off again,
and the whole transaction should not take long time.
I choose kryo rather than protobuf, because namespace structure is too complex to convert
so many java objects to PB (may be I am wrong here), and I have to dynamically transfer some
namespace data. I use kyronet, because it's seemless with kryo, it's possible to use IPC instead
if have a good reason.:) And I found deadlock while trying to serialize INode object, Kryo
seems easy to handle customized serialization as well. 
I am sorry for other uncleared statement, I will try to make them more clear.


> Move metadata across namenode - Effort to a real distributed namenode
> ---------------------------------------------------------------------
>
>                 Key: HDFS-7702
>                 URL: https://issues.apache.org/jira/browse/HDFS-7702
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Ray
>            Assignee: Ray
>
> Implement a tool can show in memory namespace tree structure with weight(size) and a
API can move metadata across different namenode. The purpose is moving data efficiently and
faster, without moving blocks on datanode.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message