Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Tue, 13 Oct 2015 20:20:06 +0000 (UTC)
From: "Colin Patrick McCabe (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12861429.1441238438000.28003.1444767606107@Atlassian.JIRA>
In-Reply-To: <JIRA.12861429.1441238438000@Atlassian.JIRA>
References: <JIRA.12861429.1441238438000@Atlassian.JIRA>
 <JIRA.12861429.1441238438110@arcas>
Subject: [jira] [Commented] (HDFS-9011) Support splitting BlockReport of a
 storage into multiple RPC
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-9011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14955565#comment-14955565 ] 

Colin Patrick McCabe commented on HDFS-9011:
--------------------------------------------

It would be simpler for the admin to create two (or more) storages on the same drive, and it wouldn't involve any code modification by us.

> Support splitting BlockReport of a storage into multiple RPC
> ------------------------------------------------------------
>
>                 Key: HDFS-9011
>                 URL: https://issues.apache.org/jira/browse/HDFS-9011
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>         Attachments: HDFS-9011.000.patch, HDFS-9011.001.patch, HDFS-9011.002.patch
>
>
> Currently if a DataNode has too many blocks (more than 1m by default), it sends multiple RPC to the NameNode for the block report, each RPC contains report for a single storage. However, in practice we've seen sometimes even a single storage can contains large amount of blocks and the report even exceeds the max RPC data length. It may be helpful to support sending multiple RPC for the block report of a storage. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)