Return-Path: X-Original-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A4DC519ED5 for ; Wed, 20 Apr 2016 21:03:26 +0000 (UTC) Received: (qmail 69998 invoked by uid 500); 20 Apr 2016 21:03:26 -0000 Delivered-To: apmail-hadoop-hdfs-issues-archive@hadoop.apache.org Received: (qmail 69921 invoked by uid 500); 20 Apr 2016 21:03:25 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-issues@hadoop.apache.org Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 69853 invoked by uid 99); 20 Apr 2016 21:03:25 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Apr 2016 21:03:25 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id A50942C1F64 for ; Wed, 20 Apr 2016 21:03:25 +0000 (UTC) Date: Wed, 20 Apr 2016 21:03:25 +0000 (UTC) From: "Chris Nauroth (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HDFS-10312) Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HDFS-10312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-10312: --------------------------------- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) Thank you for the reviews anyone. The test failures were unrelated. I corrected the whitespace warning. I have committed this to trunk, branch-2 and branch-2.8. > Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction. > ---------------------------------------------------------------------------------------------------- > > Key: HDFS-10312 > URL: https://issues.apache.org/jira/browse/HDFS-10312 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Reporter: Chris Nauroth > Assignee: Chris Nauroth > Fix For: 2.8.0 > > Attachments: HDFS-10312.001.patch, HDFS-10312.002.patch, HDFS-10312.003.patch, HDFS-10312.004.patch > > > Our RPC server caps the maximum size of incoming messages at 64 MB by default. For exceptional circumstances, this can be uptuned using {{ipc.maximum.data.length}}. However, for block reports, there is still an internal maximum length restriction of 64 MB enforced by protobuf. (Sample stack trace to follow in comments.) This issue proposes to apply the same override to our block list decoding, so that large block reports can proceed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)