Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id CB7DC200BE2 for ; Thu, 1 Dec 2016 01:01:04 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id CA205160B21; Thu, 1 Dec 2016 00:01:04 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 45C40160B13 for ; Thu, 1 Dec 2016 01:01:04 +0100 (CET) Received: (qmail 37658 invoked by uid 500); 1 Dec 2016 00:01:03 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 37605 invoked by uid 99); 1 Dec 2016 00:01:03 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Dec 2016 00:01:03 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 39F5C2C2A69 for ; Thu, 1 Dec 2016 00:01:03 +0000 (UTC) Date: Thu, 1 Dec 2016 00:01:03 +0000 (UTC) From: "Andrew Wang (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HDFS-5517) Lower the default maximum number of blocks per file MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 01 Dec 2016 00:01:05 -0000 [ https://issues.apache.org/jira/browse/HDFS-5517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-5517: ------------------------------ Resolution: Fixed Fix Version/s: 3.0.0-alpha2 Status: Resolved (was: Patch Available) Committed, thanks for the review Akira and sorry again for missing the broken tests earlier. > Lower the default maximum number of blocks per file > --------------------------------------------------- > > Key: HDFS-5517 > URL: https://issues.apache.org/jira/browse/HDFS-5517 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Affects Versions: 2.2.0 > Reporter: Aaron T. Myers > Assignee: Aaron T. Myers > Labels: BB2015-05-TBR > Fix For: 3.0.0-alpha2 > > Attachments: HDFS-5517.002.patch, HDFS-5517.003.patch, HDFS-5517.patch > > > We introduced the maximum number of blocks per file in HDFS-4305, but we set the default to 1MM. In practice this limit is so high as to never be hit, whereas we know that an individual file with 10s of thousands of blocks can cause problems. We should lower the default value, in my opinion to 10k. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org