Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9166E1187E for ; Fri, 10 May 2013 05:28:51 +0000 (UTC) Received: (qmail 89748 invoked by uid 500); 10 May 2013 05:28:46 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 89626 invoked by uid 500); 10 May 2013 05:28:46 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 89619 invoked by uid 99); 10 May 2013 05:28:46 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 May 2013 05:28:46 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of harsh@cloudera.com designates 209.85.223.171 as permitted sender) Received: from [209.85.223.171] (HELO mail-ie0-f171.google.com) (209.85.223.171) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 May 2013 05:28:40 +0000 Received: by mail-ie0-f171.google.com with SMTP id e11so7251592iej.16 for ; Thu, 09 May 2013 22:28:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:content-type:x-gm-message-state; bh=EXElYMHOwIAta1Mm8uG3iZGXYNm2Q111SfQ/2b4HTuw=; b=C4OT/joPWadAPibahibD9xJ7tt9XPrPpMloS0+Noiig2kVgk6tDonrBVTDSD6KnBN7 PrwEq38BHCwDjU10nbl9g5jUyLQFNThSVZWUAwaAXwUHwgcQ1vTwRK7aN4hy7iCsxZTT W/JvVXItOTgxCLPSB9iMLaIRhWj2ZASPhIfgYpJw+JYtUZM9JikGznFmRoa+pR/Kn7Ol RatBKsnFNFuCr5a9Dl8j8b6CDbkA+J1wQd8WxEO1oUmCHhAROMuKuNpaW82Gkl0EHKfV bbjGs1KaA2PxiO5rZTdWA6+784i4VnME3CTdCgd/QWjXgmPQBSBOj/M9lFdSBasxmGPT fV+w== X-Received: by 10.50.73.165 with SMTP id m5mr818311igv.28.1368163699462; Thu, 09 May 2013 22:28:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.50.149.165 with HTTP; Thu, 9 May 2013 22:27:59 -0700 (PDT) In-Reply-To: References: From: Harsh J Date: Fri, 10 May 2013 10:57:59 +0530 Message-ID: Subject: Re: issues with decrease the default.block.size To: "" Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQkcKeCEsUQZDQWLe3ON9qfJfin5xy7+W64iao3nnNrn79iddE9FQsBr60EA7SxtDhtEEpXl X-Virus-Checked: Checked by ClamAV on apache.org Are you looking to decrease it to get more parallel map tasks out of the small files? Are you currently CPU bound on processing these small files? On Thu, May 9, 2013 at 9:12 PM, YouPeng Yang wrote: > hi ALL > > I am going to setup a new hadoop environment, .Because of there are > lots of small files, I would like to change the default.block.size to > 16MB > other than adopting the ways to merge the files into large enough (e.g > using sequencefiles). > I want to ask are there any bad influences or issues? > > Regards > -- Harsh J