Return-Path: Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: (qmail 21509 invoked from network); 18 May 2010 12:06:54 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 18 May 2010 12:06:54 -0000 Received: (qmail 32388 invoked by uid 500); 18 May 2010 12:06:52 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 32323 invoked by uid 500); 18 May 2010 12:06:52 -0000 Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-user@hadoop.apache.org Delivered-To: mailing list common-user@hadoop.apache.org Received: (qmail 32315 invoked by uid 99); 18 May 2010 12:06:52 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 May 2010 12:06:52 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of pierreact@gmail.com designates 74.125.82.48 as permitted sender) Received: from [74.125.82.48] (HELO mail-ww0-f48.google.com) (74.125.82.48) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 18 May 2010 12:06:44 +0000 Received: by wwi14 with SMTP id 14so1217026wwi.35 for ; Tue, 18 May 2010 05:06:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=GcngHM8eeXXprmC9QCrORwbnYhHJaByVs5+ZlSNk2Gw=; b=Uvj51uObxNet8cxQyaecGvbwX6i+DwH+ZYGuF9Oj18LJ+c9Plkv3cnbGgPylx1WuLV Zq2bAvJqaKEKj5vwXPVOLLY1Pdm/ROUid54C4lbYepVk8hgOEkPN+qZSrFM3QYV1mIQI umQNSXEIacR6a6VuAlfocYqLUTuYzD90S/aPU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=uI3sM5BEFZDk/N3ACDOYpqn+EPkHaCUeG+GAfiyr9GKRym53OZe80fGaH2nlpVq1o1 +6W6drbv9CcIn3RAPY5/89nWGp3wAQSGTJyC7JjlKYuEVAXtAOtCFK4Sj+cq67l22TcN 6gQmADu8XMAR/iCoxR/KtaoHr5bZEOTuaEtbU= MIME-Version: 1.0 Received: by 10.227.134.206 with SMTP id k14mr6175981wbt.94.1274184384372; Tue, 18 May 2010 05:06:24 -0700 (PDT) Received: by 10.216.181.83 with HTTP; Tue, 18 May 2010 05:06:24 -0700 (PDT) Date: Tue, 18 May 2010 14:06:24 +0200 Message-ID: Subject: Any possible to set hdfs block size to a value smaller than 64MB? From: Pierre ANCELOT To: common-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=001636831b209bdf110486dd2bdd X-Virus-Checked: Checked by ClamAV on apache.org --001636831b209bdf110486dd2bdd Content-Type: text/plain; charset=ISO-8859-1 Hi, I'm porting a legacy application to hadoop and it uses a bunch of small files. I'm aware that having such small files ain't a good idea but I'm not doing the technical decisions and the port has to be done for yesterday... Of course such small files are a problem, loading 64MB blocks for a few lines of text is an evident loss. What will happen if I set a smaller, or even way smaller (32kB) blocks? Thank you. Pierre ANCELOT. --001636831b209bdf110486dd2bdd--