Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2EDE6BBCE for ; Sun, 15 Jan 2012 01:30:42 +0000 (UTC) Received: (qmail 63087 invoked by uid 500); 15 Jan 2012 01:30:38 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 62976 invoked by uid 500); 15 Jan 2012 01:30:37 -0000 Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-user@hadoop.apache.org Delivered-To: mailing list common-user@hadoop.apache.org Received: (qmail 62968 invoked by uid 99); 15 Jan 2012 01:30:36 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 15 Jan 2012 01:30:36 +0000 X-ASF-Spam-Status: No, hits=1.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of rmorgan466@gmail.com designates 209.85.212.48 as permitted sender) Received: from [209.85.212.48] (HELO mail-vw0-f48.google.com) (209.85.212.48) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 15 Jan 2012 01:30:31 +0000 Received: by vbbfr13 with SMTP id fr13so111604vbb.35 for ; Sat, 14 Jan 2012 17:30:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=fyDpPkkuR3BbPpRAckJFv2GoXbD49ZVLC252DELHGvM=; b=gUUlEFyChNoDxvPG1mJLnSAIqoLIeEwdqi1zN68WWSrwaIenYSsPXsby0MD0ccl72U qxgTMGga1nqj9wA7PBp2MGHmJHzuVSNZfVu3LvESyGMLJnXOYp/jHgl72N6/cM4eBXWM jxPGwBLLUd1fbMEsVsxnrzNLTDcZIQMWxZaPo= MIME-Version: 1.0 Received: by 10.52.33.99 with SMTP id q3mr3126685vdi.100.1326591010914; Sat, 14 Jan 2012 17:30:10 -0800 (PST) Received: by 10.220.214.65 with HTTP; Sat, 14 Jan 2012 17:30:10 -0800 (PST) Date: Sat, 14 Jan 2012 20:30:10 -0500 Message-ID: Subject: hadoop filesystem cache From: Rita To: "" Content-Type: multipart/alternative; boundary=20cf3079bdd4f7c17b04b687094e --20cf3079bdd4f7c17b04b687094e Content-Type: text/plain; charset=ISO-8859-1 After reading this article, http://www.cloudera.com/blog/2012/01/caching-in-hbase-slabcache/ , I was wondering if there was a filesystem cache for hdfs. For example, if a large file (10gigabytes) was keep getting accessed on the cluster instead of keep getting it from the network why not storage the content of the file locally on the client itself. A use case on the client would be like this: dfs.client.cachedirectory /var/cache/hdfs dfs.client.cachesize in megabytes 100000 Any thoughts of a feature like this? -- --- Get your facts first, then you can distort them as you please.-- --20cf3079bdd4f7c17b04b687094e--