Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CE5ABDC4F for ; Wed, 31 Oct 2012 23:20:13 +0000 (UTC) Received: (qmail 33987 invoked by uid 500); 31 Oct 2012 23:20:08 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 33865 invoked by uid 500); 31 Oct 2012 23:20:08 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 33857 invoked by uid 99); 31 Oct 2012 23:20:08 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Oct 2012 23:20:08 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of pankaj@brightroll.com designates 209.85.160.48 as permitted sender) Received: from [209.85.160.48] (HELO mail-pb0-f48.google.com) (209.85.160.48) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Oct 2012 23:19:59 +0000 Received: by mail-pb0-f48.google.com with SMTP id wy7so1376199pbc.35 for ; Wed, 31 Oct 2012 16:19:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=from:content-type:content-transfer-encoding:subject:message-id:date :to:mime-version:x-mailer:x-gm-message-state; bh=sgp7P47KUaMDkTRMN0F42+mRyNr8i/tXpoNfaqJXEEs=; b=L9hXUfAyRly0u8tDEko5hNQ6+1wgw2t7SwXAqFk/ezJHDxa6o8z2HzSS82wVy2AEw5 IcaEApwF+IhM73w97k2HNpMkl0Ntih9wsirDT66Xmohxy+ek7iGzX+ZLv02E0IjstjzF d7hMhJp5N6xYSRd/SA698Id1Y1giaGExx3iJn27ZkF/uTEJW9dpSa5dBDszl2YrHYMBy H7pe+RiGgBoMv5BwxlasLnkxNPzYl3afn1tO+RQSVvtK6KmrnEFITvNoa1hdIiQmhJLL efus8QBrwkbPvSBheS28FDi2LJfK3+XHSg5YEqjKTN0UeC+0GMcih/MQr3ZURl8Zht/M 1mXg== Received: by 10.68.253.230 with SMTP id ad6mr43236918pbd.84.1351725577817; Wed, 31 Oct 2012 16:19:37 -0700 (PDT) Received: from [192.168.1.191] (23-24-195-25-static.hfc.comcastbusiness.net. [23.24.195.25]) by mx.google.com with ESMTPS id ni3sm2980312pbc.2.2012.10.31.16.19.36 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 31 Oct 2012 16:19:37 -0700 (PDT) From: Pankaj Gupta Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: Reading part of file using Map Reduce Message-Id: <6E485975-3E63-4CD6-B2B4-36C7E5917335@brightroll.com> Date: Wed, 31 Oct 2012 16:20:05 -0700 To: user@hadoop.apache.org Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) X-Mailer: Apple Mail (2.1498) X-Gm-Message-State: ALoCoQn4Flsvzkb15BA0UsYA8A80VPLwL3b8+ICg2CRlK8J194J7LQYZrotPTNQJwW23slni5Gl3 X-Virus-Checked: Checked by ClamAV on apache.org Hi, Is it possible to run a MapReduce job on a part of file on HDFS? The use = case is using a single file on HDFS as a stream to store all log events = of a particular kind. New data can grow on top while Map Reduce can = process old data. Of course one option would be to copy part of data = into a separate file and give that to MapReduce but I was wondering if = that extra copy can be avoided. Thanks, Pankaj=