Return-Path: 
 <common-user-return-29188-apmail-hadoop-common-user-archive=hadoop.apache.org@hadoop.apache.org>
X-Original-To: apmail-hadoop-common-user-archive@www.apache.org
Delivered-To: apmail-hadoop-common-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 163996695
	for <apmail-hadoop-common-user-archive@www.apache.org>;
 Tue, 19 Jul 2011 20:26:59 +0000 (UTC)
Received: (qmail 40832 invoked by uid 500); 19 Jul 2011 20:26:56 -0000
Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org
Received: (qmail 40518 invoked by uid 500); 19 Jul 2011 20:26:55 -0000
Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:common-user-help@hadoop.apache.org>
List-Unsubscribe: <mailto:common-user-unsubscribe@hadoop.apache.org>
List-Post: <mailto:common-user@hadoop.apache.org>
List-Id: <common-user.hadoop.apache.org>
Reply-To: common-user@hadoop.apache.org
Delivered-To: mailing list common-user@hadoop.apache.org
Received: (qmail 40507 invoked by uid 99); 19 Jul 2011 20:26:55 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Jul 2011 20:26:55 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of jshrinivas@gmail.com
 designates 74.125.83.176 as permitted sender)
Received: from [74.125.83.176] (HELO mail-pv0-f176.google.com) (74.125.83.176)
    by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 19 Jul 2011 20:26:48 +0000
Received: by pve37 with SMTP id 37so6567916pve.35
        for <common-user@hadoop.apache.org>;
 Tue, 19 Jul 2011 13:26:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        bh=ZC2qAvoNe2aO0W3MwZAB61Mr3Z82oRUYM62yFhi/diY=;
        b=rQCNZL9jEsrDTz3oFIIIK+W0lh57TBNfQ+DFpE2cQ9Acgyfp9x1wBqYKgzbg5LH2fV
         q6HE78lTU90+bQuFGv7b8HGD57vVc13x4dxgTxqrUM2mxWjvt4641jqnetVZBuxVNcA4
         t9BHp8B45G2u0b3+wi6f5jSpPpCFB+7nUhrxY=
MIME-Version: 1.0
Received: by 10.142.192.15 with SMTP id p15mr3891509wff.5.1311107188367; Tue,
 19 Jul 2011 13:26:28 -0700 (PDT)
Received: by 10.142.213.17 with HTTP; Tue, 19 Jul 2011 13:26:28 -0700 (PDT)
Date: Tue, 19 Jul 2011 15:26:28 -0500
Message-ID: 
 <CAB3=nWSY6cgj4YycaQFMZQSpgJ3Frc_nhXrRzJfS+tdqyQzmLQ@mail.gmail.com>
Subject: IO pipeline optimizations
From: Shrinivas Joshi <jshrinivas@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=000e0cd23b8a39a34104a871eefa

--000e0cd23b8a39a34104a871eefa
Content-Type: text/plain; charset=ISO-8859-1

This blog post on YDN website
http://developer.yahoo.com/blogs/hadoop/posts/2009/08/the_anatomy_of_hadoop_io_pipel/has
detailed discussion on different steps involved in Hadoop IO
operations
and opportunities for optimizations. Could someone please comment on current
state of these potential optimizations? Are some of these expected to be
addressed in "next gen MR" release?

Thanks,
-Shrinivas

--000e0cd23b8a39a34104a871eefa--