From common-dev-return-79646-apmail-hadoop-common-dev-archive=hadoop.apache.org@hadoop.apache.org Mon Sep 10 04:41:50 2012 Return-Path: X-Original-To: apmail-hadoop-common-dev-archive@www.apache.org Delivered-To: apmail-hadoop-common-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 29025D313 for ; Mon, 10 Sep 2012 04:41:50 +0000 (UTC) Received: (qmail 85893 invoked by uid 500); 10 Sep 2012 04:41:48 -0000 Delivered-To: apmail-hadoop-common-dev-archive@hadoop.apache.org Received: (qmail 85298 invoked by uid 500); 10 Sep 2012 04:41:44 -0000 Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-dev@hadoop.apache.org Delivered-To: mailing list common-dev@hadoop.apache.org Received: (qmail 85254 invoked by uid 99); 10 Sep 2012 04:41:43 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Sep 2012 04:41:43 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of momina.azam@gmail.com designates 209.85.212.48 as permitted sender) Received: from [209.85.212.48] (HELO mail-vb0-f48.google.com) (209.85.212.48) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 10 Sep 2012 04:41:38 +0000 Received: by vbme21 with SMTP id e21so2716544vbm.35 for ; Sun, 09 Sep 2012 21:41:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=nEHpmRs4GcSXdoVMgRDTY87LKkDeb1aNt9Sz17q4QkA=; b=akKGq3IpsstREoi9gW3gzN1xMm/DZdfzlmP8+ll6nvHoqQUKwS4rc/+rUlBXmf35ic NO9iWU+VT5JTsz1gXbNWSUetOFb+7YUlCzxOQK2Y+XEsR3YU7Tf9+Hz53BBbxobGbF1O YclMl5fK4GRXHe9vLkfzNn4anJQWHrwKEfxDJ8e0Dxv+qQHNmnHJs4n1hf1u/zSuOjvX vmQTZ+ZP7XXOn2VX4X3IjwbEqUUMXHAbc/A3SoHKwWsKIipuQTIRUXXWFFFrfGIJ0dnR L5pKdJzR7tWNq69WOtpQpsrW3OGbvkD3s3t/EIZvP/CzhWpUhFkzXFG0Iu+fQIq0RQQJ 5A6A== MIME-Version: 1.0 Received: by 10.52.175.69 with SMTP id by5mr6246739vdc.77.1347252077781; Sun, 09 Sep 2012 21:41:17 -0700 (PDT) Received: by 10.220.143.9 with HTTP; Sun, 9 Sep 2012 21:41:17 -0700 (PDT) In-Reply-To: References: Date: Mon, 10 Sep 2012 09:41:17 +0500 Message-ID: Subject: Re: difference between hadoop code and the amazon elastic Mapreduce From: Momina Khan To: common-dev@hadoop.apache.org Content-Type: multipart/alternative; boundary=bcaec5171e1f84e31904c95191fa X-Virus-Checked: Checked by ClamAV on apache.org --bcaec5171e1f84e31904c95191fa Content-Type: text/plain; charset=ISO-8859-1 thank u that was very helpful! On Mon, Sep 10, 2012 at 1:04 AM, Eli Collins wrote: > Peter, > > Thanks for the info. Do you guys plan to contribute the rewritten s3 > code (assume you're referring to org.apache.hadoop.fs.s3) back to > Apache? > > Thanks, > Eli > > On Sun, Sep 9, 2012 at 12:38 PM, Sirota, Peter wrote: > > Hi, > > > > The major differences are in s3 file system that has been rewritten in > EMR and in Hadoop interactions with S3. Other differences are in detecting > various failure conditions. > > > > Outside these it's Apache Hadoop. Here is a list of patches EMR applied > on top of 1.0.3 Hadoop > > > http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/EnvironmentConfig_AMIHadoopPatches.html > > > > Regards, > > Peter > > > > > > > > On Sep 9, 2012, at 11:29 AM, "Momina Khan" > wrote: > > > >> hi all! > >> > >> could someone please point out key differences between hadoop code and > >> Amazon's Elastic MapReduce. I am particularly interested in ways that > >> hadoop code is changed/optimized to run on efficiently EC2. > >> > >> cheers! > >> momina > --bcaec5171e1f84e31904c95191fa--