Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 61BA0FC51 for ; Tue, 26 Mar 2013 04:41:01 +0000 (UTC) Received: (qmail 23196 invoked by uid 500); 26 Mar 2013 04:40:56 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 23103 invoked by uid 500); 26 Mar 2013 04:40:56 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 23009 invoked by uid 99); 26 Mar 2013 04:40:52 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Mar 2013 04:40:52 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of azuryyyu@gmail.com designates 209.85.223.169 as permitted sender) Received: from [209.85.223.169] (HELO mail-ie0-f169.google.com) (209.85.223.169) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Mar 2013 04:40:48 +0000 Received: by mail-ie0-f169.google.com with SMTP id qd14so6913268ieb.14 for ; Mon, 25 Mar 2013 21:40:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:content-type; bh=qnunnXRb+eROqnS3/HOtDk5860zFrtyUyYKajtX/GnY=; b=UtBYvU+cwsJZlJll4YxXiQ6RAHpMdDcMHtnlOxvZkcVYCYHNZhWf+EQi0Uzs5WYM1z 8mmpYFGX/4tQz6G6NYDfaqo8+keWYBm7JurSM4FKfTtbcwvM8giMe/zlUlRwGbSrQ5Ol XT2kKs1EIJ8mn0TItOKs8MiUY/9QO28dd9mNYV/ni/DkqCVrZ1pIuVMAp6MGKdK2dQqR eBSib7qdwGMyjS5nkM8kMNBTqJ9xQZz6kURiZSbPZLaBkUWCMlQTzP3dTWbS9KG9qyNp QOGMXMkvN/3FcBfmMrF7ex/z2/Y8/leQ90gwmrO1B8sQXV7dphyd8ExVMMt88DpD/Gsr ivwg== MIME-Version: 1.0 X-Received: by 10.42.145.137 with SMTP id f9mr8406466icv.52.1364272827939; Mon, 25 Mar 2013 21:40:27 -0700 (PDT) Received: by 10.64.26.70 with HTTP; Mon, 25 Mar 2013 21:40:27 -0700 (PDT) Received: by 10.64.26.70 with HTTP; Mon, 25 Mar 2013 21:40:27 -0700 (PDT) In-Reply-To: <1364221554.12562.6.camel@mblaptop> References: <1364221554.12562.6.camel@mblaptop> Date: Tue, 26 Mar 2013 12:40:27 +0800 Message-ID: Subject: Re: Hadoop-2.x native libraries From: Azuryy Yu To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=90e6ba6e8b504992f004d8cc8593 X-Virus-Checked: Checked by ClamAV on apache.org --90e6ba6e8b504992f004d8cc8593 Content-Type: text/plain; charset=ISO-8859-1 Its too complicated. because Hadoopv2 is wire compatable, so MRv1 should keep old RPC protocol, you can compile only common and HDFS firstly from hadoopv2, then add to MRv1 classpath, fix the issues for uncompatables. On Mar 25, 2013 10:26 PM, "Mounir E. Bsaibes" wrote: > Hi Azurry, > Do you have detail steps what did you do to make MRV1 work with HDFS2? > Thanks, > Mounir > On Mon, 2013-03-25 at 13:39 +0800, Azuryy Yu wrote: > > Thanks Harsh! > > I used -Pnative got it. > > I am compile src code. I made MRv1 work with HDFSv2 successfully. > > > > > > On Mar 25, 2013 12:56 PM, "Harsh J" wrote: > > If you're using a tarball, you'll need to build a native-added > > tarball > > yourself with "mvn package -Pdist,native,docs -DskipTests > > -Dtar" and > > then use that. > > > > Alternatively, if you're interested in packages, use the > > Apache > > Bigtop's scripts from http://bigtop.apache.org/ project's > > repository > > and generate the packages with native libs as well. > > > > On Mon, Mar 25, 2013 at 9:27 AM, Azuryy Yu > > wrote: > > > Hi, > > > How to get hadoop-2.0.3-alpha native libraries, it was > > compiled under 32bits > > > OS in the released package currently. > > > > > > > > > > > > > > -- > > Harsh J > > > --90e6ba6e8b504992f004d8cc8593 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

Its too complicated. because Hadoopv2 is wire compatable, so= MRv1 should keep old RPC protocol, you can compile only common and HDFS fi= rstly from hadoopv2, then add to MRv1 classpath, fix the issues for uncompa= tables.

On Mar 25, 2013 10:26 PM, "Mounir E. Bsaibe= s" <mb@linux.vnet.ibm.com<= /a>> wrote:
Hi Azurry,
Do you have detail steps what did you do to make MRV1 work with HDFS2?
Thanks,
Mounir
On Mon, 2013-03-25 at 13:39 +0800, Azuryy Yu wrote:
> Thanks Harsh!
> I used -Pnative got it.
> I am compile src code. I made MRv1 work with HDFSv2 successfully.
>
>
> On Mar 25, 2013 12:56 PM, "Harsh J" <harsh@cloudera.com> wrote:
> =A0 =A0 =A0 =A0 If you're using a tarball, you'll need to buil= d a native-added
> =A0 =A0 =A0 =A0 tarball
> =A0 =A0 =A0 =A0 yourself with "mvn package -Pdist,native,docs -Ds= kipTests
> =A0 =A0 =A0 =A0 -Dtar" and
> =A0 =A0 =A0 =A0 then use that.
>
> =A0 =A0 =A0 =A0 Alternatively, if you're interested in packages, u= se the
> =A0 =A0 =A0 =A0 Apache
> =A0 =A0 =A0 =A0 Bigtop's scripts from http://bigtop.apache.org/ project's
> =A0 =A0 =A0 =A0 repository
> =A0 =A0 =A0 =A0 and generate the packages with native libs as well. >
> =A0 =A0 =A0 =A0 On Mon, Mar 25, 2013 at 9:27 AM, Azuryy Yu
> =A0 =A0 =A0 =A0 <azuryyyu@gma= il.com> wrote:
> =A0 =A0 =A0 =A0 > Hi,
> =A0 =A0 =A0 =A0 > How to get hadoop-2.0.3-alpha native libraries, i= t was
> =A0 =A0 =A0 =A0 compiled under 32bits
> =A0 =A0 =A0 =A0 > OS in the released package currently.
> =A0 =A0 =A0 =A0 >
> =A0 =A0 =A0 =A0 >
>
>
>
> =A0 =A0 =A0 =A0 --
> =A0 =A0 =A0 =A0 Harsh J


--90e6ba6e8b504992f004d8cc8593--