Return-Path: X-Original-To: apmail-spark-user-archive@minotaur.apache.org Delivered-To: apmail-spark-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 62E4118AE5 for ; Fri, 9 Oct 2015 21:31:14 +0000 (UTC) Received: (qmail 45818 invoked by uid 500); 9 Oct 2015 21:31:10 -0000 Delivered-To: apmail-spark-user-archive@spark.apache.org Received: (qmail 45726 invoked by uid 500); 9 Oct 2015 21:31:10 -0000 Mailing-List: contact user-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@spark.apache.org Received: (qmail 45713 invoked by uid 99); 9 Oct 2015 21:31:10 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 09 Oct 2015 21:31:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 84BA0180EA5 for ; Fri, 9 Oct 2015 21:31:09 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.898 X-Spam-Level: ** X-Spam-Status: No, score=2.898 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id w922PtaiBmQv for ; Fri, 9 Oct 2015 21:31:08 +0000 (UTC) Received: from mail-qg0-f49.google.com (mail-qg0-f49.google.com [209.85.192.49]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 91AC8203BC for ; Fri, 9 Oct 2015 21:31:08 +0000 (UTC) Received: by qgez77 with SMTP id z77so80380214qge.1 for ; Fri, 09 Oct 2015 14:31:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=Is1fpZlyQsKBqr5xayE3JPbLGC8uUOMv5vIjdoXoytk=; b=xrDY1U0QvPy74QxrEZ2R2c3XtqcsfuOgWKK6bV7+LYfLx1t8t4dCs6PsBdsMfXba9w 8D9+cQBw6dgemsGtTT2/WdFnATJx4Y6Bfw/qQ3bwobCO+LDaseAQgRBMRVKWFW8OlL3w 4cH82tQf3E1TJqtwFUa5oIPwyn6GaetFFBOnZqGcGiBSbq0qlLC44hDSZXvyLvY6DE3B 8XrfhpVerDi6K3fzD8dTYfgCeFRYZL7+G/XaF7iCQyoUHWLSnR3ieFmqQaVlckjJdyZP QajlHwY1waf1/yntFoeuTkWcamXYmKtAj3mwrNEx2MAGo2ZZDrZrqCw+61+fAyipe8Ei C4jA== X-Received: by 10.140.151.72 with SMTP id 69mr19041980qhx.26.1444426267937; Fri, 09 Oct 2015 14:31:07 -0700 (PDT) Received: from mbp-4.fios-router.home (pool-108-49-55-3.bstnma.fios.verizon.net. [108.49.55.3]) by smtp.gmail.com with ESMTPSA id e51sm1493368qge.46.2015.10.09.14.31.06 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 09 Oct 2015 14:31:07 -0700 (PDT) Content-Type: multipart/alternative; boundary="Apple-Mail=_C2A63D2E-C4D1-4E33-A287-748FDE9BA3CC" Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: How to compile Spark with customized Hadoop? From: Matei Zaharia In-Reply-To: Date: Fri, 9 Oct 2015 17:31:06 -0400 Cc: user@spark.apache.org Message-Id: References: To: Dogtail L X-Mailer: Apple Mail (2.2104) --Apple-Mail=_C2A63D2E-C4D1-4E33-A287-748FDE9BA3CC Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii You can publish your version of Hadoop to your Maven cache with mvn = publish (just give it a different version number, e.g. 2.7.0a) and then = pass that as the Hadoop version to Spark's build (see = http://spark.apache.org/docs/latest/building-spark.html = ). Matei > On Oct 9, 2015, at 3:10 PM, Dogtail L wrote: >=20 > Hi all, >=20 > I have modified Hadoop source code, and I want to compile Spark with = my modified Hadoop. Do you know how to do that? Great thanks! --Apple-Mail=_C2A63D2E-C4D1-4E33-A287-748FDE9BA3CC Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii You can publish your version of Hadoop to your Maven cache = with mvn publish (just give it a different version number, e.g. 2.7.0a) = and then pass that as the Hadoop version to Spark's build (see http://spark.apache.org/docs/latest/building-spark.html).
Matei

On Oct 9, 2015, at 3:10 PM, Dogtail L <spark.rui92@gmail.com> wrote:

Hi all,

I = have modified Hadoop source code, and I want to compile Spark with my = modified Hadoop. Do you know how to do that? Great thanks!

= --Apple-Mail=_C2A63D2E-C4D1-4E33-A287-748FDE9BA3CC--