Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CF47619D38 for ; Sat, 26 Mar 2016 22:09:13 +0000 (UTC) Received: (qmail 91115 invoked by uid 500); 26 Mar 2016 22:09:12 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 91033 invoked by uid 500); 26 Mar 2016 22:09:12 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 91023 invoked by uid 99); 26 Mar 2016 22:09:12 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 26 Mar 2016 22:09:12 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id AB916C0217 for ; Sat, 26 Mar 2016 22:09:11 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.18 X-Spam-Level: * X-Spam-Status: No, score=1.18 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, MIME_QP_LONG_LINE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd4-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id uBEVMssFGkla for ; Sat, 26 Mar 2016 22:09:09 +0000 (UTC) Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com [74.125.82.68]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with ESMTPS id 2D0E05F282 for ; Sat, 26 Mar 2016 22:09:09 +0000 (UTC) Received: by mail-wm0-f68.google.com with SMTP id 139so6129831wmn.2 for ; Sat, 26 Mar 2016 15:09:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=bj1nKnKnqG1ufNCbm8jdlOR26c1AUcEvaCLsnB/SqMU=; b=uGxvPM0gXFRbcs1UX0MxNYCp6HXfMS25WoTS4PsLkcGe3sQDbAh/ZLAx/YEzwjowUx dFw+fNMfCrAc46ffX6xUdReWrs1czc5VEq033PVBLGFHPz353rsuteE/HYRMj2XQYimK 7vzhGmStuUQFAkkFRuEAo2nWmngO8GdTCEizAfw7CzH7WP51SE8slFiqiDkweemL8wl1 JP/yUg5jxVfHOo0BdhnHytGVUnpOnpl1g+080JrfZF/1BrEoBp5p8lwHiSz1wXBetr3r hNa5qe67Syu+5RgfwqTkiVqCZXHFuyG2JwZqJztYYW9wM9rxM0V5vUeFaKb7MnB6rqi9 zc2Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=bj1nKnKnqG1ufNCbm8jdlOR26c1AUcEvaCLsnB/SqMU=; b=eAShaYqj+GcnQEvaOtccpAFXOV9JaFSpZUJaQgaOTrdIOSVUgjYaPCDtQXBBzyJjKz ezvQT/vIuroi7GvycTCM+qmD4HyDabq5Li7U9vO1eevqFehUII9nK02+gzx9dEmNopqP oYBZewSGHyEylOYVXix2qqAsz1TFOIjm3WBqJ1zsHtHloS+j2V0bXSVTlVcSa4mfsfD5 J+mYloKMdbjOfH0+pVjpG+xVFlddNU+RzmeDuqG4Nal1in5Anff8trttXw2RYroyOtTQ G8eCQxNEKz5hHcAZpV9hkbCHb9uBWsAY01/4180JF4sH1v2EFpJGcf7u98O6q8u1w0cZ rvGQ== X-Gm-Message-State: AD7BkJJhcsCCrtxf1rgOr5X/U/kZvhIcEljA/p58WWqFtK1zeXHsGzss8grOhgYd8vGLKA== X-Received: by 10.194.134.134 with SMTP id pk6mr21424738wjb.176.1459030142308; Sat, 26 Mar 2016 15:09:02 -0700 (PDT) Received: from [10.218.53.143] ([176.0.102.53]) by smtp.gmail.com with ESMTPSA id 82sm3199548wmd.4.2016.03.26.15.09.00 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 26 Mar 2016 15:09:00 -0700 (PDT) Content-Type: multipart/alternative; boundary=Apple-Mail-BEF245CE-1477-4FD9-931E-BC220F411644 Mime-Version: 1.0 (1.0) Subject: Re: Hive on Spark engine From: =?utf-8?Q?J=C3=B6rn_Franke?= X-Mailer: iPhone Mail (13E234) In-Reply-To: Date: Sat, 26 Mar 2016 23:08:58 +0100 Cc: "user @spark" , user Content-Transfer-Encoding: 7bit Message-Id: References: To: Mich Talebzadeh --Apple-Mail-BEF245CE-1477-4FD9-931E-BC220F411644 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable If you check the newest Hortonworks distribution then you see that it genera= lly works. Maybe you can borrow some of their packages. Alternatively it sho= uld be also available in other distributions. > On 26 Mar 2016, at 22:47, Mich Talebzadeh wrot= e: >=20 > Hi, >=20 > I am running Hive 2 and now Spark 1.6.1 but I still do not see any sign th= at Hive can utilise a Spark engine higher than 1.3.1 >=20 > My understanding was that there were miss-match on Hadoop assembly Jar fil= es that cause Hive not being able to run on Spark using the binary downloads= . I just tried Hive 2 on Spark 1.6 as the execution engine and it crashed. >=20 > I do not know the development state of this cross-breed but will be very d= esirable if we could manage to sort out this spark-assembly-1.x.1-hadoop2.4.= 0.jar for once. >=20 > Thanks >=20 >=20 > Dr Mich Talebzadeh > =20 > LinkedIn https://www.linkedin.com/profile/view?id=3DAAEAAAAWh2gBxianrbJd6= zP6AcPCCdOABUrV8Pw > =20 > http://talebzadehmich.wordpress.com > =20 --Apple-Mail-BEF245CE-1477-4FD9-931E-BC220F411644 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
If you check the newest Hortonworks di= stribution then you see that it generally works. Maybe you can borrow some o= f their packages. Alternatively it should be also available in other distrib= utions.

On 26 Mar 2016, at 22:47, Mich Talebzadeh <mich.talebzadeh@gmail.com> wrote:<= br>
Hi,
<= div>
I am running Hive 2 and now Spark 1.6.1 but I still do no= t see any sign that Hive can utilise a Spark engine higher than 1.3.1
<= div>
My understanding was that there were miss-match on Hadoop= assembly Jar files that cause Hive not being able to run on Spark using the= binary downloads. I just tried Hive 2 on Spark 1.6 as the execution engine a= nd it crashed.

I do not know the development state o= f this cross-breed but will be very desirable if we could manage to sort out= this spark-assembly-1.x.1-hadoop2.4.0.jar for once.
= --Apple-Mail-BEF245CE-1477-4FD9-931E-BC220F411644--