Mailing-List: contact issues-help@spark.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 23 Aug 2016 12:42:20 +0000 (UTC)
From: "Sean Owen (JIRA)" <jira@apache.org>
To: issues@spark.apache.org
Message-ID: <JIRA.12998136.1471509936000.382789.1471956140986@Atlassian.JIRA>
In-Reply-To: <JIRA.12998136.1471509936000@Atlassian.JIRA>
References: <JIRA.12998136.1471509936000@Atlassian.JIRA> <JIRA.12998136.1471509936050@arcas>
Subject: [jira] [Commented] (SPARK-17126) Errors setting driver classpath in
 spark-defaults.conf on Windows 7
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Tue, 23 Aug 2016 12:42:23 -0000


    [ https://issues.apache.org/jira/browse/SPARK-17126?page=3Dcom.atlassia=
n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D154=
32715#comment-15432715 ]=20

Sean Owen commented on SPARK-17126:
-----------------------------------

Hm, I am not sure that "*" works on any JVM. Maybe I'm missing a reason it =
works for the env variable. But you would in general not specify it this wa=
y, which could be the problem. You would also not in general set app jar de=
pendencies this way, but rather build them into your app.

> Errors setting driver classpath in spark-defaults.conf on Windows 7
> -------------------------------------------------------------------
>
>                 Key: SPARK-17126
>                 URL: https://issues.apache.org/jira/browse/SPARK-17126
>             Project: Spark
>          Issue Type: Question
>          Components: Spark Shell, SQL
>    Affects Versions: 1.6.1
>         Environment: Windows 7
>            Reporter: Ozioma Ihekwoaba
>
> I am having issues starting up Spark shell with a local hive-site.xml on =
Windows 7.
> I have a local Hive 2.1.0 instance on Windows using a MySQL metastore.
> The Hive instance is working fine.
> I copied over the hive-site.xml to my local instance of Spark 1.6.1 conf =
folder and also copied over mysql-connector-java-5.1.25-bin.jar to the lib =
folder.
> I was expecting Spark to pick up jar files in the lib folder automaticall=
y, but found out Spark expects a spark.driver.extraC=E2=80=8C=E2=80=8BlassP=
ath and spark.executor.extraClassPath settings to resolve jars.
> Thing is this has failed on Windows for me with a DataStoreDriverNotFound=
Exception saying com.mysql.jdbc.Driver could not be found.
> Here are some of the different file paths I've tried:
> C:/hadoop/spark/v161/lib/mysql-connector-java-5.1.25-bin.jar;C:/hadoop/sp=
ark/v161/lib/commons-csv-1.4.jar;C:/hadoop/spark/v161/lib/spark-csv_2.11-1.=
4.0.jar
> ".;C:\hadoop\spark\v161\lib\*"
> ....NONE has worked so far.
> Please, what is the correct way to set driver classpaths on Windows?
> Also, what is the correct file path format on Windows?
> I have it working fine on Linux but my current engagement requires me to =
run Spark on a Windows box.
> Is there a way for Spark to automatically resolve jars from the lib folde=
r in all modes?
> Thanks.
> Ozzy


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org