spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-21639) Getting an error while installing spark on windows
Date Fri, 04 Aug 2017 09:46:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-21639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Owen resolved SPARK-21639.
-------------------------------
    Resolution: Invalid

JIRA isn't really a place for help questions. It sounds like you have some environment problem.
You haven't said how you built or ran Spark. If you can narrow it down to a problem reproducible
from a release then you can reopen this with those details.

> Getting an error while installing spark on windows
> --------------------------------------------------
>
>                 Key: SPARK-21639
>                 URL: https://issues.apache.org/jira/browse/SPARK-21639
>             Project: Spark
>          Issue Type: IT Help
>          Components: Spark Shell
>    Affects Versions: 1.6.1
>            Reporter: sandhya harane
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> Microsoft Windows [Version 6.1.7601]
> Copyright (c) 2009 Microsoft Corporation.  All rights reserved.
> C:\Users\sandhyah>spark-shell
> log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.li
> b.MutableMetricsFactory).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more in
> fo.
> Using Spark's repl log4j profile: org/apache/spark/log4j-defaults-repl.propertie
> s
> To adjust logging level use sc.setLogLevel("INFO")
> Welcome to
>       ____              __
>      / __/__  ___ _____/ /__
>     _\ \/ _ \/ _ `/ __/  '_/
>    /___/ .__/\_,_/_/ /_/\_\   version 1.6.1
>       /_/
> Using Scala version 2.10.5 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_144)
> Type in expressions to have them evaluated.
> Type :help for more information.
> Spark context available as sc.
> 17/08/04 15:01:38 WARN General: Plugin (Bundle) "org.datanucleus" is already reg
> istered. Ensure you dont have multiple JAR versions of the same plugin in the cl
> asspath. The URL "file:/E:/spark/spark-1.6.1-bin-hadoop2.6/bin/../lib/datanucleu
> s-core-3.2.10.jar" is already registered, and you are trying to register an iden
> tical plugin located at URL "file:/E:/spark/spark-1.6.1-bin-hadoop2.6/lib/datanu
> cleus-core-3.2.10.jar."
> 17/08/04 15:01:38 WARN General: Plugin (Bundle) "org.datanucleus.api.jdo" is alr
> eady registered. Ensure you dont have multiple JAR versions of the same plugin i
> n the classpath. The URL "file:/E:/spark/spark-1.6.1-bin-hadoop2.6/bin/../lib/da
> tanucleus-api-jdo-3.2.6.jar" is already registered, and you are trying to regist
> er an identical plugin located at URL "file:/E:/spark/spark-1.6.1-bin-hadoop2.6/
> lib/datanucleus-api-jdo-3.2.6.jar."
> 17/08/04 15:01:38 WARN General: Plugin (Bundle) "org.datanucleus.store.rdbms" is
>  already registered. Ensure you dont have multiple JAR versions of the same plug
> in in the classpath. The URL "file:/E:/spark/spark-1.6.1-bin-hadoop2.6/lib/datan
> ucleus-rdbms-3.2.9.jar" is already registered, and you are trying to register an
>  identical plugin located at URL "file:/E:/spark/spark-1.6.1-bin-hadoop2.6/bin/.
> ./lib/datanucleus-rdbms-3.2.9.jar."
> 17/08/04 15:01:39 WARN Connection: BoneCP specified but not present in CLASSPATH
>  (or one of dependencies)
> 17/08/04 15:01:39 WARN Connection: BoneCP specified but not present in CLASSPATH
>  (or one of dependencies)
> 17/08/04 15:01:42 WARN ObjectStore: Version information not found in metastore.
> hive.metastore.schema.verification is not enabled so recording the schema versio
> n 1.2.0
> 17/08/04 15:01:42 WARN ObjectStore: Failed to get database default, returning No
> SuchObjectException
> java.lang.RuntimeException: java.lang.NullPointerException
>         at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav
> a:522)
>         at org.apache.spark.sql.hive.client.ClientWrapper.<init>(ClientWrapper.s
> cala:204)
>         at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(Is
> olatedClientLoader.scala:238)
>         at org.apache.spark.sql.hive.HiveContext.executionHive$lzycompute(HiveCo
> ntext.scala:218)
>         at org.apache.spark.sql.hive.HiveContext.executionHive(HiveContext.scala
> :208)
>         at org.apache.spark.sql.hive.HiveContext.functionRegistry$lzycompute(Hiv
> eContext.scala:462)
>         at org.apache.spark.sql.hive.HiveContext.functionRegistry(HiveContext.sc
> ala:461)
>         at org.apache.spark.sql.UDFRegistration.<init>(UDFRegistration.scala:40)
>         at org.apache.spark.sql.SQLContext.<init>(SQLContext.scala:330)
>         at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:90)
>         at org.apache.spark.sql.hive.HiveContext.<init>(HiveContext.scala:101)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstruct
> orAccessorImpl.java:62)
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingC
> onstructorAccessorImpl.java:45)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>         at org.apache.spark.repl.SparkILoop.createSQLContext(SparkILoop.scala:10
> 28)
>         at $iwC$$iwC.<init>(<console>:15)
>         at $iwC.<init>(<console>:24)
>         at <init>(<console>:26)
>         at .<init>(<console>:30)
>         at .<clinit>(<console>)
>         at .<init>(<console>:7)
>         at .<clinit>(<console>)
>         at $print(<console>)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
> java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> sorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:
> 1065)
>         at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:
> 1346)
>         at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840
> )
>         at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871)
>         at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819)
>         at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:8
> 57)
>         at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.sca
> la:902)
>         at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:814)
>         at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply
> (SparkILoopInit.scala:132)
>         at org.apache.spark.repl.SparkILoopInit$$anonfun$initializeSpark$1.apply
> (SparkILoopInit.scala:124)
>         at org.apache.spark.repl.SparkIMain.beQuietDuring(SparkIMain.scala:324)
>         at org.apache.spark.repl.SparkILoopInit$class.initializeSpark(SparkILoop
> Init.scala:124)
>         at org.apache.spark.repl.SparkILoop.initializeSpark(SparkILoop.scala:64)
>         at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$Spark
> ILoop$$process$1$$anonfun$apply$mcZ$sp$5.apply$mcV$sp(SparkILoop.scala:974)
>         at org.apache.spark.repl.SparkILoopInit$class.runThunks(SparkILoopInit.s
> cala:159)
>         at org.apache.spark.repl.SparkILoop.runThunks(SparkILoop.scala:64)
>         at org.apache.spark.repl.SparkILoopInit$class.postInitialization(SparkIL
> oopInit.scala:108)
>         at org.apache.spark.repl.SparkILoop.postInitialization(SparkILoop.scala:
> 64)
>         at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$Spark
> ILoop$$process$1.apply$mcZ$sp(SparkILoop.scala:991)
>         at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$Spark
> ILoop$$process$1.apply(SparkILoop.scala:945)
>         at org.apache.spark.repl.SparkILoop$$anonfun$org$apache$spark$repl$Spark
> ILoop$$process$1.apply(SparkILoop.scala:945)
>         at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClass
> Loader.scala:135)
>         at org.apache.spark.repl.SparkILoop.org$apache$spark$repl$SparkILoop$$pr
> ocess(SparkILoop.scala:945)
>         at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:1059)
>         at org.apache.spark.repl.Main$.main(Main.scala:31)
>         at org.apache.spark.repl.Main.main(Main.scala)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
> java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> sorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSub
> mit$$runMain(SparkSubmit.scala:731)
>         at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:18
> 1)
>         at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
>         at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
>         at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> Caused by: java.lang.NullPointerException
>         at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:482)
>         at org.apache.hadoop.util.Shell.run(Shell.java:455)
>         at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:
> 715)
>         at org.apache.hadoop.util.Shell.execCommand(Shell.java:808)
>         at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
>         at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097)
>         at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
> loadPermissionInfo(RawLocalFileSystem.java:582)
>         at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.
> getPermission(RawLocalFileSystem.java:557)
>         at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(Sess
> ionState.java:599)
>         at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(Sess
> ionState.java:554)
>         at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.jav
> a:508)
>         ... 62 more
> <console>:16: error: not found: value sqlContext
>          import sqlContext.implicits._
>                 ^
> <console>:16: error: not found: value sqlContext
>          import sqlContext.sql
>                 ^
> scala> conf.set("spark.sql.hive.thriftServer.singleSession", "true")
> <console>:20: error: not found: value conf
>               conf.set("spark.sql.hive.thriftServer.singleSession", "true")
>               ^
> scala>



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message