Mailing-List: contact issues-help@spark.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 23 Feb 2015 09:39:11 +0000 (UTC)
From: "Alexander Bezzubov (JIRA)" <jira@apache.org>
To: issues@spark.apache.org
Message-ID: <JIRA.12753590.1415324859000.111077.1424684351841@Atlassian.JIRA>
In-Reply-To: <JIRA.12753590.1415324859000@Atlassian.JIRA>
References: <JIRA.12753590.1415324859000@Atlassian.JIRA>
 <JIRA.12753590.1415324859892@arcas>
Subject: [jira] [Comment Edited] (SPARK-4289) Creating an instance of Hadoop
 Job fails in the Spark shell when toString() is called on the instance.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/SPARK-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14333142#comment-14333142 ] 

Alexander Bezzubov edited comment on SPARK-4289 at 2/23/15 9:39 AM:
--------------------------------------------------------------------

[~sowen] Thanks, that's what I though, but it results in exactly same exception and does not mute any interpreter output on the fresh built spark master with -Phadoop2.4
Am I doing something wrong?

Uneducated guess: is that something to do with [SparkILoop . verbosity()|https://github.com/apache/spark/blob/16687651f05bde8ff2e2fcef100383168958bf7f/repl/scala-2.10/src/main/scala/org/apache/spark/repl/SparkILoop.scala#L796] impl?

Looks like another workaround is not to assign it to any variable, as advised in [@user thread|http://markmail.org/message/x77s57w47homqn6x] 


was (Author: bzz):
[~sowen] Thanks, that's what I though, but it results in exactly same exception and does not mute any interpreter output on the fresh built spark master with -Phadoop2.4
Am I doing something wrong?

Looks like another workaround is not to assign it to any variable, as advised in [@user thread|http://markmail.org/message/x77s57w47homqn6x] 

> Creating an instance of Hadoop Job fails in the Spark shell when toString() is called on the instance.
> ------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-4289
>                 URL: https://issues.apache.org/jira/browse/SPARK-4289
>             Project: Spark
>          Issue Type: Bug
>            Reporter: Corey J. Nolet
>
> This one is easy to reproduce.
> <pre>val job = new Job(sc.hadoopConfiguration)</pre>
> I'm not sure what the solution would be off hand as it's happening when the shell is calling toString() on the instance of Job. The problem is, because of the failure, the instance is never actually assigned to the job val.
> java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
> 	at org.apache.hadoop.mapreduce.Job.ensureState(Job.java:283)
> 	at org.apache.hadoop.mapreduce.Job.toString(Job.java:452)
> 	at scala.runtime.ScalaRunTime$.scala$runtime$ScalaRunTime$$inner$1(ScalaRunTime.scala:324)
> 	at scala.runtime.ScalaRunTime$.stringOf(ScalaRunTime.scala:329)
> 	at scala.runtime.ScalaRunTime$.replStringOf(ScalaRunTime.scala:337)
> 	at .<init>(<console>:10)
> 	at .<clinit>(<console>)
> 	at $print(<console>)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:789)
> 	at org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1062)
> 	at org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:615)
> 	at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:646)
> 	at org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:610)
> 	at org.apache.spark.repl.SparkILoop.reallyInterpret$1(SparkILoop.scala:814)
> 	at org.apache.spark.repl.SparkILoop.interpretStartingWith(SparkILoop.scala:859)
> 	at org.apache.spark.repl.SparkILoop.command(SparkILoop.scala:771)
> 	at org.apache.spark.repl.SparkILoop.processLine$1(SparkILoop.scala:616)
> 	at org.apache.spark.repl.SparkILoop.innerLoop$1(SparkILoop.scala:624)
> 	at org.apache.spark.repl.SparkILoop.loop(SparkILoop.scala:629)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply$mcZ$sp(SparkILoop.scala:954)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:902)
> 	at org.apache.spark.repl.SparkILoop$$anonfun$process$1.apply(SparkILoop.scala:902)
> 	at scala.tools.nsc.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:135)
> 	at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:902)
> 	at org.apache.spark.repl.SparkILoop.process(SparkILoop.scala:997)
> 	at org.apache.spark.repl.Main$.main(Main.scala:31)
> 	at org.apache.spark.repl.Main.main(Main.scala)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:328)
> 	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
> 	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org