phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Mahonin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-4056) java.lang.IllegalArgumentException: Can not create a Path from an empty string
Date Thu, 08 Feb 2018 16:39:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357183#comment-16357183
] 

Josh Mahonin commented on PHOENIX-4056:
---------------------------------------

Thanks [~stepson] for updating the patch and checking the backwards compatibility.

One nit, the "mapred.output.dir" property appears to be deprecated [1], can I suggest using
the updated property "mapreduce.output.fileoutputformat.outputdir" instead (which hortonworks-spark
has also done [2])? Or possibly chain them together?



e.g.
{noformat}
Option(
  conf.get("mapreduce.output.fileoutputformat.outputdir") 
).getOrElse(
  Option(conf.get("mapred.output.dir")
).getOrElse(""){noformat}
 

[1] [https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/DeprecatedProperties.html
[|https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/DeprecatedProperties.html]2]
[https://github.com/hortonworks-spark/shc/pull/167/files]

 

> java.lang.IllegalArgumentException: Can not create a Path from an empty string
> ------------------------------------------------------------------------------
>
>                 Key: PHOENIX-4056
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4056
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.11.0
>         Environment: CDH5.12
> Phoenix:4.11
> HBase:1.2
> Spark: 2.2.0
> phoenix-spark.version:4.11.0-HBase-1.2
>            Reporter: Jepson
>            Priority: Major
>              Labels: features, patch, test
>         Attachments: PHOENIX-4056.patch, PHOENIX-4056_v2.patch
>
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> 1.use the configuration of server and client(scala project)
>  <property>
>     <name>phoenix.schema.isNamespaceMappingEnabled</name>
>     <value>true</value>
>   </property>
>   <property>
>     <name>phoenix.schema.mapSystemTablesToNamespace</name>
>     <value>true</value>
>   </property>
> 2.The Code:
> {code:java}
>     resultDF.write
>          .format("org.apache.phoenix.spark")
>          .mode(SaveMode.Overwrite)
>          .option("table", "JYDW.ADDRESS_ORDERCOUNT")
>          .option("zkUrl","192.168.1.40,192.168.1.41,192.168.1.42:2181")
>          .save()
> {code}
> 3.Throw this error,help to fix it,thankyou :
> 7/08/02 01:07:25 INFO DAGScheduler: Job 6 finished: runJob at SparkHadoopMapReduceWriter.scala:88,
took 7.990715 s
> 17/08/02 01:07:25 ERROR SparkHadoopMapReduceWriter: Aborting job job_20170802010717_0079.
> {color:#59afe1}*java.lang.IllegalArgumentException: Can not create a Path from an empty
string*{color}
> 	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:126)
> 	at org.apache.hadoop.fs.Path.<init>(Path.java:134)
> 	at org.apache.hadoop.fs.Path.<init>(Path.java:88)
> 	at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.absPathStagingDir(HadoopMapReduceCommitProtocol.scala:58)
> 	at org.apache.spark.internal.io.HadoopMapReduceCommitProtocol.commitJob(HadoopMapReduceCommitProtocol.scala:132)
> 	at org.apache.spark.internal.io.SparkHadoopMapReduceWriter$.write(SparkHadoopMapReduceWriter.scala:101)
> 	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply$mcV$sp(PairRDDFunctions.scala:1085)
> 	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> 	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1.apply(PairRDDFunctions.scala:1085)
> 	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> 	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> 	at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> 	at org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopDataset(PairRDDFunctions.scala:1084)
> 	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply$mcV$sp(PairRDDFunctions.scala:1003)
> 	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> 	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopFile$2.apply(PairRDDFunctions.scala:994)
> 	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> 	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
> 	at org.apache.spark.rdd.RDD.withScope(RDD.scala:362)
> 	at org.apache.spark.rdd.PairRDDFunctions.saveAsNewAPIHadoopFile(PairRDDFunctions.scala:994)
> 	at org.apache.phoenix.spark.DataFrameFunctions.saveToPhoenix(DataFrameFunctions.scala:59)
> 	at org.apache.phoenix.spark.DataFrameFunctions.saveToPhoenix(DataFrameFunctions.scala:28)
> 	at org.apache.phoenix.spark.DefaultSource.createRelation(DefaultSource.scala:47)
> 	at org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:472)
> 	at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:48)
> 	at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
> 	at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
> 	at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:74)
> 	at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:117)
> 	at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:117)
> 	at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:138)
> 	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
> 	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:135)
> 	at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:116)
> 	at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:92)
> 	at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:92)
> 	at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:610)
> 	at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:233)
> 	at com.jiuye.SparkPhoenixTest2$.main(SparkPhoenixTest2.scala:62)
> 	at com.jiuye.SparkPhoenixTest2.main(SparkPhoenixTest2.scala)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
> Exception in thread "main" java.lang.IllegalArgumentException: Can not create a Path
from an empty string
> 	at org.apache.hadoop.fs.Path.checkPathArg(Path.java:126)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message