hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aviral Agarwal <aviral12...@gmail.com>
Subject Re: ORC Transaction Table - Spark
Date Wed, 23 Aug 2017 07:24:14 GMT
Hi,

Yes it caused by wrong naming convention of the delta directory :

/apps/hive/warehouse/foo.db/bar/year=2017/month=5/delta_0645253_0645253_0001

How do I solve this ?

Thanks !
Aviral Agarwal

On Tue, Aug 22, 2017 at 11:50 PM, Eugene Koifman <ekoifman@hortonworks.com>
wrote:

> Could you do recursive “ls” in your table or partition that you are trying
> to read?
>
> Most likely you have files that don’t follow expected naming convention
>
>
>
> Eugene
>
>
>
>
>
> *From: *Aviral Agarwal <aviral12028@gmail.com>
> *Reply-To: *"user@hive.apache.org" <user@hive.apache.org>
> *Date: *Tuesday, August 22, 2017 at 5:39 AM
> *To: *"user@hive.apache.org" <user@hive.apache.org>
> *Subject: *ORC Transaction Table - Spark
>
>
>
> Hi,
>
>
>
> I am trying to read hive orc transaction table through Spark but I am
> getting the following error
>
>
> Caused by: java.lang.RuntimeException: serious problem
> at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(
> OrcInputFormat.java:1021)
> at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(
> OrcInputFormat.java:1048)
> at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:202)
> .....
> Caused by: java.util.concurrent.ExecutionException: java.lang.NumberFormatException:
> For input string: "0645253_0001"
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(
> OrcInputFormat.java:998)
> ... 118 more
>
>
> Any help would be appreciated.
>
> Thanks and Regards,
> Aviral Agarwal
>

Mime
View raw message