hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Haviv <daniel.ha...@veracity-group.com>
Subject simple usage of stack UDTF causes a cast exception
Date Sun, 10 Jan 2016 12:22:20 GMT
Hi,
I'm trying to break a row into two rows based on two different columns by
using the following query:

SELECT mystack.alias1
FROM cdrtable
LATERAL VIEW stack(2,  caller_IMEI, recipient_IMEI) mystack  AS alias1;


The exception I'm hitting is:

java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.LazyString
cannot be cast to org.apache.hadoop.io.Text


this is the table's DDL:
CREATE TABLE `cdrtable`(
  `ts` string,
  `caller_msisdn` string,
  `caller_imei` string,
  `caller_imsi` string,
  `caller_cell` string,
  `recipient_msisdn` string,
  `recipient_imei` string,
  `recipient_imsi` string,
  `recipient_cell` string,
  `call_type` string,
  `call_duration` string,
  `call_length` string)
ROW FORMAT DELIMITED
  FIELDS TERMINATED BY ','
STORED AS INPUTFORMAT
  'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'


The whole stack trace:
16/01/10 14:20:34 [main]: ERROR CliDriver: Failed with exception
java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.LazyString
cannot be cast to org.apache.hadoop.io.Text
java.io.IOException: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.LazyString
cannot be cast to org.apache.hadoop.io.Text
        at
org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:152)
        at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1655)
        at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:227)
        at
org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159)
        at
org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370)
        at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazy.LazyString
cannot be cast to org.apache.hadoop.io.Text
        at
org.apache.hadoop.hive.ql.exec.ListSinkOperator.processOp(ListSinkOperator.java:90)
        at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
        at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
        at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
        at
org.apache.hadoop.hive.ql.exec.LateralViewJoinOperator.processOp(LateralViewJoinOperator.java:133)
        at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
        at
org.apache.hadoop.hive.ql.exec.UDTFOperator.forwardUDTFOutput(UDTFOperator.java:125)
        at
org.apache.hadoop.hive.ql.udf.generic.UDTFCollector.collect(UDTFCollector.java:45)
        at
org.apache.hadoop.hive.ql.udf.generic.GenericUDTF.forward(GenericUDTF.java:107)
        at
org.apache.hadoop.hive.ql.udf.generic.GenericUDTFStack.process(GenericUDTFStack.java:123)
        at
org.apache.hadoop.hive.ql.exec.UDTFOperator.processOp(UDTFOperator.java:108)
        at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
        at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
        at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
        at
org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator.processOp(LateralViewForwardOperator.java:37)
        at
org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
        at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
        at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:424)
        at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:416)
        at
org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138)
        ... 13 more
Caused by: java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.lazy.LazyString cannot be cast to
org.apache.hadoop.io.Text
        at
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableStringObjectInspector.getPrimitiveWritableObject(WritableStringObjectInspector.java:41)
        at
org.apache.hadoop.hive.serde2.lazy.LazyUtils.writePrimitiveUTF8(LazyUtils.java:220)
        at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serialize(LazySimpleSerDe.java:306)
        at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.serializeField(LazySimpleSerDe.java:261)
        at
org.apache.hadoop.hive.serde2.DelimitedJSONSerDe.serializeField(DelimitedJSONSerDe.java:72)
        at
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.doSerialize(LazySimpleSerDe.java:245)
        at
org.apache.hadoop.hive.serde2.AbstractEncodingAwareSerDe.serialize(AbstractEncodingAwareSerDe.java:50)
        at
org.apache.hadoop.hive.ql.exec.DefaultFetchFormatter.convert(DefaultFetchFormatter.java:71)
        at
org.apache.hadoop.hive.ql.exec.DefaultFetchFormatter.convert(DefaultFetchFormatter.java:40)
        at
org.apache.hadoop.hive.ql.exec.ListSinkOperator.processOp(ListSinkOperator.java:87)
        ... 32 more

Any idea why ?

Thank you.
Daniel

Mime
View raw message