hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <>
Subject [jira] [Commented] (HIVE-5431) PassthroughOutputFormat SH changes causes IllegalArgumentException
Date Sat, 05 Oct 2013 00:26:42 GMT


Sushanth Sowmyan commented on HIVE-5431:

[~thejas] & [~ashutoshc] : Writing a test for this proves to be difficult because it can't
be written as a .q test, and instead requires me to create a hbase table outside of hive and
then read using hive in a job to demonstrate the bug. I've manually tested this bug and this

Would you prefer to spin 0.12 RC0 without this patch, and then add it along with a testcase
at RC1 timeframe, or would you prefer to take this patch now and open another jira for a test?
I'm okay with either approach, and will assume the former and leave this open. If you prefer
the latter, then please feel free to close/commit and open another jira for that. I'm going
to go ahead and mark it as patch-available, though, so that the automated tests at least pick
it up and run a full run with this patch.

> PassthroughOutputFormat SH changes causes IllegalArgumentException
> ------------------------------------------------------------------
>                 Key: HIVE-5431
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.12.0
>            Reporter: Sushanth Sowmyan
>            Assignee: Sushanth Sowmyan
>         Attachments: HIVE-5431.2.patch, HIVE-5431.patch
> The recent changes with HIVE-4331 introduced a new key "hive.passthrough.storagehandler.of",
whose value is set only on storage handler writes, but obviously, will not be set on reads.
However, PlanUtils.configureJobPropertiesForStorageHandler winds up trying to set the key
for both cases into jobProperties, which cause any reads that are not preceeded by writes
to fail.
> Basically, if you have a .q in which you insert data into a hbase table and then read
it, it's okay. If you have a .q in which you only read data, it throws an IllegalArgumentException,
like so:
> {noformat}
> 2013-09-30 16:20:01,989 ERROR CliDriver ( - Failed
with exception Property value must
not be null
> java.lang.IllegalArgumentException: Property value must not be null
>         at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(
>         at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(
>         at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(
>         at org.apache.hadoop.hive.ql.Driver.getResults(
>         at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(
>         at org.apache.hadoop.hive.cli.CliDriver.processCmd(
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(
>         at org.apache.hadoop.hive.cli.CliDriver.processReader(
>         at org.apache.hadoop.hive.cli.CliDriver.processFile(
>         at org.apache.hadoop.hive.cli.CliDriver.executeDriver(
>         at
>         at org.apache.hadoop.hive.cli.CliDriver.main(
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>         at java.lang.reflect.Method.invoke(
>         at org.apache.hadoop.util.RunJar.main(
> Caused by: java.lang.IllegalArgumentException: Property value must not be null
>         at
>         at org.apache.hadoop.conf.Configuration.set(
>         at org.apache.hadoop.conf.Configuration.set(
>         at org.apache.hadoop.hive.ql.exec.Utilities.copyTableJobPropertiesToConf(
>         at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(
>         at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(
>         ... 17 more
> {noformat}

This message was sent by Atlassian JIRA

View raw message