hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <>
Subject [jira] [Commented] (HIVE-4804) parallel_orderby.q in trunk fails consistently
Date Sat, 06 Jul 2013 01:31:49 GMT


Ashutosh Chauhan commented on HIVE-4804:

I apologize for being late on this. I didn't understand the fix completely. Hopefully, @navis
or [~appodictic] can explain. This testcase was working initially. Both Navis and Ed did run
this test before checking in. Than test started to fail randomly. In various runs on apache
build machine it used to pass some time and fail at other time. So, there is some randomness
going on.
Per my understanding, fix in this patch is to restore number of reducers to 1 in case there
are not sufficient number of samples. What I don't understand is how come this scenario of
too few samples occur only randomly, shouldn't that be always the case for the test. What
further worries me was we were generating wrong results (order by clause was getting ignored
before this fix). Are we now guaranteed to never giving out wrong results to end users?
> parallel_orderby.q in trunk fails consistently
> ----------------------------------------------
>                 Key: HIVE-4804
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Query Processor
>            Reporter: Navis
>            Assignee: Navis
>             Fix For: 0.12.0
>         Attachments: HIVE-4804.D11571.1.patch
> {noformat}
> java.lang.RuntimeException: Error in configuring object
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(
> 	at org.apache.hadoop.util.ReflectionUtils.setConf(
> 	at org.apache.hadoop.util.ReflectionUtils.newInstance(
> 	at org.apache.hadoop.mapred.MapTask$OldOutputCollector.<init>(
> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(
> 	at
> 	at org.apache.hadoop.mapred.Child$
> 	at Method)
> 	at
> 	at
> 	at org.apache.hadoop.mapred.Child.main(
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> 	at java.lang.reflect.Method.invoke(
> 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(
> 	... 10 more
> Caused by: java.lang.IllegalArgumentException: Can't read partitions file
> 	at org.apache.hadoop.mapred.lib.TotalOrderPartitioner.configure(
> 	at org.apache.hadoop.hive.ql.exec.HiveTotalOrderPartitioner.configure(
> 	... 15 more
> Caused by: Split points are out of order
> 	at org.apache.hadoop.mapred.lib.TotalOrderPartitioner.configure(
> 	... 16 more
> {noformat}

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message