hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhenxiao Luo (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-2782) New BINARY type produces unexpected results with supported UDFS when using MapReduce2
Date Wed, 08 Feb 2012 23:33:01 GMT

    [ https://issues.apache.org/jira/browse/HIVE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13204109#comment-13204109
] 

Zhenxiao Luo commented on HIVE-2782:
------------------------------------

@carl: No problem with the original patch.

@Ashutosh:

My understanding is,

The original query:

select
length (cast(src.key as binary)) as len,
concat(cast(src.key as binary), cast(src.value as binary)),
substr(src.value, 5,1) as sub
from src
order by sub
limit 10;

Has the order by on sub, which is the fifth character of src.value.

In the result set, we could see sub is ordered(only showing sub == 1)

While, for the len and concat with the same sub, their orders are not deterministic.

Eg. 

2	10val_10	1
3	120val_120	1

both has sub == 1. but the order of these two tuples is not deterministic.
since we only show the top 10 results(limit 10), and there seems are more than 7 tuples with
sub == 1, so what shows in the result set is non-deterministic.

Carl solves the problem by providing order into the query.

Correct me if I am wrong.

Thanks
                
> New BINARY type produces unexpected results with supported UDFS when using MapReduce2
> -------------------------------------------------------------------------------------
>
>                 Key: HIVE-2782
>                 URL: https://issues.apache.org/jira/browse/HIVE-2782
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhenxiao Luo
>            Assignee: Carl Steinbach
>         Attachments: HIVE-2782.D1653.1.patch, HIVE-2782.D1653.1.patch
>
>
> When using MapReduce2 for Hive
> ba_table_udfs is failing with unexpected output:
> [junit] Begin query: ba_table_udfs.q
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.system.dir is deprecated. Instead,
use mapreduce.jobtracker.system.dir
> [junit] 12/01/23 13:32:28 WARN conf.Configuration: mapred.local.dir is deprecated. Instead,
use mapreduce.cluster.local.dir
> [junit] diff -a -I file: -I pfile: -I hdfs: -I /tmp/ -I invalidscheme: -I lastUpdateTime
-I lastAccessTime -I [Oo]wner -I CreateTime -I LastAccessTime -I Location -I LOCATION ' -I
transient_lastDdlTime -I last_modified_ -I java.lang.RuntimeException -I at org -I at sun
-I at java -I at junit -I Caused by: -I LOCK_QUERYID: -I LOCK_TIME: -I grantTime -I [.][.][.]
[0-9]* more -I job_[0-9]*_[0-9]* -I USING 'java -cp /home/cloudera/Code/hive/build/ql/test/logs/clientpositive/ba_table_udfs.q.out
/home/cloudera/Code/hive/ql/src/test/results/clientpositive/ba_table_udfs.q.out
> [junit] 20,26c20,26
> [junit] < 2	10val_10	1
> [junit] < 3	164val_164	1
> [junit] < 3	150val_150	1
> [junit] < 2	18val_18	1
> [junit] < 3	177val_177	1
> [junit] < 2	12val_12	1
> [junit] < 2	11val_11	1
> [junit] —
> [junit] > 3	120val_120	1
> [junit] > 3	192val_192	1
> [junit] > 3	119val_119	1
> [junit] > 3	187val_187	1
> [junit] > 3	176val_176	1
> [junit] > 3	199val_199	1
> [junit] > 3	118val_118	1
> [junit] Exception: Client execution results failed with error code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more
logs.
> [junit] junit.framework.AssertionFailedError: Client execution results failed with error
code = 1
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more
logs.
> [junit] at junit.framework.Assert.fail(Assert.java:50)
> [junit] at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ba_table_udfs(TestCliDriver.java:129)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at junit.framework.TestCase.runTest(TestCase.java:168)
> [junit] at junit.framework.TestCase.runBare(TestCase.java:134)
> [junit] at junit.framework.TestResult$1.protect(TestResult.java:110)
> [junit] at junit.framework.TestResult.runProtected(TestResult.java:128)
> [junit] at junit.framework.TestResult.run(TestResult.java:113)
> [junit] at junit.framework.TestCase.run(TestCase.java:124)
> [junit] at junit.framework.TestSuite.runTest(TestSuite.java:243)
> [junit] at junit.framework.TestSuite.run(TestSuite.java:238)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:420)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:911)
> [junit] at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:768)
> [junit] See build/ql/tmp/hive.log, or try "ant test ... -Dtest.silent=false" to get more
logs.)
> [junit] Cleaning up TestCliDriver
> [junit] Tests run: 2, Failures: 1, Errors: 0, Time elapsed: 10.751 sec
> [junit] Test org.apache.hadoop.hive.cli.TestCliDriver FAILED
> [for] /home/cloudera/Code/hive/ql/build.xml: The following error occurred while executing
this line:
> [for] /home/cloudera/Code/hive/build.xml:328: The following error occurred while executing
this line:
> [for] /home/cloudera/Code/hive/build-common.xml:453: Tests failed!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message