hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Hanson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-4685) query using LIKE does not vectorize
Date Fri, 07 Jun 2013 18:22:21 GMT

    [ https://issues.apache.org/jira/browse/HIVE-4685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13678256#comment-13678256
] 

Eric Hanson commented on HIVE-4685:
-----------------------------------

4599 appears to take out the printed messages saying whether or not the vectorization path
is being used. Can we put those back? Those are important for the user to know.
                
> query using LIKE does not vectorize
> -----------------------------------
>
>                 Key: HIVE-4685
>                 URL: https://issues.apache.org/jira/browse/HIVE-4685
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Query Processor
>    Affects Versions: vectorization-branch
>            Reporter: Eric Hanson
>
> The query 
>   select count(ddate) from factsqlengineam_vec_orc where ddate like "2013%";
> Starts up but does not run in vectorization mode. Then during non-vectorized execution
it crashes.
> Expected result:
> Query runs vectorized and runs successfully.
> Actual result:
> hive> select count(ddate) from factsqlengineam_vec_orc where ddate like "2013%";
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
>   set mapred.reduce.tasks=<number>
> Validating if vectorized execution is applicable
> Cannot vectorize the plan: org.apache.hadoop.hive.ql.metadata.HiveException: Udf: GenericUDFBridge,
is not supported
> java.lang.InstantiationException: org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator
> Continuing ...
> java.lang.Exception: XMLEncoder: discarding statement ArrayList.add(VectorGroupByOperator);
> Continuing ...
> Starting Job = job_201306061504_0041, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201306061504_0041
> Kill Command = c:\Hadoop\hadoop-1.1.0-SNAPSHOT\bin\hadoop.cmd job  -kill job_201306061504_0041
> Hadoop job information for Stage-1: number of mappers: 8; number of reducers: 1
> 2013-06-07 10:41:31,544 Stage-1 map = 0%,  reduce = 0%
> 2013-06-07 10:42:01,677 Stage-1 map = 100%,  reduce = 100%
> Ended Job = job_201306061504_0041 with errors
> Error during job, obtaining debugging information...
> Job Tracking URL: http://localhost:50030/jobdetails.jsp?jobid=job_201306061504_0041
> Examining task ID: task_201306061504_0041_m_000009 (and more) from job job_201306061504_0041
> Examining task ID: task_201306061504_0041_m_000002 (and more) from job job_201306061504_0041
> Examining task ID: task_201306061504_0041_m_000000 (and more) from job job_201306061504_0041
> Examining task ID: task_201306061504_0041_m_000004 (and more) from job job_201306061504_0041
> Task with the most failures(4):
> -----
> Task ID:
>   task_201306061504_0041_m_000006
> URL:
>   http://localhost:50030/taskdetails.jsp?jobid=job_201306061504_0041&tipid=task_201306061504_0041_m_000006
> -----
> Diagnostic Messages for this Task:
> java.lang.RuntimeException: Error in configuring object
>         at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>         at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>         at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:271)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1135)
>         at org.apache.hadoop.mapred.Child.main(Child.java:265)
> Caused by: java.lang.reflect.InvocationTargetException
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
>         ... 9 more
> Caused by: java.lang.RuntimeException: Error in configuring object
>         at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>         at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>         at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>         at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
>         ... 14 more
> Caused by: java.lang.reflect.InvocationTargetException
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
>         ... 17 more
> Caused by: java.lang.RuntimeException: Map operator initialization failed
>         at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
>         ... 22 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive internal error: cannot
find parent in the child operator!
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:360)
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
>         at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
>         at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:66)
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
>         at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
>         at org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:82)
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
>         at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
>         at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:187)
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
>         at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:543)
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
>         at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:100)
>         ... 22 more
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> MapReduce Jobs Launched:
> Job 0: Map: 8  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
> Total MapReduce CPU Time Spent: 0 msec
> hive>

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message